Uploaded image for project: 'Hippo CMS'
  1. Hippo CMS
  2. CMS-4588

Deadlock detected between CachingHierarchyManager.java and HippoLocalItemStateManager.java

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Top
    • Resolution: Fixed
    • r2.12.18
    • None
    • repository-core
    • None
    • clustered environment with 6 site servers and embedded repository

    Description

      We get a deadlock in one of our 6 site servers during normal load. The following contains the deadlock stacktrace and the complete stack trace of all threads.

                                                                                      • DEADLOCK stacktrace ***********************************************************

      Java-level deadlocks have been detected

      This means that some threads are blocked waiting to enter a synchronization block or
      waiting to reenter a synchronization block after an Object.wait() call, where each thread
      owns one monitor while trying to obtain another monitor already held by another thread.

      Deadlock:

      TP-Processor10 is waiting to lock org.hippoecm.repository.jackrabbit.HippoLocalItemStateManager@7b717490 which is held by TP-Processor18
      TP-Processor18 is waiting to lock java.lang.Object@13b3e073 which is held by TP-Processor10

      Deadlock:

      TP-Processor18 is waiting to lock java.lang.Object@13b3e073 which is held by TP-Processor10
      TP-Processor10 is waiting to lock org.hippoecm.repository.jackrabbit.HippoLocalItemStateManager@7b717490 which is held by TP-Processor18

      Thread stacks

      TP-Processor10 [BLOCKED; waiting to lock org.hippoecm.repository.jackrabbit.HippoLocalItemStateManager@7b717490]
      org.hippoecm.repository.jackrabbit.HippoLocalItemStateManager.edit(HippoLocalItemStateManager.java)
      org.hippoecm.repository.jackrabbit.HippoLocalItemStateManager.getItemState(HippoLocalItemStateManager.java:257)
      org.apache.jackrabbit.core.state.SessionItemStateManager.getItemState(SessionItemStateManager.java:182)
      org.apache.jackrabbit.core.HierarchyManagerImpl.getItemState(HierarchyManagerImpl.java:150)
      org.apache.jackrabbit.core.CachingHierarchyManager.nodeRemoved(CachingHierarchyManager.java:726)
      org.apache.jackrabbit.core.CachingHierarchyManager.nodeRemoved(CachingHierarchyManager.java:455)
      org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyNodeRemoved(StateChangeDispatcher.java:207)
      org.apache.jackrabbit.core.state.SessionItemStateManager.nodeRemoved(SessionItemStateManager.java:968)
      org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyNodeRemoved(StateChangeDispatcher.java:207)
      org.apache.jackrabbit.core.state.LocalItemStateManager.nodeRemoved(LocalItemStateManager.java:553)
      org.apache.jackrabbit.core.state.NodeState.notifyNodeRemoved(NodeState.java:871)
      org.apache.jackrabbit.core.state.NodeState.removeChildNodeEntry(NodeState.java:403)
      org.hippoecm.repository.jackrabbit.HippoLocalItemStateManager$FilteredChangeLog.invalidate(HippoLocalItemStateManager.java:488)
      org.hippoecm.repository.jackrabbit.HippoLocalItemStateManager.update(HippoLocalItemStateManager.java:225)
      org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemStateManager.java:330)
      org.hippoecm.repository.jackrabbit.HippoLocalItemStateManager.update(HippoLocalItemStateManager.java:235)
      org.hippoecm.repository.jackrabbit.HippoLocalItemStateManager.refresh(HippoLocalItemStateManager.java:246)
      org.hippoecm.repository.jackrabbit.HippoSessionItemStateManager.disposeAllTransientItemStates(HippoSessionItemStateManager.java:66)
      org.apache.jackrabbit.core.SessionImpl.refresh(SessionImpl.java:870)
      org.hippoecm.repository.decorating.SessionDecorator.refresh(SessionDecorator.java:209)
      org.hippoecm.repository.decorating.checked.SessionDecorator.refresh(SessionDecorator.java:235)
      sun.reflect.GeneratedMethodAccessor164.invoke(unknown source)
      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      java.lang.reflect.Method.invoke(Method.java:597)
      org.hippoecm.hst.core.jcr.pool.util.ProxyFactory$ReflectionInvocation.proceed(ProxyFactory.java:190)
      org.hippoecm.hst.core.jcr.pool.PooledSessionDecoratorProxyFactoryImpl$PooledSessionInterceptor.intercept(PooledSessionDecoratorProxyFactoryImpl.java:108)
      org.hippoecm.hst.core.jcr.pool.util.ProxyFactory$InterceptorInvocationHandler.invokeImpl(ProxyFactory.java:103)
      org.hippoecm.hst.core.jcr.pool.util.ProxyFactory$AbstractInvocationHandler.invoke(ProxyFactory.java:121)
      $Proxy9.refresh(unknown source)
      nl.rijksoverheid.importer.repository.pooling.BasicPoolingRepositoryWithLogging$SessionFactory.passivateObject(BasicPoolingRepositoryWithLogging.java:970)
      org.apache.commons.pool.impl.GenericObjectPool.addObjectToPool(GenericObjectPool.java:1100)
      org.apache.commons.pool.impl.GenericObjectPool.returnObject(GenericObjectPool.java:1076)
      nl.rijksoverheid.importer.repository.pooling.BasicPoolingRepositoryWithLogging.returnSession(BasicPoolingRepositoryWithLogging.java:272)
      org.hippoecm.hst.core.jcr.pool.PooledSessionDecoratorProxyFactoryImpl$PooledSessionInterceptor.intercept(PooledSessionDecoratorProxyFactoryImpl.java:129)
      org.hippoecm.hst.core.jcr.pool.util.ProxyFactory$InterceptorInvocationHandler.invokeImpl(ProxyFactory.java:103)
      org.hippoecm.hst.core.jcr.pool.util.ProxyFactory$AbstractInvocationHandler.invoke(ProxyFactory.java:121)
      $Proxy9.logout(unknown source)
      org.hippoecm.hst.core.jcr.pool.PooledSessionResourceManagement.disposeAllResources(PooledSessionResourceManagement.java:103)
      org.hippoecm.hst.core.container.InitializationValve.invoke(InitializationValve.java:65)
      org.hippoecm.hst.core.container.HstSitePipeline$Invocation.invokeNext(HstSitePipeline.java:97)
      org.hippoecm.hst.core.container.HstSitePipeline.invokeValves(HstSitePipeline.java:66)
      org.hippoecm.hst.core.container.HstSitePipeline.beforeInvoke(HstSitePipeline.java:53)
      org.hippoecm.hst.core.container.HstRequestProcessorImpl.processRequest(HstRequestProcessorImpl.java:68)
      org.hippoecm.hst.core.container.HstRequestProcessorImpl.processRequest(HstRequestProcessorImpl.java:35)
      org.hippoecm.hst.container.HstContainerServlet.invoke(HstContainerServlet.java:196)
      org.hippoecm.hst.container.HstContainerServlet.service(HstContainerServlet.java:209)
      javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:70)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:646)
      org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:436)
      org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:374)
      org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:302)
      org.hippoecm.hst.container.HstVirtualHostsFilter.doFilter(HstVirtualHostsFilter.java:129)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      nl.rijksoverheid.site.filters.LocaleFilter.doFilterInternal(LocaleFilter.java:109)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      nl.rijksoverheid.site.filters.DisableUrlSessionFilter.doFilterInternal(DisableUrlSessionFilter.java:95)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:96)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      nl.rijksoverheid.site.tags.contribution.ContributionFilter.doFilterInternal(ContributionFilter.java:29)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
      org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
      org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
      org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
      org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
      org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
      org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)
      org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
      org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:769)
      org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:698)
      org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:891)
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
      java.lang.Thread.run(Thread.java:619)

      TP-Processor18 [BLOCKED; waiting to lock java.lang.Object@13b3e073]
      org.apache.jackrabbit.core.CachingHierarchyManager.nodeAdded(CachingHierarchyManager.java:362)
      org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyNodeAdded(StateChangeDispatcher.java:159)
      org.apache.jackrabbit.core.state.SessionItemStateManager.nodeAdded(SessionItemStateManager.java:932)
      org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyNodeAdded(StateChangeDispatcher.java:159)
      org.apache.jackrabbit.core.state.LocalItemStateManager.nodeAdded(LocalItemStateManager.java:523)
      org.apache.jackrabbit.core.state.NodeState.notifyNodeAdded(NodeState.java:852)
      org.apache.jackrabbit.core.state.NodeState.addChildNodeEntry(NodeState.java:351)
      org.hippoecm.repository.jackrabbit.ViewVirtualProvider.populateChildren(ViewVirtualProvider.java:240)
      org.hippoecm.repository.jackrabbit.MirrorVirtualProvider.populate(MirrorVirtualProvider.java:131)
      org.hippoecm.repository.jackrabbit.HippoNodeId.populate(HippoNodeId.java:51)
      org.hippoecm.repository.jackrabbit.HippoLocalItemStateManager.getNodeState(HippoLocalItemStateManager.java:337)
      org.apache.jackrabbit.core.state.LocalItemStateManager.getItemState(LocalItemStateManager.java:176)
      org.apache.jackrabbit.core.state.ForkedXAItemStateManager.getItemState(ForkedXAItemStateManager.java:266)
      org.hippoecm.repository.jackrabbit.HippoLocalItemStateManager.getItemState(HippoLocalItemStateManager.java:252)
      org.apache.jackrabbit.core.state.SessionItemStateManager.getItemState(SessionItemStateManager.java:182)
      org.apache.jackrabbit.core.ItemManager.getItemData(ItemManager.java:282)
      org.apache.jackrabbit.core.ItemManager.getItemData(ItemManager.java:249)
      org.apache.jackrabbit.core.ItemManager.getNode(ItemManager.java:513)
      org.apache.jackrabbit.core.NodeImpl.getNode(NodeImpl.java:2602)
      org.hippoecm.repository.decorating.NodeDecorator.getNode(NodeDecorator.java:226)
      org.hippoecm.repository.decorating.checked.NodeDecorator.getNode(NodeDecorator.java:290)
      nl.rijksoverheid.spring.hst.linking.RijksoverheidInlineResourceContainer.resolveToResourceNode(RijksoverheidInlineResourceContainer.java:87)
      nl.rijksoverheid.site.servlet.BinariesServlet.lookUpResource(BinariesServlet.java:317)
      nl.rijksoverheid.site.servlet.BinariesServlet.doGet(BinariesServlet.java:195)
      javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
      javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
      org.springframework.web.servlet.mvc.ServletWrappingController.handleRequestInternal(ServletWrappingController.java:158)
      org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
      org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
      org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875)
      org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:807)
      org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
      org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:501)
      javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
      javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      org.hippoecm.hst.container.HstVirtualHostsFilter.doFilter(HstVirtualHostsFilter.java:109)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      nl.rijksoverheid.site.filters.LocaleFilter.doFilterInternal(LocaleFilter.java:109)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      nl.rijksoverheid.site.filters.DisableUrlSessionFilter.doFilterInternal(DisableUrlSessionFilter.java:95)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:96)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      nl.rijksoverheid.site.tags.contribution.ContributionFilter.doFilterInternal(ContributionFilter.java:29)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
      org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
      org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
      org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
      org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
      org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
      org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)
      org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
      org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:769)
      org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:698)
      org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:891)
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
      java.lang.Thread.run(Thread.java:619)

                                                                                      • Alle threads ***********************************************************

      Stacks at 11:16:19 AM (uptime 1 day 19:44:17)

      ClusterNode-s1710 [BLOCKED] CPU time: 6:35
      org.apache.jackrabbit.core.CachingHierarchyManager.nodeModified(NodeState)
      org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyNodeModified(NodeState)
      org.apache.jackrabbit.core.state.SessionItemStateManager.nodeModified(NodeState)
      org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyNodeModified(NodeState)
      org.apache.jackrabbit.core.state.LocalItemStateManager.stateModified(ItemState)
      org.apache.jackrabbit.core.state.ForkedXAItemStateManager.stateModified(ItemState)
      org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyStateModified(ItemState)
      org.apache.jackrabbit.core.state.SharedItemStateManager.stateModified(ItemState)
      org.apache.jackrabbit.core.state.ItemState.notifyStateUpdated()
      org.apache.jackrabbit.core.state.ChangeLog.persisted()
      org.apache.jackrabbit.core.state.SharedItemStateManager.doExternalUpdate(ChangeLog)
      org.apache.jackrabbit.core.state.SharedItemStateManager.externalUpdate(ChangeLog, EventStateCollection)
      org.hippoecm.repository.jackrabbit.HippoSharedItemStateManager.externalUpdate(ChangeLog, EventStateCollection)
      org.apache.jackrabbit.core.RepositoryImpl$WorkspaceInfo.externalUpdate(ChangeLog, List)
      org.apache.jackrabbit.core.cluster.ClusterNode.process(ChangeLogRecord)
      org.apache.jackrabbit.core.cluster.ChangeLogRecord.process(ClusterRecordProcessor)
      org.apache.jackrabbit.core.cluster.ClusterNode.consume(Record)
      org.apache.jackrabbit.core.journal.AbstractJournal.doSync(long)
      org.apache.jackrabbit.core.journal.AbstractJournal.sync()
      org.apache.jackrabbit.core.cluster.ClusterNode.sync()
      org.apache.jackrabbit.core.cluster.ClusterNode.run()
      java.lang.Thread.run()

      ContainerBackgroundProcessor[StandardEngine[Admin]] [SLEEPING] CPU time: 0:05
      java.lang.Thread.sleep(long)
      org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run()
      java.lang.Thread.run()

      ContainerBackgroundProcessor[StandardEngine[Catalina]] [SLEEPING] CPU time: 6:11
      java.lang.Thread.sleep(long)
      org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run()
      java.lang.Thread.run()

      EventListenersContainerSessionChecker [WAITING] CPU time: 0:00
      java.lang.Object.wait(long)
      org.hippoecm.hst.core.jcr.EventListenersContainerImpl$EventListenersContainerSessionChecker.run()

      EventListenersContainerSessionChecker [WAITING] CPU time: 0:00
      java.lang.Object.wait(long)
      org.hippoecm.hst.core.jcr.EventListenersContainerImpl$EventListenersContainerSessionChecker.run()

      GC Daemon [WAITING] CPU time: 0:00
      java.lang.Object.wait(long)
      sun.misc.GC$Daemon.run()

      HJCRQJS_QuartzSchedulerThread [SLEEPING] CPU time: 3:08
      java.lang.Thread.sleep(long)
      org.quartz.core.QuartzSchedulerThread.run()

      HJCRQJS_Worker-0 [WAITING] CPU time: 0:03
      java.lang.Object.wait(long)
      org.quartz.simpl.SimpleThreadPool.getNextRunnable()
      org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool)
      org.quartz.simpl.SimpleThreadPool$WorkerThread.run()

      http-10.2.16.10-8544-1 [WAITING] CPU time: 0:21
      java.lang.Object.wait(long)
      java.lang.Object.wait()
      org.apache.tomcat.util.net.JIoEndpoint$Worker.await()
      org.apache.tomcat.util.net.JIoEndpoint$Worker.run()
      java.lang.Thread.run()

      http-10.2.16.10-8544-Acceptor-0 [RUNNABLE, IN_NATIVE] CPU time: 0:00
      java.net.PlainSocketImpl.socketAccept(SocketImpl)
      java.net.PlainSocketImpl.accept(SocketImpl)
      java.net.ServerSocket.implAccept(Socket)
      java.net.ServerSocket.accept()
      org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(ServerSocket)
      org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run()
      java.lang.Thread.run()

      http-127.0.0.1-8180-Acceptor-0 [RUNNABLE, IN_NATIVE] CPU time: 0:00
      java.net.PlainSocketImpl.socketAccept(SocketImpl)
      java.net.PlainSocketImpl.accept(SocketImpl)
      java.net.ServerSocket.implAccept(Socket)
      java.net.ServerSocket.accept()
      org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(ServerSocket)
      org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run()
      java.lang.Thread.run()

      IndexMerger [WAITING] CPU time: 0:34
      java.lang.Object.wait(long)
      java.lang.Object.wait()
      org.apache.commons.collections.buffer.BlockingBuffer.remove()
      org.apache.jackrabbit.core.query.lucene.IndexMerger.run()

      main [RUNNABLE, IN_NATIVE] CPU time: 1:16
      java.net.PlainSocketImpl.socketAccept(SocketImpl)
      java.net.PlainSocketImpl.accept(SocketImpl)
      java.net.ServerSocket.implAccept(Socket)
      java.net.ServerSocket.accept()
      org.apache.catalina.core.StandardServer.await()
      org.apache.catalina.startup.Catalina.await()
      org.apache.catalina.startup.Catalina.start()
      sun.reflect.NativeMethodAccessorImpl.invoke0(Method, Object, Object[])
      sun.reflect.NativeMethodAccessorImpl.invoke(Object, Object[])
      sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[])
      java.lang.reflect.Method.invoke(Object, Object[])
      org.apache.catalina.startup.Bootstrap.start()
      org.apache.catalina.startup.Bootstrap.main(String[])

      ObservationManager [WAITING] CPU time: 0:02
      java.lang.Object.wait(long)
      java.lang.Object.wait()
      org.apache.commons.collections.buffer.BlockingBuffer.remove()
      org.apache.jackrabbit.core.observation.ObservationDispatcher.run()
      java.lang.Thread.run()

      org.springframework.scheduling.quartz.SchedulerFactoryBean#0_QuartzSchedulerThread [SLEEPING] CPU time: 2:44
      java.lang.Thread.sleep(long)
      org.quartz.core.QuartzSchedulerThread.run()

      org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-0 [WAITING] CPU time: 0:03
      java.lang.Object.wait(long)
      org.quartz.simpl.SimpleThreadPool.getNextRunnable()
      org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool)
      org.quartz.simpl.SimpleThreadPool$WorkerThread.run()

      org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-1 [WAITING] CPU time: 0:03
      java.lang.Object.wait(long)
      org.quartz.simpl.SimpleThreadPool.getNextRunnable()
      org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool)
      org.quartz.simpl.SimpleThreadPool$WorkerThread.run()

      org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-2 [WAITING] CPU time: 0:03
      java.lang.Object.wait(long)
      org.quartz.simpl.SimpleThreadPool.getNextRunnable()
      org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool)
      org.quartz.simpl.SimpleThreadPool$WorkerThread.run()

      org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-3 [WAITING] CPU time: 0:03
      java.lang.Object.wait(long)
      org.quartz.simpl.SimpleThreadPool.getNextRunnable()
      org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool)
      org.quartz.simpl.SimpleThreadPool$WorkerThread.run()

      org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-4 [WAITING] CPU time: 0:03
      java.lang.Object.wait(long)
      org.quartz.simpl.SimpleThreadPool.getNextRunnable()
      org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool)
      org.quartz.simpl.SimpleThreadPool$WorkerThread.run()

      org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-5 [WAITING] CPU time: 0:03
      java.lang.Object.wait(long)
      org.quartz.simpl.SimpleThreadPool.getNextRunnable()
      org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool)
      org.quartz.simpl.SimpleThreadPool$WorkerThread.run()

      org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-6 [WAITING] CPU time: 0:03
      java.lang.Object.wait(long)
      org.quartz.simpl.SimpleThreadPool.getNextRunnable()
      org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool)
      org.quartz.simpl.SimpleThreadPool$WorkerThread.run()

      org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-7 [WAITING] CPU time: 0:03
      java.lang.Object.wait(long)
      org.quartz.simpl.SimpleThreadPool.getNextRunnable()
      org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool)
      org.quartz.simpl.SimpleThreadPool$WorkerThread.run()

      org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-8 [WAITING] CPU time: 0:03
      java.lang.Object.wait(long)
      org.quartz.simpl.SimpleThreadPool.getNextRunnable()
      org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool)
      org.quartz.simpl.SimpleThreadPool$WorkerThread.run()

      org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-9 [WAITING] CPU time: 0:03
      java.lang.Object.wait(long)
      org.quartz.simpl.SimpleThreadPool.getNextRunnable()
      org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool)
      org.quartz.simpl.SimpleThreadPool$WorkerThread.run()

      RMI Reaper [WAITING] CPU time: 0:00
      java.lang.Object.wait(long)
      java.lang.ref.ReferenceQueue.remove(long)
      java.lang.ref.ReferenceQueue.remove()
      sun.rmi.transport.ObjectTable$Reaper.run()
      java.lang.Thread.run()

      RMI RenewClean-[10.2.16.10:45065] [WAITING] CPU time: 0:15
      java.lang.Object.wait(long)
      java.lang.ref.ReferenceQueue.remove(long)
      sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run()
      java.lang.Thread.run()

      RMI Scheduler(0) [WAITING] CPU time: 0:09
      sun.misc.Unsafe.park(boolean, long)
      java.util.concurrent.locks.LockSupport.parkNanos(Object, long)
      java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long)
      java.util.concurrent.DelayQueue.take()
      java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take()
      java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take()
      java.util.concurrent.ThreadPoolExecutor.getTask()
      java.util.concurrent.ThreadPoolExecutor$Worker.run()
      java.lang.Thread.run()

      RMI TCP Accept-0 [RUNNABLE, IN_NATIVE] CPU time: 0:00
      java.net.PlainSocketImpl.socketAccept(SocketImpl)
      java.net.PlainSocketImpl.accept(SocketImpl)
      java.net.ServerSocket.implAccept(Socket)
      java.net.ServerSocket.accept()
      sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop()
      sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run()
      java.lang.Thread.run()

      RMI TCP Accept-1099 [RUNNABLE, IN_NATIVE] CPU time: 0:00
      java.net.PlainSocketImpl.socketAccept(SocketImpl)
      java.net.PlainSocketImpl.accept(SocketImpl)
      java.net.ServerSocket.implAccept(Socket)
      java.net.ServerSocket.accept()
      sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop()
      sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run()
      java.lang.Thread.run()

      RMI TCP Connection(5769)-10.2.16.10 [RUNNABLE, IN_NATIVE] CPU time: 0:02
      java.net.SocketInputStream.socketRead0(FileDescriptor, byte[], int, int, int)
      java.net.SocketInputStream.read(byte[], int, int)
      java.io.BufferedInputStream.fill()
      java.io.BufferedInputStream.read()
      java.io.FilterInputStream.read()
      sun.rmi.transport.tcp.TCPTransport.handleMessages(Connection, boolean)
      sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0()
      sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run()
      java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Runnable)
      java.util.concurrent.ThreadPoolExecutor$Worker.run()
      java.lang.Thread.run()

      RMI TCP Connection(5772)-127.0.0.1 [RUNNABLE, IN_NATIVE] CPU time: 0:00
      java.net.SocketInputStream.socketRead0(FileDescriptor, byte[], int, int, int)
      java.net.SocketInputStream.read(byte[], int, int)
      java.io.BufferedInputStream.fill()
      java.io.BufferedInputStream.read()
      java.io.FilterInputStream.read()
      sun.rmi.transport.tcp.TCPTransport.handleMessages(Connection, boolean)
      sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0()
      sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run()
      java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Runnable)
      java.util.concurrent.ThreadPoolExecutor$Worker.run()
      java.lang.Thread.run()

      RMI TCP Connection(idle) [WAITING] CPU time: 0:10
      sun.misc.Unsafe.park(boolean, long)
      java.util.concurrent.locks.LockSupport.parkNanos(Object, long)
      java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue$TransferStack$SNode, boolean, long)
      java.util.concurrent.SynchronousQueue$TransferStack.transfer(Object, boolean, long)
      java.util.concurrent.SynchronousQueue.poll(long, TimeUnit)
      java.util.concurrent.ThreadPoolExecutor.getTask()
      java.util.concurrent.ThreadPoolExecutor$Worker.run()
      java.lang.Thread.run()

      Thread-1 [SLEEPING] CPU time: 0:00
      java.lang.Thread.sleep(long)
      org.apache.log4j.helpers.FileWatchdog.run()

      Thread-2 [SLEEPING] CPU time: 0:05
      java.lang.Thread.sleep(long)
      org.hippoecm.repository.LoadInitializationModule.run()

      Thread-54 [RUNNABLE, IN_NATIVE] CPU time: 0:00
      java.net.PlainSocketImpl.socketAccept(SocketImpl)
      java.net.PlainSocketImpl.accept(SocketImpl)
      java.net.ServerSocket.implAccept(Socket)
      java.net.ServerSocket.accept()
      com.sun.jmx.remote.socket.SocketConnectionServer.accept()
      com.sun.jmx.remote.generic.SynchroMessageConnectionServerImpl.accept()
      javax.management.remote.generic.GenericConnectorServer$Receiver.run()

      Timer-0 [WAITING] CPU time: 0:22
      java.lang.Object.wait(long)
      java.util.TimerThread.mainLoop()
      java.util.TimerThread.run()

      Timer-1 [WAITING] CPU time: 0:53
      java.lang.Object.wait(long)
      java.util.TimerThread.mainLoop()
      java.util.TimerThread.run()

      Timer-2 [WAITING] CPU time: 0:00
      java.lang.Object.wait(long)
      java.lang.Object.wait()
      java.util.TimerThread.mainLoop()
      java.util.TimerThread.run()

      TP-Monitor [WAITING] CPU time: 0:00
      java.lang.Object.wait(long)
      org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable.run()
      java.lang.Thread.run()

      TP-Processor1 [RUNNABLE, IN_NATIVE] CPU time: 18:04
      java.net.SocketInputStream.socketRead0(FileDescriptor, byte[], int, int, int)
      java.net.SocketInputStream.read(byte[], int, int)
      java.io.BufferedInputStream.fill()
      java.io.BufferedInputStream.read1(byte[], int, int)
      java.io.BufferedInputStream.read(byte[], int, int)
      org.apache.jk.common.ChannelSocket.read(MsgContext, byte[], int, int)
      org.apache.jk.common.ChannelSocket.receive(Msg, MsgContext)
      org.apache.jk.common.ChannelSocket.processConnection(MsgContext)
      org.apache.jk.common.ChannelSocket$SocketConnection.runIt(Object[])
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()
      java.lang.Thread.run()

      TP-Processor10 [BLOCKED] CPU time: 35:22
      org.hippoecm.repository.jackrabbit.HippoLocalItemStateManager.edit()
      org.hippoecm.repository.jackrabbit.HippoLocalItemStateManager.getItemState(ItemId)
      org.apache.jackrabbit.core.state.SessionItemStateManager.getItemState(ItemId)
      org.apache.jackrabbit.core.HierarchyManagerImpl.getItemState(ItemId)
      org.apache.jackrabbit.core.CachingHierarchyManager.nodeRemoved(NodeState, Path, NodeId)
      org.apache.jackrabbit.core.CachingHierarchyManager.nodeRemoved(NodeState, Name, int, NodeId)
      org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyNodeRemoved(NodeState, Name, int, NodeId)
      org.apache.jackrabbit.core.state.SessionItemStateManager.nodeRemoved(NodeState, Name, int, NodeId)
      org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyNodeRemoved(NodeState, Name, int, NodeId)
      org.apache.jackrabbit.core.state.LocalItemStateManager.nodeRemoved(NodeState, Name, int, NodeId)
      org.apache.jackrabbit.core.state.NodeState.notifyNodeRemoved(ChildNodeEntry)
      org.apache.jackrabbit.core.state.NodeState.removeChildNodeEntry(NodeId)
      org.hippoecm.repository.jackrabbit.HippoLocalItemStateManager$FilteredChangeLog.invalidate()
      org.hippoecm.repository.jackrabbit.HippoLocalItemStateManager.update(ChangeLog)
      org.apache.jackrabbit.core.state.LocalItemStateManager.update()
      org.hippoecm.repository.jackrabbit.HippoLocalItemStateManager.update()
      org.hippoecm.repository.jackrabbit.HippoLocalItemStateManager.refresh()
      org.hippoecm.repository.jackrabbit.HippoSessionItemStateManager.disposeAllTransientItemStates()
      org.apache.jackrabbit.core.SessionImpl.refresh(boolean)
      org.hippoecm.repository.decorating.SessionDecorator.refresh(boolean)
      org.hippoecm.repository.decorating.checked.SessionDecorator.refresh(boolean)
      sun.reflect.GeneratedMethodAccessor164.invoke(Object, Object[])
      sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[])
      java.lang.reflect.Method.invoke(Object, Object[])
      org.hippoecm.hst.core.jcr.pool.util.ProxyFactory$ReflectionInvocation.proceed()
      org.hippoecm.hst.core.jcr.pool.PooledSessionDecoratorProxyFactoryImpl$PooledSessionInterceptor.intercept(Invocation)
      org.hippoecm.hst.core.jcr.pool.util.ProxyFactory$InterceptorInvocationHandler.invokeImpl(Object, Method, Object[])
      org.hippoecm.hst.core.jcr.pool.util.ProxyFactory$AbstractInvocationHandler.invoke(Object, Method, Object[])
      $Proxy9.refresh(boolean)
      nl.rijksoverheid.importer.repository.pooling.BasicPoolingRepositoryWithLogging$SessionFactory.passivateObject(Object)
      org.apache.commons.pool.impl.GenericObjectPool.addObjectToPool(Object, boolean)
      org.apache.commons.pool.impl.GenericObjectPool.returnObject(Object)
      nl.rijksoverheid.importer.repository.pooling.BasicPoolingRepositoryWithLogging.returnSession(Session)
      org.hippoecm.hst.core.jcr.pool.PooledSessionDecoratorProxyFactoryImpl$PooledSessionInterceptor.intercept(Invocation)
      org.hippoecm.hst.core.jcr.pool.util.ProxyFactory$InterceptorInvocationHandler.invokeImpl(Object, Method, Object[])
      org.hippoecm.hst.core.jcr.pool.util.ProxyFactory$AbstractInvocationHandler.invoke(Object, Method, Object[])
      $Proxy9.logout()
      org.hippoecm.hst.core.jcr.pool.PooledSessionResourceManagement.disposeAllResources()
      org.hippoecm.hst.core.container.InitializationValve.invoke(ValveContext)
      org.hippoecm.hst.core.container.HstSitePipeline$Invocation.invokeNext()
      org.hippoecm.hst.core.container.HstSitePipeline.invokeValves(HstContainerConfig, HttpServletRequest, HttpServletResponse, Valve[])
      org.hippoecm.hst.core.container.HstSitePipeline.beforeInvoke(HstContainerConfig, HttpServletRequest, HttpServletResponse)
      org.hippoecm.hst.core.container.HstRequestProcessorImpl.processRequest(HstContainerConfig, HttpServletRequest, HttpServletResponse, String)
      org.hippoecm.hst.core.container.HstRequestProcessorImpl.processRequest(HstContainerConfig, HttpServletRequest, HttpServletResponse)
      org.hippoecm.hst.container.HstContainerServlet.invoke(HttpServletRequest, HttpServletResponse)
      org.hippoecm.hst.container.HstContainerServlet.service(HttpServletRequest, HttpServletResponse)
      javax.servlet.http.HttpServlet.service(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(ServletRequest, ServletResponse, FilterChain)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationDispatcher.invoke(ServletRequest, ServletResponse, ApplicationDispatcher$State)
      org.apache.catalina.core.ApplicationDispatcher.processRequest(ServletRequest, ServletResponse, ApplicationDispatcher$State)
      org.apache.catalina.core.ApplicationDispatcher.doForward(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationDispatcher.forward(ServletRequest, ServletResponse)
      org.hippoecm.hst.container.HstVirtualHostsFilter.doFilter(ServletRequest, ServletResponse, FilterChain)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse)
      nl.rijksoverheid.site.filters.LocaleFilter.doFilterInternal(HttpServletRequest, HttpServletResponse, FilterChain)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(ServletRequest, ServletResponse, FilterChain)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse)
      nl.rijksoverheid.site.filters.DisableUrlSessionFilter.doFilterInternal(HttpServletRequest, HttpServletResponse, FilterChain)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(ServletRequest, ServletResponse, FilterChain)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse)
      org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(HttpServletRequest, HttpServletResponse, FilterChain)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(ServletRequest, ServletResponse, FilterChain)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse)
      nl.rijksoverheid.site.tags.contribution.ContributionFilter.doFilterInternal(HttpServletRequest, HttpServletResponse, FilterChain)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(ServletRequest, ServletResponse, FilterChain)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.StandardWrapperValve.invoke(Request, Response)
      org.apache.catalina.core.StandardContextValve.invoke(Request, Response)
      org.apache.catalina.core.StandardHostValve.invoke(Request, Response)
      org.apache.catalina.valves.ErrorReportValve.invoke(Request, Response)
      org.apache.catalina.core.StandardEngineValve.invoke(Request, Response)
      org.apache.catalina.connector.CoyoteAdapter.service(Request, Response)
      org.apache.jk.server.JkCoyoteHandler.invoke(Msg, MsgContext)
      org.apache.jk.common.HandlerRequest.invoke(Msg, MsgContext)
      org.apache.jk.common.ChannelSocket.invoke(Msg, MsgContext)
      org.apache.jk.common.ChannelSocket.processConnection(MsgContext)
      org.apache.jk.common.ChannelSocket$SocketConnection.runIt(Object[])
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()
      java.lang.Thread.run()

      TP-Processor11 [RUNNABLE, IN_NATIVE] CPU time: 41:31
      java.net.SocketInputStream.socketRead0(FileDescriptor, byte[], int, int, int)
      java.net.SocketInputStream.read(byte[], int, int)
      java.io.BufferedInputStream.fill()
      java.io.BufferedInputStream.read1(byte[], int, int)
      java.io.BufferedInputStream.read(byte[], int, int)
      org.apache.jk.common.ChannelSocket.read(MsgContext, byte[], int, int)
      org.apache.jk.common.ChannelSocket.receive(Msg, MsgContext)
      org.apache.jk.common.ChannelSocket.processConnection(MsgContext)
      org.apache.jk.common.ChannelSocket$SocketConnection.runIt(Object[])
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()
      java.lang.Thread.run()

      TP-Processor12 [BLOCKED] CPU time: 45:10
      org.hippoecm.repository.jackrabbit.HippoLocalItemStateManager.edit()
      org.apache.jackrabbit.core.state.SessionItemStateManager.edit()
      org.hippoecm.repository.jackrabbit.HippoSessionItemStateManager.disposeAllTransientItemStates()
      org.apache.jackrabbit.core.SessionImpl.refresh(boolean)
      org.hippoecm.repository.decorating.SessionDecorator.refresh(boolean)
      org.hippoecm.repository.decorating.checked.SessionDecorator.refresh(boolean)
      sun.reflect.GeneratedMethodAccessor164.invoke(Object, Object[])
      sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[])
      java.lang.reflect.Method.invoke(Object, Object[])
      org.hippoecm.hst.core.jcr.pool.util.ProxyFactory$ReflectionInvocation.proceed()
      org.hippoecm.hst.core.jcr.pool.PooledSessionDecoratorProxyFactoryImpl$PooledSessionInterceptor.intercept(Invocation)
      org.hippoecm.hst.core.jcr.pool.util.ProxyFactory$InterceptorInvocationHandler.invokeImpl(Object, Method, Object[])
      org.hippoecm.hst.core.jcr.pool.util.ProxyFactory$AbstractInvocationHandler.invoke(Object, Method, Object[])
      $Proxy9.refresh(boolean)
      nl.rijksoverheid.importer.repository.pooling.BasicPoolingRepositoryWithLogging$SessionFactory.passivateObject(Object)
      org.apache.commons.pool.impl.GenericObjectPool.addObjectToPool(Object, boolean)
      org.apache.commons.pool.impl.GenericObjectPool.returnObject(Object)
      nl.rijksoverheid.importer.repository.pooling.BasicPoolingRepositoryWithLogging.returnSession(Session)
      org.hippoecm.hst.core.jcr.pool.PooledSessionDecoratorProxyFactoryImpl$PooledSessionInterceptor.intercept(Invocation)
      org.hippoecm.hst.core.jcr.pool.util.ProxyFactory$InterceptorInvocationHandler.invokeImpl(Object, Method, Object[])
      org.hippoecm.hst.core.jcr.pool.util.ProxyFactory$AbstractInvocationHandler.invoke(Object, Method, Object[])
      $Proxy9.logout()
      org.hippoecm.hst.core.jcr.pool.PooledSessionResourceManagement.disposeAllResources()
      org.hippoecm.hst.core.container.InitializationValve.invoke(ValveContext)
      org.hippoecm.hst.core.container.HstSitePipeline$Invocation.invokeNext()
      org.hippoecm.hst.core.container.HstSitePipeline.invokeValves(HstContainerConfig, HttpServletRequest, HttpServletResponse, Valve[])
      org.hippoecm.hst.core.container.HstSitePipeline.beforeInvoke(HstContainerConfig, HttpServletRequest, HttpServletResponse)
      org.hippoecm.hst.core.container.HstRequestProcessorImpl.processRequest(HstContainerConfig, HttpServletRequest, HttpServletResponse, String)
      org.hippoecm.hst.core.container.HstRequestProcessorImpl.processRequest(HstContainerConfig, HttpServletRequest, HttpServletResponse)
      org.hippoecm.hst.container.HstContainerServlet.invoke(HttpServletRequest, HttpServletResponse)
      org.hippoecm.hst.container.HstContainerServlet.service(HttpServletRequest, HttpServletResponse)
      javax.servlet.http.HttpServlet.service(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(ServletRequest, ServletResponse, FilterChain)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationDispatcher.invoke(ServletRequest, ServletResponse, ApplicationDispatcher$State)
      org.apache.catalina.core.ApplicationDispatcher.processRequest(ServletRequest, ServletResponse, ApplicationDispatcher$State)
      org.apache.catalina.core.ApplicationDispatcher.doForward(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationDispatcher.forward(ServletRequest, ServletResponse)
      org.hippoecm.hst.container.HstVirtualHostsFilter.doFilter(ServletRequest, ServletResponse, FilterChain)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse)
      nl.rijksoverheid.site.filters.LocaleFilter.doFilterInternal(HttpServletRequest, HttpServletResponse, FilterChain)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(ServletRequest, ServletResponse, FilterChain)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse)
      nl.rijksoverheid.site.filters.DisableUrlSessionFilter.doFilterInternal(HttpServletRequest, HttpServletResponse, FilterChain)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(ServletRequest, ServletResponse, FilterChain)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse)
      org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(HttpServletRequest, HttpServletResponse, FilterChain)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(ServletRequest, ServletResponse, FilterChain)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse)
      nl.rijksoverheid.site.tags.contribution.ContributionFilter.doFilterInternal(HttpServletRequest, HttpServletResponse, FilterChain)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(ServletRequest, ServletResponse, FilterChain)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.StandardWrapperValve.invoke(Request, Response)
      org.apache.catalina.core.StandardContextValve.invoke(Request, Response)
      org.apache.catalina.core.StandardHostValve.invoke(Request, Response)
      org.apache.catalina.valves.ErrorReportValve.invoke(Request, Response)
      org.apache.catalina.core.StandardEngineValve.invoke(Request, Response)
      org.apache.catalina.connector.CoyoteAdapter.service(Request, Response)
      org.apache.jk.server.JkCoyoteHandler.invoke(Msg, MsgContext)
      org.apache.jk.common.HandlerRequest.invoke(Msg, MsgContext)
      org.apache.jk.common.ChannelSocket.invoke(Msg, MsgContext)
      org.apache.jk.common.ChannelSocket.processConnection(MsgContext)
      org.apache.jk.common.ChannelSocket$SocketConnection.runIt(Object[])
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()
      java.lang.Thread.run()

      TP-Processor13 [RUNNABLE, IN_NATIVE] CPU time: 20:08
      java.net.SocketInputStream.socketRead0(FileDescriptor, byte[], int, int, int)
      java.net.SocketInputStream.read(byte[], int, int)
      java.io.BufferedInputStream.fill()
      java.io.BufferedInputStream.read1(byte[], int, int)
      java.io.BufferedInputStream.read(byte[], int, int)
      org.apache.jk.common.ChannelSocket.read(MsgContext, byte[], int, int)
      org.apache.jk.common.ChannelSocket.receive(Msg, MsgContext)
      org.apache.jk.common.ChannelSocket.processConnection(MsgContext)
      org.apache.jk.common.ChannelSocket$SocketConnection.runIt(Object[])
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()
      java.lang.Thread.run()

      TP-Processor14 [RUNNABLE, IN_NATIVE] CPU time: 39:10
      java.net.SocketInputStream.socketRead0(FileDescriptor, byte[], int, int, int)
      java.net.SocketInputStream.read(byte[], int, int)
      java.io.BufferedInputStream.fill()
      java.io.BufferedInputStream.read1(byte[], int, int)
      java.io.BufferedInputStream.read(byte[], int, int)
      org.apache.jk.common.ChannelSocket.read(MsgContext, byte[], int, int)
      org.apache.jk.common.ChannelSocket.receive(Msg, MsgContext)
      org.apache.jk.common.ChannelSocket.processConnection(MsgContext)
      org.apache.jk.common.ChannelSocket$SocketConnection.runIt(Object[])
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()
      java.lang.Thread.run()

      TP-Processor15 [WAITING] CPU time: 33:44
      java.lang.Object.wait(long)
      java.lang.Object.wait()
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()
      java.lang.Thread.run()

      TP-Processor16 [WAITING] CPU time: 0:00
      java.lang.Object.wait(long)
      java.lang.Object.wait()
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()
      java.lang.Thread.run()

      TP-Processor17 [WAITING] CPU time: 0:00
      java.lang.Object.wait(long)
      java.lang.Object.wait()
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()
      java.lang.Thread.run()

      TP-Processor18 [BLOCKED] CPU time: 25:17
      org.apache.jackrabbit.core.CachingHierarchyManager.nodeAdded(NodeState, Name, int, NodeId)
      org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyNodeAdded(NodeState, Name, int, NodeId)
      org.apache.jackrabbit.core.state.SessionItemStateManager.nodeAdded(NodeState, Name, int, NodeId)
      org.apache.jackrabbit.core.state.StateChangeDispatcher.notifyNodeAdded(NodeState, Name, int, NodeId)
      org.apache.jackrabbit.core.state.LocalItemStateManager.nodeAdded(NodeState, Name, int, NodeId)
      org.apache.jackrabbit.core.state.NodeState.notifyNodeAdded(ChildNodeEntry)
      org.apache.jackrabbit.core.state.NodeState.addChildNodeEntry(Name, NodeId)
      org.hippoecm.repository.jackrabbit.ViewVirtualProvider.populateChildren(NodeId, NodeState, NodeState)
      org.hippoecm.repository.jackrabbit.MirrorVirtualProvider.populate(HippoNodeId, NodeId)
      org.hippoecm.repository.jackrabbit.HippoNodeId.populate()
      org.hippoecm.repository.jackrabbit.HippoLocalItemStateManager.getNodeState(NodeId)
      org.apache.jackrabbit.core.state.LocalItemStateManager.getItemState(ItemId)
      org.apache.jackrabbit.core.state.ForkedXAItemStateManager.getItemState(ItemId)
      org.hippoecm.repository.jackrabbit.HippoLocalItemStateManager.getItemState(ItemId)
      org.apache.jackrabbit.core.state.SessionItemStateManager.getItemState(ItemId)
      org.apache.jackrabbit.core.ItemManager.getItemData(ItemId, Path, boolean)
      org.apache.jackrabbit.core.ItemManager.getItemData(ItemId)
      org.apache.jackrabbit.core.ItemManager.getNode(NodeId, NodeId)
      org.apache.jackrabbit.core.NodeImpl.getNode(String)
      org.hippoecm.repository.decorating.NodeDecorator.getNode(String)
      org.hippoecm.repository.decorating.checked.NodeDecorator.getNode(String)
      nl.rijksoverheid.spring.hst.linking.RijksoverheidInlineResourceContainer.resolveToResourceNode(Session, String)
      nl.rijksoverheid.site.servlet.BinariesServlet.lookUpResource(Session, String)
      nl.rijksoverheid.site.servlet.BinariesServlet.doGet(HttpServletRequest, HttpServletResponse)
      javax.servlet.http.HttpServlet.service(HttpServletRequest, HttpServletResponse)
      javax.servlet.http.HttpServlet.service(ServletRequest, ServletResponse)
      org.springframework.web.servlet.mvc.ServletWrappingController.handleRequestInternal(HttpServletRequest, HttpServletResponse)
      org.springframework.web.servlet.mvc.AbstractController.handleRequest(HttpServletRequest, HttpServletResponse)
      org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(HttpServletRequest, HttpServletResponse, Object)
      org.springframework.web.servlet.DispatcherServlet.doDispatch(HttpServletRequest, HttpServletResponse)
      org.springframework.web.servlet.DispatcherServlet.doService(HttpServletRequest, HttpServletResponse)
      org.springframework.web.servlet.FrameworkServlet.processRequest(HttpServletRequest, HttpServletResponse)
      org.springframework.web.servlet.FrameworkServlet.doGet(HttpServletRequest, HttpServletResponse)
      javax.servlet.http.HttpServlet.service(HttpServletRequest, HttpServletResponse)
      javax.servlet.http.HttpServlet.service(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse)
      org.hippoecm.hst.container.HstVirtualHostsFilter.doFilter(ServletRequest, ServletResponse, FilterChain)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse)
      nl.rijksoverheid.site.filters.LocaleFilter.doFilterInternal(HttpServletRequest, HttpServletResponse, FilterChain)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(ServletRequest, ServletResponse, FilterChain)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse)
      nl.rijksoverheid.site.filters.DisableUrlSessionFilter.doFilterInternal(HttpServletRequest, HttpServletResponse, FilterChain)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(ServletRequest, ServletResponse, FilterChain)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse)
      org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(HttpServletRequest, HttpServletResponse, FilterChain)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(ServletRequest, ServletResponse, FilterChain)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse)
      nl.rijksoverheid.site.tags.contribution.ContributionFilter.doFilterInternal(HttpServletRequest, HttpServletResponse, FilterChain)
      org.springframework.web.filter.OncePerRequestFilter.doFilter(ServletRequest, ServletResponse, FilterChain)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse)
      org.apache.catalina.core.StandardWrapperValve.invoke(Request, Response)
      org.apache.catalina.core.StandardContextValve.invoke(Request, Response)
      org.apache.catalina.core.StandardHostValve.invoke(Request, Response)
      org.apache.catalina.valves.ErrorReportValve.invoke(Request, Response)
      org.apache.catalina.core.StandardEngineValve.invoke(Request, Response)
      org.apache.catalina.connector.CoyoteAdapter.service(Request, Response)
      org.apache.jk.server.JkCoyoteHandler.invoke(Msg, MsgContext)
      org.apache.jk.common.HandlerRequest.invoke(Msg, MsgContext)
      org.apache.jk.common.ChannelSocket.invoke(Msg, MsgContext)
      org.apache.jk.common.ChannelSocket.processConnection(MsgContext)
      org.apache.jk.common.ChannelSocket$SocketConnection.runIt(Object[])
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()
      java.lang.Thread.run()

      TP-Processor19 [RUNNABLE, IN_NATIVE] CPU time: 28:39
      java.net.SocketInputStream.socketRead0(FileDescriptor, byte[], int, int, int)
      java.net.SocketInputStream.read(byte[], int, int)
      java.io.BufferedInputStream.fill()
      java.io.BufferedInputStream.read1(byte[], int, int)
      java.io.BufferedInputStream.read(byte[], int, int)
      org.apache.jk.common.ChannelSocket.read(MsgContext, byte[], int, int)
      org.apache.jk.common.ChannelSocket.receive(Msg, MsgContext)
      org.apache.jk.common.ChannelSocket.processConnection(MsgContext)
      org.apache.jk.common.ChannelSocket$SocketConnection.runIt(Object[])
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()
      java.lang.Thread.run()

      TP-Processor2 [WAITING] CPU time: 43:42
      java.lang.Object.wait(long)
      java.lang.Object.wait()
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()
      java.lang.Thread.run()

      TP-Processor20 [RUNNABLE, IN_NATIVE] CPU time: 28:09
      java.net.SocketInputStream.socketRead0(FileDescriptor, byte[], int, int, int)
      java.net.SocketInputStream.read(byte[], int, int)
      java.io.BufferedInputStream.fill()
      java.io.BufferedInputStream.read1(byte[], int, int)
      java.io.BufferedInputStream.read(byte[], int, int)
      org.apache.jk.common.ChannelSocket.read(MsgContext, byte[], int, int)
      org.apache.jk.common.ChannelSocket.receive(Msg, MsgContext)
      org.apache.jk.common.ChannelSocket.processConnection(MsgContext)
      org.apache.jk.common.ChannelSocket$SocketConnection.runIt(Object[])
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()
      java.lang.Thread.run()

      TP-Processor3 [RUNNABLE, IN_NATIVE] CPU time: 47:57
      java.net.SocketInputStream.socketRead0(FileDescriptor, byte[], int, int, int)
      java.net.SocketInputStream.read(byte[], int, int)
      java.io.BufferedInputStream.fill()
      java.io.BufferedInputStream.read1(byte[], int, int)
      java.io.BufferedInputStream.read(byte[], int, int)
      org.apache.jk.common.ChannelSocket.read(MsgContext, byte[], int, int)
      org.apache.jk.common.ChannelSocket.receive(Msg, MsgContext)
      org.apache.jk.common.ChannelSocket.processConnection(MsgContext)
      org.apache.jk.common.ChannelSocket$SocketConnection.runIt(Object[])
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()
      java.lang.Thread.run()

      TP-Processor4 [RUNNABLE, IN_NATIVE] CPU time: 48:31
      java.net.SocketInputStream.socketRead0(FileDescriptor, byte[], int, int, int)
      java.net.SocketInputStream.read(byte[], int, int)
      java.io.BufferedInputStream.fill()
      java.io.BufferedInputStream.read1(byte[], int, int)
      java.io.BufferedInputStream.read(byte[], int, int)
      org.apache.jk.common.ChannelSocket.read(MsgContext, byte[], int, int)
      org.apache.jk.common.ChannelSocket.receive(Msg, MsgContext)
      org.apache.jk.common.ChannelSocket.processConnection(MsgContext)
      org.apache.jk.common.ChannelSocket$SocketConnection.runIt(Object[])
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()
      java.lang.Thread.run()

      TP-Processor5 [RUNNABLE, IN_NATIVE] CPU time: 0:02
      java.net.PlainSocketImpl.socketAccept(SocketImpl)
      java.net.PlainSocketImpl.accept(SocketImpl)
      java.net.ServerSocket.implAccept(Socket)
      java.net.ServerSocket.accept()
      org.apache.jk.common.ChannelSocket.accept(MsgContext)
      org.apache.jk.common.ChannelSocket.acceptConnections()
      org.apache.jk.common.ChannelSocket$SocketAcceptor.runIt(Object[])
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()
      java.lang.Thread.run()

      TP-Processor6 [WAITING] CPU time: 51:05
      java.lang.Object.wait(long)
      java.lang.Object.wait()
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()
      java.lang.Thread.run()

      TP-Processor7 [WAITING] CPU time: 42:18
      java.lang.Object.wait(long)
      java.lang.Object.wait()
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()
      java.lang.Thread.run()

      TP-Processor8 [RUNNABLE, IN_NATIVE] CPU time: 30:52
      java.net.SocketInputStream.socketRead0(FileDescriptor, byte[], int, int, int)
      java.net.SocketInputStream.read(byte[], int, int)
      java.io.BufferedInputStream.fill()
      java.io.BufferedInputStream.read1(byte[], int, int)
      java.io.BufferedInputStream.read(byte[], int, int)
      org.apache.jk.common.ChannelSocket.read(MsgContext, byte[], int, int)
      org.apache.jk.common.ChannelSocket.receive(Msg, MsgContext)
      org.apache.jk.common.ChannelSocket.processConnection(MsgContext)
      org.apache.jk.common.ChannelSocket$SocketConnection.runIt(Object[])
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()
      java.lang.Thread.run()

      TP-Processor9 [RUNNABLE, IN_NATIVE] CPU time: 50:40
      java.net.SocketInputStream.socketRead0(FileDescriptor, byte[], int, int, int)
      java.net.SocketInputStream.read(byte[], int, int)
      java.io.BufferedInputStream.fill()
      java.io.BufferedInputStream.read1(byte[], int, int)
      java.io.BufferedInputStream.read(byte[], int, int)
      org.apache.jk.common.ChannelSocket.read(MsgContext, byte[], int, int)
      org.apache.jk.common.ChannelSocket.receive(Msg, MsgContext)
      org.apache.jk.common.ChannelSocket.processConnection(MsgContext)
      org.apache.jk.common.ChannelSocket$SocketConnection.runIt(Object[])
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run()
      java.lang.Thread.run()

      Transient File Reaper [WAITING] CPU time: 0:00
      java.lang.Object.wait(long)
      java.lang.ref.ReferenceQueue.remove(long)
      java.lang.ref.ReferenceQueue.remove()
      org.apache.jackrabbit.util.TransientFileFactory$ReaperThread.run()

      Attachments

        1. allthreads-deadlock.txt
          36 kB
        2. deadlock.txt
          16 kB
        3. logfiles2.zip
          828 kB
        4. patch-20101110-2.16.xx-r24927.patch
          8 kB
        5. patch-20101110-2.17.xx-r24926.patch
          8 kB

        Activity

          People

            berry van halderen Berry van Halderen (Inactive)
            jcoenradie Jettro Coenradie (Inactive)
            Votes:
            1 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: