Uploaded image for project: '[Read Only] - Hippo Site Toolkit 2'
  1. [Read Only] - Hippo Site Toolkit 2
  2. HSTTWO-4077

[Backport 3.2] In case of an exception BinaryServlet should remove lock

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • High
    • Resolution: Fixed
    • None
    • 3.2.7
    • None
    • None
    • Turing Sprint 159

    Description

      1.
      We are starting from the BinariesServlet.getPageFromCacheOrLoadPage() method. When a requested binary is not found in the cache, then binariesCache.getPageFromBlockingCache() returns null and acquires a write lock on the cache key for the requested binary. After this it reads the binary from the repository within the getBinaryPage() method and puts it in the cache with the binariesCache.putPage() method. This also removes the lock on the cache key.

      Now to the possible problem: If the getBinaryPage() method receives a RepositoryException, it will catch this and return an errorPage. Now in this case the errorPage is put into the cache and the lock on the original cache key will never be released.

      You can simulate this behaviour by throwing an RepositoryException on the first call (only on the first, not on following calls) in the BinariesServlet.createBinaryPage() method.

      2.
      The second problem comes into effect whenever there exists a write lock like described above. The method BinariesCache.getPageFromBlockingCache() returns null when trying to get a read lock and receiving a LockTimeoutException. Returning null is the same behaviour like in a cache miss case.

      This leads the BinariesServlet.getPageFromCacheOrLoadPage() method to trying to put the binary page in the cache. Within the binariesCache.putPage() method it will so try to get a write lock an wait forever (without a timeout).

      When the BinariesCache.getPageFromBlockingCache() method catches the LockTimeoutException it writes some log entry like "Possible threading issue or high load?". We can find these log entries on our live system and they are correlating with our thread and session pool problems.

      The hanging threads are showing the following stack in JavaMelody:
      sun.misc.Unsafe.park(Native Method)
      java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
      java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
      java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
      java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
      java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
      net.sf.ehcache.concurrent.ReadWriteLockSync.lock(ReadWriteLockSync.java:50)
      net.sf.ehcache.constructs.blocking.BlockingCache.doAndReleaseWriteLock(BlockingCache.java:295)
      net.sf.ehcache.constructs.blocking.BlockingCache.put(BlockingCache.java:217)
      org.hippoecm.hst.cache.ehcache.HstCacheEhCacheImpl.put(HstCacheEhCacheImpl.java:112)
      org.hippoecm.hst.servlet.utils.BinariesCache.putPage(BinariesCache.java:116)
      com.ifm.salesplatform.cms.servlets.BinariesServletPlainHippoCopy.getPageFromCacheOrLoadPage(BinariesServletPlainHippoCopy.java:391)
      com.ifm.salesplatform.cms.servlets.BinariesServletPlainHippoCopy.doGet(BinariesServletPlainHippoCopy.java:265)
      javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
      javax.servlet.http.HttpServlet.service(HttpServlet.java:729)

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              aschrijvers Ard Schrijvers
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: