Tackle Performance problems associated with the instantiation of new sessions

The problem

TLDR when we have a lot of domains on the system and a lot of sessions need to be created, the performance is very poor due to bottlenecks in concurrent operations.

Based on some feedback from customers (PaaS), we have seen that in some cases creating a new session can be an expensive operation.

After some digging, we found that the main reason for this was the large number of domains the user had, plus the way we use a single session to retrieve the domains, which creates a bottleneck if many threads are trying to do the same operation because there are some locks to prevent many operations to be performed at the same time on the same session.

Usually, we don't need to create so many sessions, because we rely on a session pool that will have sessions for us to borrow, however, when the server is first started, if a lot of traffic reaches the server at the same time, a lot of sessions will need to be created concurrently and this problem can be reproduced. There are also some internal listeners that on certain events logout sessions from the pool and can cause the same effect.

 The solution

TLDR we will cache the result of the expensive operation

There are different solutions we could apply here, for example using a different session per request to perform the operations of the domain would ease the concurrent problems (although we would still have some problems due to this implementation that belongs to Lucene).

The approach we will take is to cache the results of the poor performant operation, since it's something that we don't need to alter at all, plus it rarely changes, so it will stay on the cache for a long time.