Preliminary

As this is a small change in behavior (improvement!) by having the HST serving gzipped responses for cached pages, I prefer we do the backport with by default keeping the old behavior and include a setting / config to switch to the new behavior. I will ask nvankampenhout to document this setting once this has been backported

Also see https://bloomreach.slack.com/archives/C04JKLWA0V7/p1673560752458989

Current SaaS situation

Regardless whether a client (browser, curl, etc) does a Page Delivey API (PDA) or Content Delivery API (CDA) or upcoming Menu Delivery API (MDA) request with or without

Accept-Encoding: gzip

then when this request hits CloudFlare (CF), CF will always proxy the request with Accept-Encoding: gzip , caches it if needed compressed, and then depending on the client request, returns it compressed or uncompressed. The CF gets a PDA which is not compressed, it will do the compression itself. Also see https://support.cloudflare.com/hc/en-us/articles/200168396-What-will-Cloudflare-compress-

At this moment, for SaaS, PDA requests are compressed on the fly by Cloudflare as neither Envoy nor the CMS or any other proxy compresses. This is very problematic as CF can be 'far' away (cross zone/region) resulting in 10 fold data transfer sizes. Most likely, this is a significant AWS cost. This must be addressed ASAP

Proposed Solution

1. When page caching (for live PDA/CDA/MDA requests) is switched on in the CMS, we cache the results compressed and serve them
   1. compressed if the request contains Accept-Encoding : gzip
   2. uncompressed if the request does not contain Accept-Encoding : gzip
      Compression in java is most likely less efficient than with Envoy / CF, however, for cached PDA responses, the CMS only needs to do it when there is a cache miss: once cached, the compression does not need to happen any more
2. For all other resources, we do not serve compressed from the CMS module. Eg preview PDA/CDA/MDA requests, management apis, CMS interactions etc all won't be compressed by the CMS (tomcat) : this is not efficient. Instead, Envoy will take care of compressing these requests

Note Envoy will compress every request if uncompressed. As live PDA/CDA/MDA request will already be compressed when page caching is switched on (default for SaaS), Envoy will not need to compress these and is assumed to skip compressing (we assume that when the response from CMS contains

content-encoding: gzip

Envoy will skip compression, just like CF only compresses if not yet compressed. bkuzmic Can you confirm this?

Page Cache Size

As page cache entries will now be stored compressed, memory usage will most likely be around 1/10 of the size. For that reason, we increase the page cache size (max entries) from 1000 to 5000, which is expected with an average 10Kb compressed PDA in about 50Mb memory usage when cache is full

Testing QA

When 'page cache' is enabled (note, the QA profile by default disables page caching however SaaS customers have it enabled, see /hst:brxsaas/hst:hosts/dev-localhost/localhost/hst:root/@hst:cacheable or /hst:brxsaas/hst:hosts/br-cloud/cloud/bloomreach/$%7Bbrc.namespace%7D/$%7Bbrc.environmentname%7D/hst:root/@hst:cacheable etc to switch on caching), the Content Module will serve by default:

1. compressed response with Content-Encoding : gzip if the request has Accept-Encoding:gzip (,deflate, br ...optional)
2. uncompressed response without Content-Encoding header

If testing on BRC, you will always get a compressed response with Content-Encoding : gzip if the request has Accept-Encoding:gzip as CloudFlare will compress by default if the response is not yet compressed.

for local CMS instance, when testing with default wget / curl / ab (apache benchmark), the response will be uncompressed, even with page cache enabled. When testing with wget / curl / ab and adding custom header 'Accept-Encoding:gzip', the response will be compressed

when requesting the PDA in the browser the response will be compressed for LIVE PDA/CDA requests. For preview / cms requests, the response won't be compressed. If testing however in BRC, the responses will be compressed as Cloudflare does compression by default if a response is not yet compressed