Uploaded image for project: '[Read Only] - Hippo Configuration Management'
  1. [Read Only] - Hippo Configuration Management
  2. HCM-121

Fix jcr based Value resource access and file mapping

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Top
    • Resolution: Fixed
    • None
    • 1.0.0
    • None
    • 0.5
    • Platform Sprint 157: Finish, Platform Sprint 158: Hardening

    Description

      The current ValueImpl handling for jcr based resources is not working correctly.

      This concerns and affect both 'plain' export (REPO-1688) and 'delta' autoexport (REPO-1685).

      1. ValueImpl.toString() cannot be used at the same time for both a external resource file path and internal (jcr) resource lookup through .getResourceInputStream().
        This conflict has been worked around in REPO-1688 by not marking the value as resource but embedding the binary (byte[]) data directly.
        While this technically works (for this part), it isn't a acceptable scalable solution.
      2. The above workaround relies on ValueFileMapper to generate a new external resource file path (everytime).
        Which is incorrect/broken for several reasons:
        1. For content export, this requires all the needed parent property/nodes (e.g. value 'context') to be exported as well for ValueFileMapper to succeed.
          If however this context is not fully available, for example when trying to "YAML Export" /content/gallery/hcmdemo/samples/animal-2883_640.jpg/animal-2883_640.jpg this fails with "Root node does not have a parent" (thrown by DefinitionItemImpl.getParent()).
        2. For delta export, this more evidently is broken, as in that case an update/replacement to an existing binary resource, already having a named resource file path in the ConfigurationModel, gets 'disconnected' when replacing the Value with only embedded binary data.
          And even for a new binary resource, this might be the only 'delta' for a specific DefinitionNode, thereby also not providing enough value 'context' for ValueFileMapper to succeed.
        3. What really is needed is a different implementation of ValueFileMapper, using a jcr context to generate a proper resource path.
          The current ValueFileMapper implementation really only is reliably useful during the migration from esv2yaml, for which it was created,  but not as a general purpose runtime solution.
          Note also: HCM-94 (Move HippoImageFileMapper out of HCM and into the CMS) will require a jcr based solution as well.

      So these are two fundamental problems which needs to be fixed.

      In addition, we have another needed feature related to ValueFileMapper for which we don't have a solution yet: value file mapping for specific String properties.
      For example, if a new update script is added at runtime under /hippo:configuration/hippo:update/hippo:registry, we should be able to export the update node its hipposys:script property (type String) to <nodename>.groovy.
      And there are several 'script' properties in use which could/should get the same handling, like hipposcxml:source (mapped to <nodename>.scxml).
      So we need to provide an additional out-of-the-box StringValueFileMapper solution, possibly together/combined with a jcr based ValueFileMapper, or separate.
      And both these mapper solutions need to be pluggable, through Spring configuration or annotation scanning.

      Concerning the first two problems described above, which are blocking for the delta autoexport implementation:

      1. The first problem I will fix by:
        • Adding an additional internalResourcePath property to ValueImpl, to be set with a value generated by a (new) JcrResourceInputProvider.createResourcePath(property, valueIndex), and then passed back to that JcrResourceInputProvider for getting its InputStream.
        • To retain the reference to the original Source (and hence: its ResourceInputProvider), I'll adjust the ValueImpl.clone() method to at that time create an internal reference to it current Source. Currently this can/must be done by setting the foreignSource property externally, but by doing this during the clone() operation, this no longer is needed. I'll therefore will remove all the setForeignSource(Source) calls as well.
        • I've also added a ValueImpl.detach() method, which can (and maybe should) be called only after it is serialized.
          This method then will detach the ValueImpl from its Source origin.
          Alternatively, the whole ModuleImpl may be left to garbage collect, but if the ModuleImpl is retained (e.g. like after autoexport?) then the ValueImpl.detach() method should be called.
      2. The second problem is more hairy.
        Which I'll fix for now as follows:
        • When creating a binary ValueImpl (in jcr context) it will be marked as a resource and its value (external resource file path) will be seeded with "data.bin" or "data[valueIndex].bin" for multi-value properties.
          Also, the internalResourcePath property will be set (see above), so the binary data can/will be retrieved only during serialization.
        • Another Value boolean method, Value.isNewResource(), is added to the interface, indicating the value resource file path is to be treated as a candidate path, when there is no (known) existing file mapping yet.
          For the "YAML Export" feature this is always the case, and for delta autoexport this will be set true for new values (not being matching to an existing value in the ConfigurationModel).
        • FileConfigurationWriter currently collects all known resource files up front through ModuleContext.addExistingFilesToKnownList().
          This will be improved to also recognize isNewResource() Value resources, generate a final resource file path for them, and thereafter update the ValueImpl its (resource path) value accordingly.
          Note: this is also needed as one or more Source definitions may use/generate the same "resource" path value for different new values, which may even conflict/clash with existing resource paths.
          And to better reflect its purpose, the ModuleContext.addExistingFilesToKnownList() method will be renamed to ModuleContext.collectExistingFilesAndResolveNewResources().
          Note: to be able to implement this, all ModuleContext implementation now require a ModuleImpl parameter not just the Module interface, which triggered another bunch of (trivial) changes.
        • The above fixes 'work' for the "YAML Export" feature.
          However because the ValueFileMapper no longer is/can be used, there is currently no 'nice' naming of the binary resources, other that default "data[x]-y.bin".
          The limitation needs to be fixed by providing a jcr based ValueFileMapper, then to be invoked during the creation of a binary ValueImpl within the context of jcr, to set a more appropriate resource path.

      Remaining tasks (for which additional JIRA tickets are needed):

      • required: add a jcr based ValueFileMapper solution (and maybe move the existing one to the migration module?): HCM-124
      • later (but before GA): add a jcr based StringValueFileMapper to support serializing specific (type of) String properties to external resource files: HCM-125

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              adouma Ate Douma
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: