Page tree
Skip to end of metadata
Go to start of metadata

In this session we discussed the problems and painpoints of working with Magnlolia configuration, some of the approaches people use to overcome them, and some directions how Magnolia could improve the workflow.

Problems working with Configuration:

  • Its difficult to work in the admincentral configuration tree.
    • Time consuming
    • Repetitive
    • Error Prone - you can mis-spell things etc.
  • Its difficult to work in a team with the configuration.
    • Changes to configuration are different then changes in code or resources.
    • You must remember to export them.
    • Others must think to import them.
    • Bootstrap file format is difficult to read and therefore merge.
  • Configuration is hard to update
    • We have the Module Version Handler (MVH) system to update configuration.
    • Its more time consuming to write and manage these then a normal code update.
  • Workflow / Convenience / Efficiency
    • Developers are used to working in IDE - strange to context switch to working in tree.
  • Configuration is very deep and complicated now.

Important: A major benefit of Magnolia is the ability to tweak the configuration on a live instance. We must ensure that we dont loose this feature. Agility.
Note that this change is persisted - if you restart the change is still in place. 

Things we could do to help the situation

  • Better file format (human readable / human writable) for configuration than the JCR system view
  • Better tooling in the admincentral config tree. (See AutoConfig App! Concept - Autosuggest in Configuration Tree)
  • Alternative mechanisms for creating configuration - See http://kickstart.io/register.html
  • Improved visualization of active configuration in configtree. (Ie show what is being extended - show what is created by code (blossom builders).
  • ConfigByFile - More code builders for example - for the apps.
  • We should try to provide autocomplete / autosuggest for configuration.

Existing Resources @ Magnolia.

Developers have different ways of working with Configuration to mitigate the above problems. They have even written some tools.

  • Magnolia best practice
    • Change configuration tree to suit requirements, export bootstrap files for each change and add to module + Write MVH to update configuration in fine-grained way so as not to clobber any changes that clients have made to the configuration.
    • Pros: Clean
    • Cons: Time consuming
    • Cons: Remember to export, Remember to import.
  • "I hate brusselsprouts" Tshirt developer
    • Instead of configuring in the tree - they write code in the install tasks to write the configuration that they want. Some of this code is run on every startup.
    • Pros: You dont have to remember to do something.
    • Cons: You cannot manually edit the configuration tree - because it would get overwritten at next reboot.
  • Use Blossom builders
    • To use code to write "dynamic configuration" for templates & dialogs at least. 
    • Cons: This configuration is not visible in configuration tree.
    • Pros: No need to write MVH or to remember to export/import.
  • Shared Instance
    • All devs work on same configuration tree, periodically export the full configuration and add to bootstraps. MVH load the full configuration overwriting old config.
    • Pros: Efficient / Easy
    • Cons: Configuration can get out of sync with the code. Can cause failures for other devs. Makes it hard/impossible to work on branches. Cannot roll back / revert code.

Developers have already written tools to assist in working with Configuration:

Major Questions

 

  • Should ConfigByFile/ConfigByCode write to the configuration, or be "volatile".
    • Writing to the configuration once:
      • + Obvious how it would be activatable to public instances.
      • + Obvious how to view the config in admincentral configtree.
      • -  MVH / exports / imports
    • Writing to the configuration on every startup:
      • + No MVH necessary
      • - How can users tweak values in config tree?
    • Volatile "dynamic configration".
      • No MVH necessary.
      • - How can users tweak values in config tree?


  • What format should the native MGNL config file have?
    • Format
      • xml
      • json
      • yaml
      • other
    • Should it mirror the tree? Or allow shortcuts? (Skipping items node in an actionbar group.)

 

  • No labels

9 Comments

  1. I would look around http://symfony.com/doc/current/book/index.html they do a very good job in configuration and fluent developement in the web. I don't like massive overhead in configfiles. I guess this kills xml and maybe json from the list. yaml is simple flat and easy to use. I would like to see if the tree is splitted up to modulebased yml files. Then its up to the module dev to split it again in smaller steps like apps or subapps, with a simple yml include. 

    1. Thanks for the Symfony tip. Ive heard many good things about the framework. Im really not sure yet how its all going to pan out - but certainly in considering refactoring the STK, modularizing the functionality is an obvious and important step. I could see how this could be valuable for apps as well - ie, maybe a standard default actionbar configuration for contentapps.

  2. My opinion on "Should ConfigByFile/ConfigByCode write to the configuration, or be "volatile".": I'd clearly prefer that the configurations in files are loaded "volatile". Why?

    1. It's basically the same mechanism as in resources, for example: you can put something into the resources workspace which overwrites resources loaded from the classpath (unless you tell the resource to be bypassed)
    2. in fact, I'd propose to stack configuration loading: 
      1. configuration is loaded by file from the classpath, 
      2. overwritten (merged) with configuration loaded file from webapp (optional), 
      3. overwritten (merged) with configuration specified in the configuration tree
    3. of course, it would be optimal, if there was a mode in the AdminCentral configuration tree to turn on "computed configuration", which shows the final configuration along with the sources each property stems from.
    4. the mechanism would/should work perfectly with hot deployment features (not sure about webapp files without JRebel though)
    5. loading it once/at each startup to the configuration workspace is really a pain; this mechanism is/was employed for themes as far as I know (or at least it has been in the past); it's very hard to remember the procedure on getting things persistently into the theme... 
    6. if people want to have it bootstrapped into the repository we could easily provide an update/startup task for this, but this shouldn't be the default mechanism IMHO

    Thanks for listening (wink)

    1. Thanks Vivian. Yeah that has merits. As much as possible id like all "resources" to behave the same - and why not consider configuration as resources as you suggest? So what you are proposing is similar to how we were thinking about harmonizing/improving how resources & templates work - with a "loading cascade" (see section on this page Concept - Ramp - (Lower entry barriers) ).
      What about activating/publishing changes to public instances, any thoughts on that?

      1. Yes. +1 for the loading cascade (wink)

        Oh and yes, I forgot to comment on the activation/publishing thing. This is IMHO really not an issue at all. During development I rarely work with a public instance at all. And if so, both must have the exact same state anyway, otherwise testing doesn't make sense. So you basically start the same webapp source twice, hence if you make a change on a resource in the one it's also done in the other instance immediately.

        And the other thing is: is activating configurations really recommended at all? If you activate the entire server configuration for instance, your public instance dies immediately...

        On the other hand, of course you could activate the configuration, but than again, only the things you want to overwrite. I thinks this works perfectly fine.

        And as I'm writing about it an additional advantage comes to my mind: if we had cascaded configuration loading, one could even load different configurations for different instances, which is a common requirement in practical projects: for instance, on a staging instance you load subscribers to other public instances in the staging environment, on a production author instance you load subscribers pointing to public instances of the production environment; There exists a mechanism for bootstrap files, however, it's unfortunately only working on clean installs, not on updates which makes the mechanism rather useless by now.

    2. Agree 100%

      #2 would be exactly the way to do it

      #3 would make it manageable and would need to be there from the start.

  3. My 2cts. regarding "What format should the native MGNL config file have?":

    Although I know people kind'a dislike XML, I'd really vote for using XML in the first place/as "native" format, because:

    1. XML can be validated perfectly well in terms of syntactic validation (with standards like XSD) and even semantically (using Schematron); I don't know whether the same powerful, mature tools exist for JSON or YAML? (probably they exist, but we should really investigate this)
    2. XML+XSD provides out-of-the-box code-completion in modern editors such as IntelliJ (and most probably Eclipse also); just by providing the schema location. I might be wrong, but I think you don't get the tool support for free for JSON/YAML?
    3. you can generate XSD schemas as is/out-of-the-box from existing Model/Server configuration classes
    4. XML can be easily imported to JCR - well we might get some trouble/need to tweak the import for proper property-type resolution (the JCR spec leaves this to implementations; we might suggest something for JCR 2.x/3.x?)
    5. XML provides namespaces which could turns out to be really powerful and proper: there could be special shortcuts (see below) for custom configurations: just use a different namespace and at runtime transform this to the Magnolia "native" format (in the default namespace)

    Having said this, I'd like to say that I'm not against the other formats. However, I'd prefer seeing XML as the native format and implementing compilers of other formats into the native one.

    And yes, shortcuts would be a great thing to make the configuration much less verbose. I'd suggest putting them into a different namespace though. This way it remains really clear that every node (within the Magnolia configuration namespace) is equivalent to a node in the config tree.

    1. I really like YAML because: so dang clean. Merges would be sweet. - but the code-competion could be a deal breaker there. I checked yesterday and there appears to be only one plugin for eclipse -and it doesnt appear that mature, doesnt appear to have code completion either - though I didnt try it. (http://marketplace.eclipse.org/content/yedit#.U7VqqKghskg) There might be a symfony or rails plugin that provides more, but afaik YAML doesnt even have a proper schema definition standard in the way xml does.
      Json does have schema. http://json-schema.org/

    2. I disagree with this one.

      IMHO XML is too verbose and cumbersome if you find a generic formulation for the configs. And formulating specific formats isn't practical, as magnolias config tree is totally general.

      And while true that schema verification is possible, that will add yet another layer of (meta-)configuration to magnolia. And who will write/maintain all these XSD files? In the end only a few config use cases will be covered, common ones like templates and dialogs.

      Also, importantly, XML doesn't work well with diff tools and VCS (git). And IMHO enabling VCS for the config files and being able to diff and merge them is a high priority requirement for moving to config by file...

       

      Regards from Vienna,

       

      Richard