Page tree
Skip to end of metadata
Go to start of metadata

Dear Magnolians,

Do you use the Processed Resources feature in your projects? At Magnolia, we would like to hear about your specific use cases.

(This is the feature where a resource file can be an processed as an ftl, and also automatically concatenated together.)

Why am I asking? - Because we have a new resources servlet - which makes working with resources much more straightforward. Its time to re-evaluate what functionality is required - balancing the benefits of a feature, vs the complexity it adds to learning Magnolia, implementing and maintaining Magnolia web projects - as well as the complexity of the Magnolia codebase.

A lot has changed in the web world since we first introduced that feature. Now many teams do their less, sass, concatination, minification, etc processing beforehand - using tools like grunt, gulp, browserify etc. And they add these pre-processed resources to Magnolia. And of course this frontend world will continue to evolve rapidly. There are two use-cases that I know of that the pre-processing does not exactly cover. (Also there are some nice Maven plugins that make adding the preprocessing to your build chain easy.)

Use Case: Project specific processing

One use case that this "offline" preprocessing cannot help with - is where you want to do project-specific processing - such as grabbing a color value from each site configuration to use in the css of that site.

A replacement is to compute these values in a normal component and render them in the template: as an inline style in the case of css - or as a parameter to a javascript function. So the javascript resource can be static - but use the value provided in the processed template.

Another possible approach - if you have a resource that you want to be highly processed, just like a Magnolia component - with a custom renderer, model class, etc... Is to simply really use an actual component template or page template that outputs css or js code. This content could simply be included inline in the page header - or actually linked to like a normal resource using Direct Component Rendering (https://documentation.magnolia-cms.com/display/DOCS/Rendering+content#Renderingcontent-Directcomponentrendering).

Use Case: Hotfix originals

Another use case is that you like the ability to hotfix a css or js resource directly on your server. It is convenient to hotfix the clean unprocessed file in adminCentral - and then Magnolia processes it again for you.

This is true - but if you did pre-process your resources with grunt and include that file with Magnolia - you would still be able to hotfix manually via adminCentral - though the file will probably be harder to read! Or you could rebuild the file locally with grunt again and hotfix the whole file.

Feedback

Are one of these use cases important on your project? Would the alternative approaches work for your case? Why not? Do you have other important use cases?

Please add comments to this wiki document. (Create an account. Or to the forum if you dont have an account here).

Thanks for your input on the topic!

  • No labels

24 Comments

  1. We are using processed resources for a big project - currently at 5.3.10 - where a "Theming" app allows the editors to define subsites with distinct styles - such as color, fonts etc. Once the styles have been defined in our app, there are 2 types of resources delivered as processed ones:

    • A CSS template including placeholders is filled with default values or the ones defined in the Themings-app for this site
    • Most SVG-icons are processed ones. The color-definitions for the graphs have been replaced by placeholders and the color of the icons is defined based on the settings in the Theming-app
  2. I think rendering dynamic css or js resources by using website components is not the appropriate way to go. It would be an ugly workaround.

  3. In the use case mentioned above (btw. for a prominent customer ) the customer wanted easy and direct influence on the colors and styles of his subsites. Magnolia apps as frontend are made for this. The customer needed something an in-house editor can handle and where the immediate effect can be checked in another browser window. And the customer strictly denied the installation of any other tools or processes on his server (e.g. like sass). And anyway this would have been hard to handle - either using a native library for the OS or some Java-sass which is nothing more than a Java wrapper around a Ruby-implementation of it. We tried the Java-Ruby-sass in another project and resigned because of the resources and time this one needed for merely creating a CSS.

    1. Thanks for commenting - especially with the details.

  4. Forum post:

    Please don't deprecate the old (pre 5.4) resources. I am missing the processed js/css, references, model classes, folders, ...
    I used the old resources to create CSS and JS bundles, minimizing and SASSing them. I tried to do that with the new resource-loader but that is not possible for several reasons. I cannot put generic yaml in the resources, just templates, renderers, dialogs, ... and the path must match a specific regular expression. Also creating the bundles by adding references to the resources as subnodes to the bundle was so elegant.

    https://forum.magnolia-cms.com/forum/thread.html?threadId=75f45a9e-fc23-451e-8c3f-4d89a0183dc8

  5. We use it a lot in our project (Hotetec). Our use case is to have one heavily customizable template for several different websites (currently more than 50). We developed custom dialogs allowing the user to easily choose different font types, sizes, colors, and so on for each websites. These values are stored as website content. All the system is working with processed CSS that pick-up the values of the variables in the website content.

    We invested a considerable amount of time to create the generic CSS & custom dialogs (it's one of the points that will be painful in our migration to Magnolia 5, currently we use 4.5), so our position right now is: pretty please, don't touch it too much (big grin).

  6. What if you need translatable labels in a Javascript file? It would be great to use the same I18n mechanism we use in the freemarker templates. Or how would you solve that otherwise?

     

    1. Hi Pierre, thanks for your comment.
      Some common approaches in JS plugins/libraries for this are:

      • If the JS file contains functions to call: You would pass in the labels when calling the method., 
      • If the JS file has code that is simply run "inline" as it were - you can provide configuration for it by defining global variables in the html file that calls the page. 

      So the template that calls the method would get the localized labels and pass them to the JS. for example:

      myJSMethod(${myTitle}, ${mySubject);

      or as one parameter...

      myJSMehod({
           title: ${myTitle},
           subject: ${mySubject}
      }); 

      Would an approach like this work in your case, or would it not work - or be clunky for some reason?

      1. Thanks for your anwer.

        We do not call JS methods, the JS simply registers events on buttons etc. So the parameter passing method does not work in this case.

        But defining global JS variables for the labels in the HTML code would work for us and be an acceptable solution.

  7. Yes, we are also using processed resources. In our current project for example in the FTL's of our custom login forms (in the mgnl-resources folder). There we use stuff like the ctx object and such. In the past we have also used processed CSS resources.

  8. The case for hotfixes is moot. The processing doesn't need to happen offline. It's relatively easy to implement a sass-or-other processing in a filter. IIRC, we drafted some ideas about this too. (e.g to have metadata attached to resources, which isn't possible at the moment)

  9. Please DONT remove processed resources.

    We use them for a variety of little jobs, it is very convenient to have this feature. Some examples include:

    • adding the context path when accessing URLs in other sites (we have a EE multi-site setup and some sites 'share' some resources - in this case you want server-absolute URLs to access the other site's content)
    • when generating content in css:before and css:after and similar content-creating CSS jobs.
    • to create dynamic javascripts
    • to aggregate CSS or JS files into an "all.js"
    • to compress aggregated CSS via yuicompressor or CSSO

     

    There is also my SCSS module on forge which builds on the processed resource idea:

    SCSS

     

    Finally I would put in a plea to stop changing the way templating, resources and DAM work with every release. The changes in templating and DAM between 4.4 and 5 have meant that we have had to more or less recreate dozens of sites. Some customers were lost due to this, as they were not prepared to pay for the changes. Leaving sites on old versions is not a long term option since magnolia does not support the old versions.

    So please don't do the same thing with resources. Please ensure that the changes and improvements you make to resources module remain compatible with the way resources have worked until now. We are really tired of migrating sites for no reason other than to maintain compatibility.

     

     

     

     

    1. I have to agree with Richard here. Of course we understand that the product needs to evolve and I think it is evolving in the right direction (especially with 5.4). However I think Magnolia should take greater care in remaining backwards compatible because we have really spend a lot of time and our clients' money on Magnolia upgrades over the last few years.

      I also think Magnolia should be somewhat more aware of all our often highly customised Magnolia projects that need to be migrated for every new Magnolia release and that proper testing on the side of Magnolia for such upgrade scenario's needs to be done. I am afraid that too often we are finding bugs in the new Magnolia release that really should have been found by the Magnolia development team before releasing.

      1. But it IS 100% backwards compatible. If you reference resources by {{/resources}}, the "old" mechanism applies, and your resources can be "processed". If you use the new mechanism through /.resources (note the leading dot), then you benefit from a consistent resource loading/overloading (which was inconsistent between web resources, classpath resources, templates, scripts, etc, before), but you currently don't have the possibility to post-process them yet.

        1. Hi Gregory, I am afraid I do not quite agree.. We have created a ticket for this and I understood earlier from our related support issue that this is hot on the list to be solved in 5.4.2.

          1. Ha! That is indeed a regression (see my comment on  MAGNOLIA-6355 - Getting issue details... STATUS ), but has very little to do with resources-as-referred-to-here (the login form is one of two cases where FreeMarker is used "before" we can use the now-out-of-core freemarker-support module; the second is the module installation ui)

        2. In addition to the missing processed support for non-JCR resources, and the issues Edgar mentions below, the Resources App has been crippled (in 5.4.0): You can't upload new resources in JCR, the functions to add resources have disappeared from the JCR app.

          Examining the changes made in resources, It looks to me like you're changing Magnolia's content model away from JCR. That would be a move I would find extremely bad. I think the clean seperation of Java Code (Web App), content and layout (which includes design resources) is (was) one of magnolia's key strengths.

          The thought of having to create and deploy websites as if they were java developments is, IMHO, a very bad plan.

           

          On the other hand, I don't want to be all negative - on the whole I love magnolia 5's new interface, and the new possibilities with content apps, so please don't think I'm against change in principle.

           

  10. Another feedback:

    Please provide a clean API for the resources module. At the moment there is no way to actually load a resource without duplicating a bunch of code out of the resources module.

    Please provide a nice API like an injectable ResourceLoader which provides methods like:

    InputStream getBinaryResourceContents(Node)

    and

    InputStream getTextResourceContents(Node)

    1. Hey Richard,

      Have you looked at 5.4 yet ? Try injecting ResourceOrigin somewhere (smile)

      1. Thanks, yes, I noted it but haven't looked at 5.4 in detail yet. I noticed in 5.4.0 that there is no way to create JCR Resources, which I think is very bad, but I assume the feature will be added back in later versions?

        Also I noted that you can activate the "old" resources app, but if you do then it does not have any lables, which makes it hard to use... I assume that is also something to be fixed in a later release?

        I will look at 5.4.2 in the next few days, if I find time...

         

         

  11. The point about all this is not if there might be a better way to do something or not. The point is that in a simple minor version change of this software a feature - which had existed for several years -  disappears without prior notice or discussion with the customers and users. This is just a disrespect for the amount of time and money the customers have invested - based on this feature.

  12. Currently we use processed resources mostly in the processing of our javascript.

    As we have a full Development/Testing/Staging/Production setup each of these have there own external services (api's to media,search and javascript cdn)
    So the javascript includes a configuration.ftl which access the system properties to inject the correct urls and api-keys for use of those external services.

    This also supports versioning (using a tag of our own)  of the external javascript as each site can have their own version of the javascript libraries.
    This inherits from the global version which can be overruled in each site.

     

     

  13. First - thanks a lot to everyone for being willing to share about your usage of processed resources - and taking the time to do it. Also, we hear your frustrations about the change in API, and do work to maintain backwards compatibilitity as we work to add the features that we believe improve the platform. 

    But an apology is definitely in order for how we handled the introduction of the new resources in 5.4.  We should have been more careful to maintain the processed resources functionality of the "old resources" app. Our intention was that you could continue to use the old app by re-enabling it in the app launcher - but this was not well executed by us, and there were issues with the language files. I'm truly sorry about that.

    So much for the preamble. Now to what we have planned.

    5.4.4 will introduce a new module which will enable those who are using the current processed resources mechanism with the /resources path. This "Processed Resources" module contains the app and functionality for that mechanism. Everything works as it has previously, remember this works via the /resources path and is only available on resources stored in JCR repository.

    Additionally - based on your input on this page and discussions here at Magnoila we decided that we will indeed support a similar processing of resources on the new resources servlet (Available at /.resources). We acknowledge the value of the "different style per site" use case, the power of being able to tweak styles and have the system re-process those for you, and the productivity benefits of magnolia being able to process files for you (such as .scss).  This will work on all resources regardless of origin: on filesystem, classpath & JCR. It won't work exactly the same. For example concatenation via nested resources does not make sense on the file system - but the key features will be included. I do not know when.

    So - processing on new resources is coming - and in the meantime you will be able to use the previous mechanism and previous app on the 5.4.x series when 5.4.4 comes out.

     

  14. Hi Topher,

    Still awaiting the processed resources feature in 5.4.7 (wink)

    In the meantime I have had a chance to look at the new Resources API in detail, and wanted to give some feedback.

    Processed Resources

    I don't fully understand what happened with the design of the new API. The old setup was as follows:

    1) There was a ClasspathSpoolServlet, which handled /.resources/, and basically just found the static resources in the classpath, and served them.

    2) There was a rendering mechnism via /resources/, using RepositoryMapping, ResourceLoaders to find the resource, and the possibility to "bypass" JCR stored resources for development.

    I agree the old setup was a bit confusing, especially the "bypass" concept, and needed consolidation.

    The new setup introduces the idea of unified resource loading from ResourceOrigins, and a LayeredResourceOrigin to combine resources from the three sources into a unified view. That is actually very cool.

    The bad part is the following: the new mechanism was built along the ClasspathSpoolServlet path, and the rendering step was therefore omitted. The immediate consequence was that processed resources stopped working.

    However, I think the idea that resources get "rendered" is an essential one: not only does it allow the concept of processed resources (where CSS, JS or other text resources get produced as output of freemarker templates) but is is also needed for any other "advanced" type of resource involving interpretation or compilation, i.e, where the rendered output to the browser is different from the raw input file. Examples would include web technologies like LESS/SASS/SCSS which compile richer descriptions into simple CSS, and also optimization techniques such as aggregation or minification of CSS/JS. (It could even be used to implement cool things like "automatic sprite images")

    (Note: I'm the author of the magnolia-scss-module which can be found on forge. I'm directly affected by this in that module.)

    IMHO, what should be done is the following:

    1) Get rid of the old resources app. Having two that behave differently is totally confusing, and not a good solution.

    2) Teach the new resources app to handle processed resources by (re-)introducing a rendering step for resources. Provide default Text- and BinaryResourceRenderers, as well as a FreeMarkerTextResourceRenderer.

    Unfortunately there are two problems now faced by the current solution:

    1) The standard rendering mechanism is linked to the JCR Node API so either a new Rendering mechanism for resources has to be defined, or the standard mechanism has to be refactored to allow rendering content that doesn't come from JCR.

    2) The "processed" attribute is stored in JCR, and there is no good way to store this metadata field for the filesystem or classpath resources. I suggest changing the mechanism to be based on the resource filename extension: so processed css could be ".pcss" or ".ftlcss". The resource filename extension should determine the resource template definition, which in turn determines the Renderer to use. A ResourceDefintionFinder class can encapsulate this behaviour to make it extensible.

    ResourceOrigin API

    I like where this is going, but unfortunately it is currently not in a useful state. It's too hard-coded and much too closed. Here are some specifics:

    • The LayeredResourceOrigins with the three standard resource loaders is hard-coded in DefaultResourceOrigins. This should be configurable in JCR, either in the server setup or resources module
    • There is no way to access the individual layers of LayeredResourceOrigin
    • There is no way to access the details of a Resource or ResourceOrigin implementation, eg to get the JCR node, the FileSystem File or Classpath Resource.
    • The API is read-only

    IMHO there is really a lot missing. Want to make a module that only deals with a certain layer? you can't. Want to get at the underlying resource storage (node or file)? You can't.

    Why would you want to do such things? IMHO there are important use cases: consider for example modules that want to compare resources between the layers, things like importer/exporter modules for the resources, specialized editors, migration tools, etc, etc...

    General Remark

    Don't make everything private! It really doesn't make your code any better or safer. In 99% of cases protected is a better option.

    Magnolia code used to be so easy to extend, but since the 5.0 development rush everything is private, private, private, and you wind up having to duplicate entire class structures to implement tiny extensions.

     

    Thanks for listening and regards from Vienna,

    Richard