Build a central place for maven dependency management of all 3rd party libraries for all magnolia modules (bundled, unbundled, community, enterprise, etc).
Purpose
The Problem
We manage most dependencies in magnolia main & ui. We import main and/or ui in other modules to reuse the dependency management.
This turned out to be not the best idea because of the following reasons:
- We have the policy to alway depend on the oldest main version possible. Which means we tend to depend always on outdated 3rd-party libs in modules.
- We want to reduce the footprint of main and because of we managed some dependencies in other modules like UI specific libs in UI, REST libraries in rest, etc. This leads to potential divergent versions (like we actually had with the jackson library)
Goals
The most agreed on solution is to have a specific project/pom where we will just define the dependency management. This project will then be imported in all modules.
Use Cases
> AKA: User Stories
> List concrete ways in which this feature will be used.
Proposal
Concept
Special BOM project which resides at the same place like the super-poms.
Would look like:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd "> <modelVersion>4.0.0</modelVersion> <parent> <groupId>info.magnolia.maven.poms</groupId> <artifactId>magnolia-parent-pom-community</artifactId> <version>33</version> </parent> <groupId>info.magnolia.boms</groupId> <artifactId>magnolia-external-dependencies</artifactId> <packaging>pom</packaging> <name>Magnolia 3rd-party dependencies</name> <version>5.6-SNAPSHOT</version> <!-- Bill of material for every magnolia module to be used. --> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> <scmTagPrefix>magnolia-bom</scmTagPrefix> </properties> <dependencyManagement> <dependencies> <dependency> <groupId>javax.servlet</groupId> <artifactId>javax.servlet-api</artifactId> <version>3.0.1</version> <scope>provided</scope> </dependency> <dependency> <groupId>javax.jcr</groupId> <artifactId>jcr</artifactId> <version>2.0</version> </dependency> <dependency> <groupId>org.apache.jackrabbit</groupId> <artifactId>jackrabbit-core</artifactId> <version>2.12.4</version> <exclusions> <exclusion> <groupId>commons-logging</groupId> <artifactId>commons-logging</artifactId> </exclusion> <!-- maybe this will have to be un-excluded if we remove our explicit usage of it in magnolia-core --> <exclusion> <groupId>xerces</groupId> <artifactId>xercesImpl</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>org.apache.jackrabbit</groupId> <artifactId>jackrabbit-jcr-commons</artifactId> <version>2.12.4</version> </dependency> ... </dependencies> </dependencyManagement> </project>
And be used like this:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd "> <modelVersion>4.0.0</modelVersion> <parent> <groupId>info.magnolia.maven.poms</groupId> <artifactId>magnolia-parent-pom-community</artifactId> <version>33</version> </parent> <groupId>info.magnolia.bundle</groupId> <artifactId>magnolia-bundle-parent</artifactId> <version>5.6-SNAPSHOT</version> ... <dependencyManagement> <dependencies> <dependency> <groupId>info.magnolia.boms</groupId> <artifactId>magnolia-external-dependencies</artifactId> <version>5.6-SNAPSHOT</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> ... </project>
Simplified overview of dependencies between super-pom, bom, modules and bundles:
Decisions
We made the following design decisions:
Design Decision | Reasoning | |
---|---|---|
1 | We want to define the dependency management in a separate BOM project and not in one of the super-poms | We want to try to keep parent poms in sync over all release streams. Both 5.4 and 5.5 should use the most current magnolia-parent-pom (currently 33). We can basically do that for build plugins but we can't do that for 3rd party libraries. |
2 | We bump BOM Version together with Magnolia Release even if it has no changes | Its a simple POM project/file and causes no overhead even if we do a lot of version bumps. One reason for creating BOMs is to get a grip on the version hell in magnolia. If the BOM version now diverges from the Magnolia Release version its more complex and harder to follow for us and customers. |
3 | The 3rd-party BOM resides in a separate repository in the build project. | This way its easier to maintain. Easier to work with the different BOMs for the release streams (5.4.x, 5.5.x, 5.6.x ...) Reasons for packing it with the Super POMs were:
|
4 | The Magnolia BOM resides in the same repository as the 3rd-party BOM | Those two projects have the exact same release cycle It has actually the same release cycle like ce & ee & ce-packs & ee-packs. But because those are in separate repositories the Magnolia BOM can not reside with them. |
5 | The Magnolia BOM becomes the driver for a Magnolia Release | Because the Magnolia BOM includes versions from bundled and unbundled modules it can become the single source of truth for the releases instead of the JIRA changelog filter. |
6 | We backport the 3rd-party BOM to the 5.5 release stream starting with 5.5.7. | We have several modules which are atm backwards compatible with 5.5 with their current release. Additional one of them The versions of 3rd-party libraries is atm we made that decision (Sept. 11, 2017) identical and because the BOM is transparent to end users, the backport should have no impact on customers. |
Current places where we maintain 3rd party library versions
- platform/main (magnolia-project)
- used ... everywhere
- platform/ui (magnolia-ui-project)
- used .. most places
- module/rest (magnolia-rest-parent)
- used in cache
- platform/ce (for test libraries)
Scenarios
The following scenarios high-light how specific use cases would look like with the new central dependency management in place.
Simplified software structure
For the following scenarios we use this simplified setup. |
Release new version of the super POMs
Release of new magnolia module version
Library update without changes in modules
Security Release which includes 3rd party lib
If we want to make a module hotfix release, because of a security issue for example, we would temporarily manage the dependency in that specific module. Later we move it to the 3rd-party BOM and remove the explicit management in the module. We should have tooling to validate that no versions are managed in individual modules. Issue with patched version: customer potentially still imports bom 5.5 in his bundle and overrides version from rest. |
Implementation
Following steps:
- Move main dependencies without changes to `magnolia-external-dependencies`
- Use `magnolia-external-dependencies` in main
- Move ui dependencies without changes to `magnolia-external-dependencies`
- Use `magnolia-external-dependencies` in UI
- ... repeat those steps for any other place dependencies are managed atm like Rest or CE
- Reorganize dependencies in `magnolia-external-dependencies`
Thoughts
We should probably do the same for our own artifacts as well like indirectly requested by MAGNOLIA-6343
Open Questions
Question | Answer | Reasoning |
---|---|---|
| ||
Which dependencies do we manage?
| All dependencies, all scopes | Simplicity & atm, no reason why we can't |
What's the release cycle of this pom? | Same as Magnolia Bundle Releases, Yes this includes empty releases (see above) Jan 12/04/18: Contrary to previous agreement and line of thinking, we found out (by accident) that it is not necessary to keep version numbers in sync. One can simply rely on Magnolia bundle version and import dependency management section of the pom as shown in documentation. | Simplicity |
Does it make sense to test different depMan sections (via profiles) to check compatibility? | --- | |
Validate whether modules still work with base line (when specifying dep. to e.g. 5.4.x) | Let's burn that bridge when we get there | |
How does the standard release work like related to the changes from this concept?! Do we need/want some kind of release aggregator? (BOM) | 3rd-party BOM becomes first in line in release process. | |
How do we version the BOM? | It should be somehow in sync with the main magnolia versions | |
Do we wanna be always on the latest version of the BOM? (similar question goes to all dependencies) | Is tied to the general question of always on current. For the moment we keep the same policy as with dep on main. | |
How can we find unused dependencies? (for cleanup) | --- |
Specific for Magnolia BOM
Can we have ee (non accessible) dependencies managed in the BOM? Can community users still use it?
Do we have dependencies we have to manage to the bundles?
And if so (like webapps) how do we handle them?
Glossary
Name | Definition |
---|---|
Magnolia Release | The full stack release or the bundle release. |
Super-POM | Actually a set of parent poms. |
BOM | Bill of material. A single place to define material of in this case dependencies we can use. |
3rd-party BOM | BOM for all external dependencies. |
Magnolia BOM | BOM for all magnolia modules |
Etcetera
> Possible Improvements
> Discarded Proposals
> Discussion
> References