Also known as Public API. The main idea is to declare explicitly what code we expect to be used by customers and what code we don't expect them to use. This will reduce the amount of code Locked because of backward compatibility. This reduction will make us more flexible and agile with code changes while at the same time strengthen the backward compatibility.
This will deliver the following benefits:
- More powerful innovation capability – because we can change code without breaking compatibility
- Shorter release cycles for features – because if a feature can be implemented without changing any or at max one API the complexity is not heavily increased.
- Even more stable compatibility with releases – as we declare the API explicitly we can test automatically against it (atm we just produce a report where a human has to evaluate all the finding because some breaks in the current setup are to be expected)
- Magnolia is easier to learn for new developers because the to be used API is more explicit – the less clutter or not used code is presented the easier it is to find and understand what you want.
- Faster build times – because the extraction of Public APIs remove compile time dependencies. This is only true if we're going to break down long dependency chains like for commenting.
- Restrict or more conscious use of type-mappings and injection of components
- Will take a long time to implement, not because of the effort, but because it needs several iterations to get it right. The technology decision is here clearly the smaller part. The bigger effort is realizing and leaning over several iterations where our current setup is orthogonal to the Public API approach and adapt it.
- For an actual higher level API or actual Platform of APIs for Magnolia we have to answer soon two question:
- What is the minimal set of modules we expect customers to use which provide the platform or minimal set of Public APIs the rest of the modules build upon (incl. customer modules).
- Do we understand our solution to be more a Product (where we define the lifecycle of the produced solution and define clear extensibility points for customer to build upon) or more a Framework (where the customer builds together with a set of magnolia modules the solution)
- Doing Public APIs with the current setup of around 50 modules will introduce an insane level of complexity. In a way it would double the amount of modules or artifacts and would exponentially increase the amount of dependencies. This is because of the potential separate versioning of the API. As the current setup is already too much ... this would then be way too much
Because of that we have to have a concept for a higher level of abstraction where this kind of Public API could reside ... hence the importance of the previous point.
- Done wrong can end horrid: If we look at it as just a technical problem and implement something quick and dirty we risk being in a worse situation than before. As said in the previous point, adding boundaries to the modules will increase the complexity. If we're not able to leverage the potential of Public APIs to reduce complexity, because of bad designed APIs still leaking internals to the public, too many APIs, etc, we will have just increased the complexity of our system without adding any benefit.
- Public API is a M-NOW topic: Although it might be a quite powerful advantage for it, it has actually nothing to do with M-NOW.
- Microservices will solve that for us: Yes if we have microservices we have solved the problem of defining components with a specific boundary or API. But: its the same as saying I climb mount everest to solve my issue of not being able to climb. Yes if I reach the top I have should that I learned to climb ... IF. Thats basically the statement every article about microservices is making: don't do it if you haven't already well defined components.
- We don't have that problem with LD so it might solve itself in the future: The only reason why we don't have the same issue atm is because we have LD version 1.0. In the future we will have the same issue because for example definitions are translated more or less 1:1 to Java objects and specifications for YAML files will change as well. In some way the problem is even worse because the compatibility breaks in Java are obvious at compile time where as with LD the issues will show up only at runtime.
- We don't need that for M-NOW because we can solve it over services: Although external services move part of the issue from Magnolia to the Service and therefore we rely on a good API of the service, a service still has impact on Magnolia meaning relies on code inside Magnolia and we have the same issue.
- Headless CMS == Public API: Although a headless approach requires well defined and organized APIs, it actually has nothing to do with each other. Public API or open / closed code base is first and foremost about maintaining innovation capabilities. Without leaking internals to other subsystems we can do more radical changes and better prepare for them. Think about the fundamental change of Apple from PowerPC to Intel or from Objective-C to Swift. Those changes were only possible because the internals are abstracted with higher level functions and therefore unimportant for users.
First steps (my proposal)
- Pick an existing API we expect customers to use. Maybe even one the service team has created solution modules on top of it.
- For example the Workflows with the new Workflow Editor could be a candidate (I just picked it at random, because the last demo I saw was Adrien Manzoni's presentation of the Workflow Editor)
- Try out several technical solutions to extract/expose the API — this set of PoC is ideally done by a small group of developers how then can easily compare the different approaches and select the one best suited for us
- Start actually adapting an existing module
- Here the fun begins
- Realize where our current APIs are not well enough designed (a common issue is, that if you pull up the interfaces and classes a level higher, half or more of the system has to be pulled up as well)
- Create concept / design principles for APIs and adapt to them
- Realize where our architecture doesn't support or doesn't well support a Public API
- Create concepts for architecture adjustments
- Iterate over the previous points until we have something to potentially release to a customer as beta
- Release a Beta version for selected partners/customers to use
- Probably start all over again because of the feedback
- Create an official release which is part of the next major release and can be used in parallel with the previous API
Some additional food for thought
- Technical solutions
- Java 9 module system (if we want to have versioned APIs and having several at once ... this might not be an ideal solution.)
- OSGi (since Java 9 this might be outdated except we need runtime functionality or versioned APIs)
- Use Annotations to declare/export API (https://developer.atlassian.com/jiradev/jira-apis/java-api-policy-for-jira)
Annotations can be used to provide additional meta information about an API.
We could introduce a
@Privateannotation at class level to clearly mark which classes/interfaces in our API may change or be removed even in minor versions without prior notice.
- IDEs would detect the annotation more easily than e.g. plain Javadoc
- We would have one place in our codebase where to consistently define what is private API for Magnolia
- Unlike marking a class as
finalor reducing its visibility (default package,
protected/default constructors) there would be no technical limitations for us to override or extend "private" functionality if needed. This latter approach isn't ruled out by the annotation anyway and can be used alongside in some cases.
- Dependency injection will play an important role. Therefore whatever solution we choose the DI has to be part of it.
- Our definitions are part of the Public API or Vice Versa. This is even more important with LD as compatibility issues there will show up on runtime and not already on compile time. (And LD stuff is not migrated like JCR)
- Interesting video from Java9 seems they have the same issues as we have: GOTO 2017 • Javalution - How I Learned to Stop Worrying & Play Jenga • G. Saab & A. García-Ribeyro