The performance load test project has been used to for estimating the impact of the UI performance improvements (mainly new IoC mechanism but other smaller improvements too). Two test instances have undergone several load test runs with a varying amount of concurrent users. The main emphasis of this load testing session has been put onto benchmarking basic UI usage scenarios: logging in, opening and closing apps and sub-apps. The tests typically include a larger amount of the users who just open up apps and wander around AdminCentral whereas a lesser amount of users render the pages. Unfortunately, most of the test runs do not include the page editing capabilities.
the tests needed adaptation and it took some time to figure out (...and after they were finally fixed - the test machine has died and at the moment of writing this is still at the service
That functionality is not directly relevant to the benchmarked scenarios, so its absence in report should not be a huge issue.
The tests have been conducted with the Macbook Pro 2016 (i7 2700 Ghz, 8 computational units), the JVM heap size was 2 GB for the normal tests and 10GB - for the more hardcore ones.
What improvements were added
- MGNLUI-4180 - Getting issue details... STATUS
AnnotationUtilsinvocation results caching
Testing Mgnl 5.5.3 with up to 100 concurrent users
The test indicates that the most expensive [dynamic] requests are those related to actual logging in and opening the apps. On average it takes more than a second to start an app and to log in. openPage opens the page editor sub-app and also gets expensive under heavy load.
The second test run with 5.5.3 and up to 100 concurrent users
The second run with the same tests and the same JVM instance has been conducted and has shown much better results (instance got properly warmed up and JIT kicked in). However, the biggest "culprits" are the same. It should be notes though that on a warmed-up JVM a hundred users, who just browse the data and and open up this or that app occasionally, should not impose any severe problems, though some sluggishness might happen.
Testing improved Mgnl 5.5.4 with up to 100 concurrent users
Then the tests were run against the 5.5.4 SNAPSHOT with the new IoC and some other improvements added. There were also two test runs made in order to see the behaviour of "cold" and "warm" JVM.
The first test run, cold JVM
A major performance improvement when it comes to opening the apps: mean 34ms vs 1900+ms!
Logging in has become significantly cheaper as well, though not that dramatically since there are other heavy operations involved besides of the improved ones, anyhow mean 756ms vs 413ms is an improvement. Opening page editor has also become much faster (15ms vs 489ms on the average)
The second run, warm JVM
As expected - the second run has improved the results even more. Actually all the request take way below a second to respond (can assume good stable performance )
Resource consumption analysis
The above results can be explained with some use of the profiler. The following screenshot illustrates the CPU consumption hotspots during a load test run with 5.5.3.
Similar output scenario with the improved 5.5.4:
Monitor output of 5.5.3 test run:
Monitor output of 5.5.4 test run:
Heavy test run:
We gave a shot at a use case when up to 400 users bombard AdminCentral (the larger heap up to 10GB has been granted as well). 5.5.3 did not perform that well at all, falling under huge response times, whereas 5.5.4 more or less survived (although some major sluggishness has been observed).