Skip to Content.
Sympa Menu

grouper-users - [grouper-users] RE: Load testing the Grouper UI with JMeter

Subject: Grouper Users - Open Discussion List

List archive

[grouper-users] RE: Load testing the Grouper UI with JMeter


Chronological Thread 
  • From: "Hyzer, Chris" <>
  • To: "Redman, Chad Eric" <>, "" <>
  • Subject: [grouper-users] RE: Load testing the Grouper UI with JMeter
  • Date: Thu, 19 Jan 2017 19:14:44 +0000
  • Accept-language: en-US
  • Authentication-results: spf=none (sender IP is ) ;
  • Ironport-phdr: 9a23:TcEo1xA4F//P2vb7d7z7UyQJP3N1i/DPJgcQr6AfoPdwSP36o86wAkXT6L1XgUPTWs2DsrQf2raQ7/mrADVcqb+681k6OKRWUBEEjchE1ycBO+WiTXPBEfjxciYhF95DXlI2t1uyMExSBdqsLwaK+i764jEdAAjwOhRoLerpBIHSk9631+ev8JHPfglEnjSwbLd9IRmsogjdq8YajZZ/Jqs+1xDEvmZGd+NKyG1yOFmdhQz85sC+/J5i9yRfpfcs/NNeXKv5Yqo1U6VWACwpPG4p6sLrswLDTRaU6XsHTmoWiBtIDBPb4xz8Q5z8rzH1tut52CmdIM32UbU5Uims4qt3VBPljjoMOiUn+2/LlMN/kKNboAqgpxNhxY7UfJqVP+d6cq/EYN8WWXZNUsNXWidcAI2zcpEPAvIBM+hGsof9u1UAoxiwBQauBePg1jBGiXDt0K0myuQtDR3K0BA8E94TtnnfsdX7NL0VUeCw1KTEwzfNbvNX2Djj8ojEag0qrO+MXbJqcMrR0kcjHB7Cg1WVtYPlPzOU1+UTvGiB9OdgVPmvhHI9pwFsujig2MEsio/Tio0L11/E6Dx0zYAoLtO2T057ZMSrEJpWtyyCN4t2Xt8iQ3lztykg0b0GuIK0czIXyJQ7xx7QdfqHc5SJ4h3/TumRPTZ4hHVjeLOxnRmy6lWgyvbiWcapylpKryxFnsHRuXAKyxPT6daISuFn8ki93jaP0hjf5ftcIU8uiaXbK4AuzqQ3lpoJvkTPBi/2mFnqjK+OdkUr5PKn5Pn5bbX8vZ+QLZN0igb4P68zmcK/Gfw1PhYUUGSH5Oix0aDv8VDkTLhPgP07nbXVvIzfKMkZuKG0DA9Y34Mm5hqhEzur3swUkWQJIV5ZYh6Ki5XlN0vQLPzlCfqzmUmgnTVqyv3AI7bvGI/CLmLZn7fkZbt961BTyA40zd1H/5xZFrYPLO7tVkPoqtHXDAI1PxWzw+n8FtpxzIQeWX+TAqCCN6PSrFmI6f81L+mUfo8Vvyr9JOY56P7yjH85nlkdcbOu3ZsKdHC4GvNmI0KaYXb2ntgBFmIKshI/TOzsllKCTSZea2ioUK4g+z02DZ+qAZreSo2ojryM0zu3EoFTa21EFF+BHnfleoCBVvsSdC6fLNdtkjkeWrigT48h2wuutAj/y7d/K+rb5i0YtJLi1ddr/e3ejhAy9SdoAMiHzmGNV397nmUORzMswq9wvFF9xk+Z0adkm/xYCcBT5/RRXwc1L57cyPF6C8jsVQLbZ9uJVU2mTcu8ATEqVd8xx94OY11hG9W5kBzPxSuqA7kJl7OVHpw09LzT32TvK8pn1XnJybQh3BEaRZ4FD2Cqg6d5s0D4B4fF22iDlq3gPfAX3CfBw32OwW+HumlFVgU2XKnYCzRXLETMqsnh61mHUqSjE68PMw1dxNSEJ7cQLNDlkB8OEPj5P8nGbnj0hny9Hw2gx7WQYZDsdnlHmijRFR5XvRoU+CPMFRksCz3l60neFj11Xxq7Zkjs4Ptzslu6VURy0hmHaUsn2raorE1GzceAQu8ei+pX8BwqrC95SRPkh4rb
  • Spamdiagnosticmetadata: NSPM
  • Spamdiagnosticoutput: 1:99

That's great Chad. Thanks so much for doing this work and sharing the
results.

Do you think it would be helpful to put some of this on a wiki for others to
find?

Overall do you consider performance to be acceptable, i.e. the test is a
pass? No jiras about performance?

Yes, large datasets are slower than small data sets.

Wheel has better performance than non wheel due to privilege checking as you
mentioned.

Folder tree browsing is slow, but I don't think its impractical right? (I
think of impractical as over ~15 seconds)

Searching for subjects looks at groups (since groups are subjects), and when
adding in privileges it gets tedious for the database.

Thanks
Chris




-----Original Message-----
From:


[mailto:]
On Behalf Of Redman, Chad Eric
Sent: Thursday, January 19, 2017 1:50 PM
To:

Subject: [grouper-users] Load testing the Grouper UI with JMeter


During our process of upgrading Grouper to the latest version, we wanted to
do some load testing in a pre-preproduction environment. Since our
non-production installs are lightly used, our primary aim was to exercise the
application, Java container, and web server under a light but steady load, as
a predictor of how the application will fare in production. A secondary
purpose was to get timing results for the various features, comparing them
with the previous version, which for us was the quite old 1.6.3. Stress
testing wasn't a goal, as the UI for our production install isn't used that
heavily.

For our load tests, we used Apache JMeter 3.0 from http://jmeter.apache.org/.
Tests were done from a Windows 7 desktop. The specs for our institutional
Grouper installation are:

- Apache web server, Glassfish 3.1.2 Java container (2-server cluster)
- Oracle 11 database
- Grouper 2.3.0 with patches through api #22

For comparison, we also ran the same load test against Unicon's Grouper
Docker container (https://hub.docker.com/r/unicon/grouper-demo/, version
2.3.0-2017-01-08), running from the same Windows 7 desktop.

JMeter Implementation

The JMeter test plan uses a few different data sources in order to vary the
tests during multiple loops. A login source specifies usernames and passwords
for administrator, privileged user (e.g., has admin access to certain
groups), and normal users. A folder id source lists the UUID values for a
small number of stems, to simulate expanding folders in the directory tree
browser. A group and member source lists groups, stems, and users by UUID to
test calls to access various content.

Our institutional server uses Shibboleth SSO for authentication. When testing
this server, a one-time step per thread loop does the initial SSO session
setup, extracting session parameters as necessary. For the Docker
application, HTTP Basic authentication was used. In this case, a BeanShell
step Base64 encodes the username and password to include in the HTTP headers.
To switch from one authentication method to the other, the pertinent steps
are enabled and disabled as appropriate.

Because the new UI includes CSRF protection, it's necessary to include the
value for OWASP_CSRFTOKEN for most of the requests. At the start of each
thread loop, the value for OWASP_CSRFTOKEN is extracted from the first page
that is accessed. This value is then included in subsequent pages where it's
required.

The pages and Ajax requests accessed in each loop are:

1. Initial landing page
2. Directory browser (lower left) - root level
3. Home page, main content area
4. Directory tree, opening various folders (loop)
5. My groups
6. My folders
7. My favorites
8. Search result (the upper right corner)
9. View group
10. View stem
11. View users
12. Add member autocompletion result (after 2 chars) - Dojo Ajax return HTML
13. Trace membership
14. View group privileges
15. Trace privileges
16. Group memberships in other groups
17. View group audit log (if user is in wheel group)

Results

Non-Production Institutional server

Function | msec (wheel user) | msec (privileged user)
---------+-------------------+-----------------------
Home Page authorized         | 43  | 56
Directory Browser frame - root folder | 50 | 1506
Content frame - home         | 305 | 240
Open Stem in Directory Tree  | 48  | 1523
Content frame - my groups    | 185 | 81
Content frame - my folders   | 59  | 586
Content frame - my favorites | 74  | 87
Content frame - search result (upper right) | 163 | 3516
Content frame - view group   | 649 | 483
Content frame - view stem    | 107 | 2531
Content frame - view user    | 282 | 1562
Content frame - add member autocomplete (after 2 chars) | 626 | 866
Content frame - trace membership | 616 | 1741
Content frame - view group privileges | 189 | 168
Content frame - group memberships in other groups | 122 | 122
Content frame - view group audit log | (not tested)

Note: Tracing privileges was not performed, as the installation did not
include the fix from patch #38, and the request timed out after 2 minutes.

Docker Image
Function | msec (wheel user) | msec (privileged user)
---------+-------------------+-----------------------
Home Page authorized             | 26 | 19 | 17
Directory Browser frame - root folder | 38 | 105 | 94
Content frame - home             | 88 | 61 | 60
Open Stem in Directory Tree      | 27 | 91 | 92
Content frame - my groups        | 84 | 28 | 32
Content frame - my folders       | 37 | 57 | 59
Content frame - my favorites     | 15 | 21 | 16
Content frame - search result (upper right) | 92 | 175 | 182
Content frame - view group       | 285 | 25 | 88
Content frame - view stem        | 87 | 178 | 175
Content frame - view user        | 101 | 54 | 72
Content frame - add member autocomplete (after 2 chars) | 33 | 45 | 68
Content frame - trace membership | 211 | 21 | 61
Content frame - view group privileges | 312 | 23 | 23
Content frame - trace privileges (note 1) | 1374 | 27 | 25
Content frame - group memberships in other groups | 126 | 37 | 44
Content frame - view group audit log | 430 | 23 | 32

(note 1) Also tested with the 2.3.0-2016-12-13 Unicon Docker image, where the
time for wheel user was 1767 msec instead of 1374. This build was from before
API patch #38, a patch which addressed performance issues with privilege
tracing.

Comparison with existing production version 1.6.3

Function | msec (wheel user) | msec (privileged user) || v1.6.3 (wheel user) 
| v1.6.3 (privileged user)
---------+-------------------+-----------------------
SSO Login                              | 85  | 98   ||   78 | 56
SP Start Session                       | 41  | 39   ||   41 | 36
Content frame - home                   | 305 | 240  ||   182 | 155
Open Stem in Directory Tree            | 48  | 1523 ||   152 | 186
Content frame - search result (note 1) | 163 | 3516   ||   14525 | 25639
Content frame - view group (note 2)    | 649 | 483  ||   98 | 50
view group list members                |  -  |  -   ||  561 | 37
Content frame - view stem              | 107 | 2531  ||   179 | 480
Content frame - view user              | 282 | 1562  ||   340 | 314
Content frame - trace membership       | 616 | 1741  ||   147 | 38
Content frame - view group privileges  | 189 | 168   ||   151 | 38
Content frame - group memberships in other groups | 122 | 122   ||   66 | 60

(note 1) Changes in our sources.xml configuration led to great improvement in
search time between 1.6.3 and 2.3.0
(note 2) The group page in the v2 new UI includes the list of group members
in the group page, whereas the membership listing is a separate page in 1.6.3.

Analysis

For some pages, there is a large difference between the Docker image and our
installation. This is likely due to the much larger data set: We have 23,000
group, 168,000 members, and 845,000 memberships in our development
installation, while the Docker setup has 43 groups, 1044 members, and 5175
memberships.

Some of the unusually low response times for non-wheel users are not due to
good performance, but rather represent an error response due lack of access.
This is especially true for the unprivileged user, which explains why some of
the responses are faster for an unprivileged user than for a privileged one.

Some functions show a large difference between wheel and non-wheel access.
This is likely due to the additional privilege checking performed for users,
which is bypassed for wheel members. The biggest differences are in the
folder tree browser (in the lower left of the page), viewing stems and users,
tracing memberships, and viewing the list of search results. The slowness is
particularly noticeable in the folder tree browser, where it's slow to the
point of being impractical for users as a way to navigate to a group. In the
Docker installation, differences can be seen between wheel and users for most
of the same functions, but the difference is less pronounced.

In the past, we had looked into the folder tree performance issue. Adding
additional Oracle indexes would improve the speed by about 33%, but the
response per folder was still over 1 second. Note that this response is with
the Grouper setting security.show.folders.where.user.can.see.subobjects =
false. Without this setting, the response would be around 3 seconds per
folder list.

The speed difference for a general subject search between wheel and normal
users is also notable. We did not do much further analysis to understand the
cause. It does not seem strongly correlated with the number of results; even
queries that return zero subjects take over 1.5 seconds to return the
response for a non-wheel user.

Comparing a v1.6.3 release with similar data, not all of the new UI functions
mapped well to pages in the older UI. Where there were similar functional
pages between versions, there was a similar performance in most pages for a
wheel user. Viewing a stem or user was faster. Viewing the home page or a
group's membership in other groups was slower, but still in a sub-300
millisecond response that would not be very noticeable by the user. Tracing a
membership was significantly slower in 2.3.0, going from 150 msec to 600+
msec. Viewing a group was also slower; however, the 1.6.3 version kept group
data and membership in separate pages. When combining the responses of the
group page plus the membership listing, the timing is comparable to the group
page of 2.3.0 which includes membership. For non-wheel users between 1.6.3
and 2.3.0, there are remarkable differences in response time, but these have
all been mentioned above as being specific to privilege checking in the new
UI. What is notable is that there were not significant differences in
response times between wheel and non-wheel users in 1.6.3, whereas now there
are.

A zip file of the JMeter test plan is attached, which includes sample data
that works against the QuickStart demo application, and shell scripts to run
the test in headless mode.

-Chad




Archive powered by MHonArc 2.6.19.

Top of Page