Grouper Developers Forum

[Emily Eisbruch <>]

Draft Minutes: Grouper call of May 7, 2014

Attending

Chris Hyzer, University of Pennsylvania

Shilen Patel, Duke

Emily Eisbruch, Internet2, scribe

======================

New Action Items

[AI] (Shilen) look at caching JEXL expressions in Grouper Util.

[AI] (Shilen) look at the performance of the main Grouper 2.2 UI page, and continue investigating other performance issues

{AI] (Dave) decide on how to get the new Hibernate jar and use it with Maven

Carry Over Action items

[AI] (Dave) update the Release Steps checklist for PSP/ connector steps

https://spaces.internet2.edu/display/Grouper/Release+steps

[AI] (Tom) ask U. Chicago security staff if they can do Penetration testing on the Grouper Demo

[AI] (Shilen) create a Grouper training video on the new Grouper UI

[AI] (DaveL) move the SCIM code to SVN. 

[AI] (Shilen) will ask Scott Koranda to review the upgrade process. 

[AI] (Jim) make the University of Washington group events AD updating code available on GitHub and document its availability  on the Community Contributions page.

https://spaces.internet2.edu/display/Grouper/University+of+Washington+Grouper+Project+Page

   and in table at bottom of this page:

https://spaces.internet2.edu/display/Grouper/Community+Contributions

[AI) (DaveL) look at PSP ChangeLogDataConnector Inconsistency issue

https://lists.internet2.edu/sympa/arc/grouper-users/2014-02/msg00047.html

[AI] (DaveL) write up the doc related to the Shib Grouper question.https://lists.internet2.edu/sympa/arc/grouper-users/2014-02/msg00004.html 

[AI] (DaveL) work on the PSP aspect of GRP 914.  

[AI] (Shilen) investigate ways to get new attribtues in a single step (after 2.2 release)

========
Grouper wiki: https://spaces.internet2.edu/display/Grouper/Grouper+Wiki+Home
========

DISCUSSION

Always Available Web Services

https://spaces.internet2.edu/display/Grouper/Grouper+always+available+web+services+and+client

-Chris reported that University of Pennsylvania now has an offsite database that is used to replicate the Grouper database in read-only mode.

-There is an app server off site, running Grouper web services in read-only mode

-There is DNS-based load balancing that checks the primary to see it is alive, and if not, it will change the DNS of a domain name over to the secondary.

-There is also load balancing in the Grouper Client

-Chris is doing a presentation on this topic for Penn at end of May. He will put the presentation on the Grouper wiki and send a link to the list.

Grouper Performance improvements

Shilen recently worked on caching JEXL expressions in the Subject API to improve performance. This involves creating one instance of the JEXL engine instead of a new instance every time.

https://bugs.internet2.edu/jira/browse/GRP-972

Shilen will look at caching of JEXL expressions in Grouper Util also. 

[AI] (Shilen) look at caching JEXL in Grouper

It was noted that it is challenging to work with custom jars in Maven compared with SVN.

[AI] (Dave) decide on how to get the new Hibernate jar and use it with Maven

[AI] (Shilen) look at the performance of the main Grouper 2.2 UI page, and continue investigating other performance issues

Upgrade Issues

Shilen noted that when running through the registry check, the upgrade process creates lines for the table index change, and as it creates each line, it checks to see if the script has the line already (string.indexof).  This can take a long time if there are many rows.

Would it be OK if that part of the check was not in there, or would this lead to potential duplicates?

Suggestion to use the approach of appending the lines, and keeping  a set of all the lines you've added.  Then do a check to see if something in the set is a duplicate. (If it is a duplicate, then don't add it)

The upgrade to Grouper 2.2 involves several elements:

-Database changes and table index

-legacy attribute migration 

-new privileges and group sets

-stem set table

Can require time to do upgrade, and there can be a problem of having an "outage" of Grouper during that time.

Could some of these upgrade processes run at the same time? Yes, that is worth exploring.

Possibly explore using multiple threads as well.

A large number of groups and group sets being added can cause a memory shortage issue when doing a query.

One solution is to use scrollable result set.  This means more round trips to the database, but avoids the memory issue.

Better approach: run the query, but get only 100K rows at a time, and use a loop. 

Unit Testing

Chris is working on unit testing. Still has some issues to work out. 

Then Chris will go through more of the release steps.

https://spaces.internet2.edu/display/Grouper/Release+steps

Next Grouper-dev call : Wed., May 21, 2014  at noon ET

 

 

Emily Eisbruch, Technology Transfer Analyst

Internet2

office: +1-734-352-4996 | mobile +1-734-730-5749