grouper-dev - [grouper-dev] Draft Minutes: Grouper Call 21-July-2010
Subject: Grouper Developers Forum
List archive
- From: Emily Eisbruch <>
- To: Grouper Dev <>
- Subject: [grouper-dev] Draft Minutes: Grouper Call 21-July-2010
- Date: Mon, 26 Jul 2010 16:05:52 -0400
Grouper Call 21-July-2010 **Attending** Chris Hyzer, U. Penn (stand-in chair) Shilen Patel, Duke Gary Bristol, Bristol Jim Fox, University of Washington Tom Zeller, U. Memphis Steve Olshansky, Internet2 Emily Eisbruch, Internet2 (scribe) **Action Items** *New Action Items* [AI] (Shilen) will develop a test case for groups with thousands of members and add it to the test suite. [AI] (Shilen) will update the JIRA issue for GRP-458 (Manage Groups in UI should perform well when user is admin of many groups) *Carry Over Action Item* [AI] (Jim) will do performance testing to help evaluate potential issues with the flattened memberships table. [AI] (Rob) will look at issues relating to testing the ESB Connector. **Discussion** *Error Deleting a Group* Jim had reported a problem on the email list with deleting a group, and this problem is continuing. Shilen stated that when a large group with is deleted, there will be a lot of immediate membership deletions involved (including a change log entry). This is all part of one transaction, and it could possibly cause problems if the database gets so big it can’t hold all that data. Jim noted the problem occurs in deleting groups with a few thousand members. If the cause is in fact that the database gets too big, then what will happen with deleting groups with 100,000 members? It was agreed that in terms of stress on the database the operation really should not cause trouble. To understand the issue further, Jim will try deleting each member individually (using a batch method) and then deleting the group, to see if the problem still occurs. Other things Jim may try: - Increase the shared buffer space - Check to see if there are background processing causing trouble. It was suggested that the Grouper test suite should include a test for groups with thousands of members. It could take a while to run, so it should be an optional test. [AI] (Shilen) will develop a test case for groups with thousands of members and add it to the test suite. *Early Experience with Grouper 1.6* Jim reported that Grouper 1.6 is working well at U. Washington aside from the delete group issue. *Schedule and Scope of 1.6.1* Targets for Grouper 1.6.1 - Code freeze at end of Sunday, Aug 1. - Do light testing Aug. 2 and Aug. 3 - Release on Wed. Aug. 4 Brief review of some of a few of the Jira issues to be included: GRP 469: This Jira will provide another check on the registry, which will be helpful at Penn: GRP 466: “Put the username using UI/WS in log message from threadlocal” Gary will work on this. GRP 459: “postgres might not drop all views, we should check the order here or something... (cascade will drop dependent views)” It was suggested to eliminate the dependence for now if we can’t otherwise fix the problem. GRP 443: “If there is an exception when copying / moving stems/groups the user does not see an appropriate error message” Gary is working on this. It might be irreproducible. GRP 458: “Manage group in UI not performing well in user admin of many groups.” This JIRA issue is more substantial than the others. Problem is that if a person is an admin of 100,000 groups, the query will return 100,000 results. Also, for every membership, it runs Get-Groups and that causes an additional performance issue. Gary noted that we tried an alternative approach several years ago. But it didn’t work very well. Suggestion for a temporary solution : Do a count quickly to see how many groups an admin has. Then have a different behavior if an admin has many groups. The approach was discussed of just displaying folders or just stems for a person who is an admin of many groups. But there are still issues if there is a hierarchy with much nesting. [AI] (Shilen) will update the JIRA issue for GRP-458 (Manage Groups in UI should perform well when user is admin of many groups ) *Performance and Flattened Tables* Shilen’s performance testing found that if a group with 66,000 members is added to another group, it takes about 6 minutes for the change log daemon to process. It was agreed that this performance could be problematic in certain environments, including with the usage scenarios at U. Washington. Jim stated that one supposed advantage of using the flattened table is to quickly get all groups someone is part of (to get a person’s effective memberships). But Jim has found that people typically don’t want to know just a list of what groups they are a member of. They want to know the path by which they got to be a member, and the flattened table will not help with that. Another reason the flattened table was created was for point in time audit. There has not yet been a big need for that at U. Washington, though that could change in the future. A batch approach for flat membership updates (GRP 477) will be in Grouper 1.6.1 and may help performance somewhat. Still, it probably does not make sense to expand the use of flattened tables to handling of Roles and Permissions in Grouper v 2.0. With roles and permissions the size of tables could multiply quickly. Chris suggested approaches for handling the database for permissions: 1 Uncouple the data for roles and for permissions so we don’t store each permission for each member 2. Use a dynamic role approach, where people who have a certain attribute will be assigned to a role when they log in Next Meeting: Wed. Aug 4 at noon ET Emily Eisbruch, Technology Transfer Analyst Internet2 office: +1-734-352-4996 | mobile +1-734-730-5749 Visit our website: www.internet2.edu Follow us on Twitter: www.twitter.com/internet2 Become a Fan on Facebook: www.internet2.edu/facebook |
- [grouper-dev] Draft Minutes: Grouper Call 21-July-2010, Emily Eisbruch, 07/26/2010
Archive powered by MHonArc 2.6.16.