grouper-dev - Re: [grouper-dev] gsh scripting operations - questions about API

Subject: Grouper Developers Forum

List archive

Re: [grouper-dev] gsh scripting operations - questions about API

From: Julien Gribonvald <>
To:
Subject: Re: [grouper-dev] gsh scripting operations - questions about API
Date: Wed, 22 Sep 2010 12:13:19 +0200

Hum... I answered too fast.... Finaly it's not working. Here is my script

grouperSession = GrouperSession.startRootSession();
stem = StemFinder.findByName(grouperSession, "esco:Etablissements");
System.out.println("users with privileges : " + stem.getStemmers());

showCreators (Stem stem) { System.out.println("====Creators : ") ; for (creator : stem.getCreators()) {System.out.println(creator.getName());}}

showStemmers (Stem stem) { System.out.println("====Stemmers : ") ; for (stemmer : stem.getStemmers()) {System.out.println(stemmer.getName());}}

stemracine = new ArrayList(stem.getChildStems(Stem.Scope.ONE));

for (stem : stemracine){stemList = new ArrayList(stem.getChildStems(Stem.Scope.SUB));Collections.sort(stemList);Collections.reverse(stemList);for(childStem : stemList) { System.out.println("childStem: " + childStem.getName()); System.out.println("users with privileges : stemmers = " + stem.getStemmers().size()+ " creators = " + stem.getCreators().size()); showCreators(stem);showStemmers(stem);}}

And as result I obtain (a little exemple) :

childStem: esco:Etablissements:JEHAN DE BEAUCE_0280044W:groupes_locaux
users with privileges : 11
====Creators :
GrouperSysAdmin
====Stemmers :
GrouperSysAdmin

When the Grouper UI returns :

for creators and the same result for stemmers.

Thanks
Julien

Le 22/09/2010 11:27, Julien Gribonvald a écrit :

Ok, so I tested again today and my script worked, so like you told it might be a caching issue, I do nothing on database so...

Thanks for your help.

Julien

Le 22/09/2010 06:13, Chris Hyzer a écrit :

Can you show me an example? Might be a caching issue... Here is my example that shows it working:

gsh 0% grouperSession = GrouperSession.startRootSession();
edu.internet2.middleware.grouper.GrouperSession: 4dee333f98214723acd63c481436da2a,'GrouperSystem','application'
gsh 1% stem = new StemSave(grouperSession).assignCreateParentStemsIfNotExist(true).assignName("test:testStem").save();
stem: name='test:testStem' displayName='test:testStem' uuid='1c25aab533a04f5d9a0eaafb78668c07'
gsh 2% subject0 = SubjectFinder.findById("test.subject.0", true);
subject: id='test.subject.0' type='person' source='jdbc' name='my name is test.subject.0'
gsh 3% subject1 = SubjectFinder.findById("test.subject.1", true);
subject: id='test.subject.1' type='person' source='jdbc' name='my name is test.subject.1'
gsh 4% group = new GroupSave(grouperSession).assignCreateParentStemsIfNotExist(true).assignName("test:testGroup").save();
group: name='test:testGroup' displayName='test:testGroup' uuid='c289e19e43e6425c86ceb8aa8f137d13'
gsh 5% group.addMember(subject1);
gsh 6% stem.grantPriv(subject0, NamingPrivilege.CREATE);
gsh 7% stem.grantPriv(group.toSubject(), NamingPrivilege.CREATE);
gsh 8% stem.getCreators();
subject: id='c289e19e43e6425c86ceb8aa8f137d13' type='group' source='g:gsa' name='test:testGroup'
subject: id='test.subject.0' type='person' source='jdbc' name='my name is test.subject.0'
subject: id='test.subject.1' type='person' source='jdbc' name='my name is test.subject.1'
gsh 9% subject2 = SubjectFinder.findById("test.subject.2", true);
subject: id='test.subject.2' type='person' source='jdbc' name='my name is test.subject.2'
gsh 10% subject3 = SubjectFinder.findById("test.subject.3", true);
subject: id='test.subject.3' type='person' source='jdbc' name='my name is test.subject.3'
gsh 11% group2 = new GroupSave(grouperSession).assignCreateParentStemsIfNotExist(true).assignName("test:testGroup2").save();
group: name='test:testGroup2' displayName='test:testGroup2' uuid='79f2c84f53ff4b5ca8a20a49263e1df4'
gsh 12% group2.addMember(subject2);
gsh 13% stem.grantPriv(subject3, NamingPrivilege.STEM);
gsh 14% stem.grantPriv(group2.toSubject(), NamingPrivilege.STEM);
gsh 15% stem.getStemmers();
subject: id='79f2c84f53ff4b5ca8a20a49263e1df4' type='group' source='g:gsa' name='test:testGroup2'
subject: id='GrouperSystem' type='application' source='g:isa' name='GrouperSysAdmin'
subject: id='test.subject.3' type='person' source='jdbc' name='my name is test.subject.3'
subject: id='test.subject.2' type='person' source='jdbc' name='my name is test.subject.2'
gsh 16%

-----Original Message-----
From: Julien Gribonvald []
Sent: Friday, September 17, 2010 8:37 AM
To: Chris Hyzer
Cc:
Subject: Re: [grouper-dev] gsh scripting operations - questions about API

Hi Chris,

Now I'm trying to do the same thing on stems, but I tried at least with
stem.getCreators() and stem.getStemmers() and it returns only the
GrouperSystem user when the Grouper UI return GrouperSyteme user and all
groups which have privileges, do you know why ?

thanks

Julien

Le 15/09/2010 18:35, Chris Hyzer a écrit :

Well, if we want to add something to Grouper 2.0 we can. For now, you could do SQL (I know you said you don’t want to, but that is where we are, and there are no plans for this to break anytime soon). Here is an example:

gsh 1% grouperSession = GrouperSession.startRootSession();

gsh 2% readerMemberIds(Group group, String listName) {list = new ArrayList(); list.add(listName); list.add(group.getId()); return HibernateSession.bySqlStatic().listSelect(String.class, "select distinct gmav.member_id from grouper_memberships_all_v gmav, grouper_fields gf where gmav.MSHIP_TYPE = 'immediate' and gmav.field_id = gf.ID and gf.TYPE = 'access' and gf.NAME = ? and gmav.OWNER_GROUP_ID = ?", list);}

gsh 3% group = GroupFinder.findByName(grouperSession, "test:testGroup", true);

gsh 4% printReaders(GrouperSession grouperSession, Group group) {for (memberId : readerMemberIds(group, "readers")) {member = MemberFinder.findByUuid(grouperSession, memberId, false); System.out.println(member.getSubject().getName()); }}

gsh 5% printReaders(grouperSession, group);
EveryEntity
gsh 6% grantPriv("test:testGroup", "10021368", AccessPrivilege.READ);
true
gsh 7% printReaders(grouperSession, group);
Michael Christopher Hyzer
EveryEntity

Thanks,
Chris

-----Original Message-----
From: Julien Gribonvald []
Sent: Wednesday, September 15, 2010 4:43 AM
To: Chris Hyzer
Cc:
Subject: Re: [grouper-dev] gsh scripting operations - questions about API

Hi Chris,

So after an Analyzing on all tables containing data the result is the
same (we are using Mysql), nothing change, it takes always more than 30
min to pass my script on all groups that we have. For me the slower
request is when I do a GrouperHelper.hasSubjectImmPrivForGroup(...).
Maybe my script isn't working with good methods, but in this case how
can I do this checking ?

Thanks

Julien

Le 14/09/2010 19:11, Chris Hyzer a écrit :


Thanks for sharing.

I mean, please analyze your tables, then run the script that is slow that you need to be faster (you said get groups someone has). If it is still slow, send me that script (you said it is slow to get all groups someone has).

Thanks,
Chris

-----Original Message-----
From: [] On Behalf Of Julien Gribonvald
Sent: Tuesday, September 14, 2010 1:07 PM
To:
Subject: Re: [grouper-dev] gsh scripting operations - questions about API

I have done this GSh script, if you want to watch it :

grouperSession = GrouperSession.startRootSession();
stem = StemFinder.findByName(grouperSession, "esco");
System.out.println("users with privileges : " + stem.getStemmers());

showAdmins (Group group) { System.out.println("====Admins : ") ; for
(admin :
GrouperHelper.getSubjectsWithPriv(group,AccessPrivilege.ADMIN.getName())
) { if
(GrouperHelper.hasSubjectImmPrivForGroup(grouperSession,admin,group,AccessPrivilege.ADMIN.getName())){System.out.println(admin.getName());
} /*else {System.out.println("===Not immediate : " +
admin.getName());}*/} };

showReaders (Group group) { System.out.println("====Readers : ") ; for
(reader :
GrouperHelper.getSubjectsWithPriv(group,AccessPrivilege.READ.getName()))
{ if
(GrouperHelper.hasSubjectImmPrivForGroup(grouperSession,reader,group,AccessPrivilege.READ.getName())){System.out.println(reader.getName());
}/*else {System.out.println("===Not immediate : " +
reader.getName());}*/} };

showViewers (Group group) { System.out.println("====Viewers : ") ; for
(viewer :
GrouperHelper.getSubjectsWithPriv(group,AccessPrivilege.VIEW.getName()))
{ if
(GrouperHelper.hasSubjectImmPrivForGroup(grouperSession,viewer,group,AccessPrivilege.VIEW.getName())){System.out.println(viewer.getName());
}/*else {System.out.println("===Not immediate : " +
viewer.getName());}*/} };

showUpdaters (Group group) { System.out.println("====Updaters : ") ; for
(updater :
GrouperHelper.getSubjectsWithPriv(group,AccessPrivilege.UPDATE.getName()))
{ if
(GrouperHelper.hasSubjectImmPrivForGroup(grouperSession,updater,group,AccessPrivilege.UPDATE.getName())){System.out.println(updater.getName())
; }/*else {System.out.println("===Not immediate : " +
updater.getName());}*/} };

for(child : stem.getChildGroups(Stem.Scope.SUB))
{System.out.println("childName " + child.getName()); showAdmins(child);
showReaders(child); showViewers(child);
showUpdaters(child);System.out.println("============================="); }

"esco" is the root branch, under this branch we have 3 branch, one for
admin rigths on 5 applications, so not too much groups (one per
organization and per applications), one branch for applications access
(so around 10 applications where there is on group per organization),
and the last branch is organisation with all repartition of users,
around 170 groups per organisation in this branch, we have a bit less
than 10000 groups to check now.

This script is only to show rigths on all groups, it's only the
beginning of my script for checking all rigths and to print it on an
output file for example.

Before that i look for checking immediate rigths or not I used
group.getAdmin() ... but this wasn't telling me immediate or not, but it
took only 5/10 minutes, unlike this script wich is taking around 30
minutes, the slow code is when asking
GrouperHelper.hasSubjectImmPrivForGroup, i tested without it and it was
OK before.

But like I told you we will use such script only few times per years, so
there is no real need now. But maybe we will pass it one time per week
or month in some times to check all, I can't tell you...

Thanks
Julien

Le 14/09/2010 18:38, Chris Hyzer a écrit :


Ok, no problem. Try it after analyzing tables and let me know how it goes.

If it is still a problem, please give me which GSH commands are slow. I assume you mean just typing into a GSH shell, and now waiting for GSH to startup...

Thanks,
Chris

-----Original Message-----
From: [] On Behalf Of Julien Gribonvald
Sent: Tuesday, September 14, 2010 12:24 PM
To:
Subject: Re: [grouper-dev] gsh scripting operations - questions about API

I'm using GSH will grouper API methods only and with GrouperSystem user,
I can do all I want but I would prefer to avoid to make request on
tables directly.

For now we don't have plans for an upgrade to 1.6.3, we will continue to
works with grouper 1.5.2 (with 1.5.3 upgrades) at least until january.
We have too many works at the beginning of a new year school so we
prefer to avoid to works on a migration of a critical application
(critical because we need to have a good acknowledge of versions that we
have in production and more we need really good data's at the beginning
because grouper take a big place in our schema). Also there is a certain
cost to maintain the ESCO Grouper UI, so we can't follow all versions.
Also we have more times to make migrations after january.

Thanks,

Julien

Le 14/09/2010 18:06, Chris Hyzer a écrit :


How are you exactly checking all groups that you have? Is it as groupersystem or as another user? Can you analyze all your tables and try again? Just curious, what are your upgrade plans for 1.6.3?

Doc on analyzing tables:
https://spaces.internet2.edu/pages/viewpage.action?pageId=11076532#APIBuilding%26Configuration-AnalyzingTablestoImproveQueryPerformance

Thanks,
Chris

-----Original Message-----
From: Julien Gribonvald []
Sent: Tuesday, September 14, 2010 12:03 PM
To: Chris Hyzer
Cc: GW Brown, Information Systems and Computing;
Subject: Re: [grouper-dev] gsh scripting operations - questions about API

Thank you Chris,

What i'm doing is not something that we will do each days, so there is
no real need, the only things is that it's slow when we have to check
all groups that we have, and for the moment we have only a small part of
our perimeter (only 9000 groups), and I think we will up to 50000 groups.

Thanks,
Julien

Le 14/09/2010 16:41, Chris Hyzer a écrit :


There is no method in the AccessAdapter, so there is no way to efficiently do this in one query. So if the strategy with two methods you are doing performs well enough (N+1 at least queries), then go for it. Otherwise we could add a method to AccessAdapter (though you would need an upgrade)...

Thanks,
Chris

-----Original Message-----
From: [] On Behalf Of Julien Gribonvald
Sent: Tuesday, September 14, 2010 5:55 AM
To: GW Brown, Information Systems and Computing
Cc:
Subject: Re: [grouper-dev] gsh scripting operations - questions about API

Thank you Gary,

Finaly I use the two method, the first to get users with privilegies and
yourth to know if the user has immediates rigths, but maybe you have a
better way to get users with privilegies on a group ?

Thanks
Julien

Le 14/09/2010 11:15, GW Brown, Information Systems and Computing a écrit :


Hi Julien,

I'm not sure that method really does what you want - it just calls
Group.getAdmins() etc - and does not filter immediate vs indirect.

There is another method which can be applied to each subject:

public static boolean hasSubjectImmPrivForGroup(GrouperSession
s,Subject subject,Group group,String privilege) throws
MemberNotFoundException,SchemaException{

However, Chris may have a better way to do it. I wrote many of the
GrouperHelper methods to work around things the official API did not
do, however, there have been many API changes and GrouperHelper may
not have kept pace.

Gary

--On 14 September 2010 11:00 +0200 Julien Gribonvald
       wrote:


Sorry, finaly I found my needs in the GrouperHelper classe with the
method getSubjectsWithPriv(Group group, String privilege)

Thanks,

Julien Gribonvald
GIP RECIA

Le 14/09/2010 10:48, Julien Gribonvald a écrit :

Hi all,

I'm looking for scripting some checking and administration operations
with GSH, I have already done some script wich are checking and updating
our tree of groups structure.
But I have a little problem with the API, found here :
http://www.internet2.edu/grouper/release/1.5.0/doc/api/index.html (we
are
using grouper 1.5.2, with 1.5.3 updates), I can't find an easy way to
know, on a given group, if a subject has immediate privileges or herited
from a group. For exemple, currently, on a Group g I do like that to get
users with privilegies :

g.getReaders(); g.getViewers();g.getUpdaters();g.getAdmin();

These methods are good to obtain Subjects, but we don't know if they
have
immediate privileges or not. My wisch would be to get only immediate
subjects which have a Privilege P on a Group G.

So is there someone who could give me some informations, like the good
class to watch and/or where I will find my solution ? Or any idea on how
to do that ?

Thanks for your help.

Julien Gribonvald
GIP RECIA


----------------------
GW Brown, Information Systems and Computing

RE: [grouper-dev] gsh scripting operations - questions about API, (continued)
- [grouper-dev] Re: [paccman] grouper rules update, Tom Zeller, 09/15/2010
  - [grouper-dev] RE: [paccman] grouper rules update, Chris Hyzer, 09/16/2010
    - [grouper-dev] Re: [paccman] grouper rules update, Tom Zeller, 09/17/2010
      - [grouper-dev] RE: [paccman] grouper rules update, Chris Hyzer, 09/17/2010

List archive

Re: [grouper-dev] gsh scripting operations - questions about API