Grouper Users - Open Discussion List

[Wallaert-Taquet Brigitte <>]

Hello,

In fact, my administrator's ldap doesn't understand why I need to ask 
each people (1275 people), I show him the script below and now, he agree 
Grouper do that once each night. It's ok for me. In fact, we can't do 
simplier that this script below, isn't it ?

For information :
My script is now very simple :
grouperSession = GrouperSession.startRootSession();
groupetemoin = 
GroupFinder.findByAttribute(grouperSession,"ustlRole","ustlRoleTemoin",true);
for (Subject sujet : SubjectFinder.findAll("*")) { 
groupetemoin.addMember(sujet,false) ;  };

with modify in sources.xml for search :
<search>
<searchType>search</searchType>
<param>
<param-name>filter</param-name>
<param-value>
                (ustlRole=%TERM%)
</param-value>
</param>
<param>
<param-name>scope</param-name>
<param-value>
               SUBTREE_SCOPE
</param-value>
</param>
<param>
<param-name>base</param-name>
<param-value>
                ou=people,dc=univ-lille1,dc=fr
</param-value>
</param>
</search>

Now, Grouper ask for each entry ldap like that :

Oct 10 17:00:02 anubis slapd[14439]: conn=56335 fd=19 ACCEPT from 
IP=193.49.225.121:52652 (IP=0.0.0.0:389)
Oct 10 17:00:02 anubis slapd[14439]: conn=56335 op=0 BIND dn="" method=128
Oct 10 17:00:02 anubis slapd[14439]: conn=56335 op=0 RESULT tag=97 err=0 text=
Oct 10 17:00:02 anubis slapd[14439]: conn=56335 op=1 SRCH 
base="ou=people,dc=univ-lille1,dc=fr" scope=2 deref=3 
filter="(&(uid=morel)(objectClass=supannPerson))"
Oct 10 17:00:02 anubis slapd[14439]: conn=56335 op=1 SRCH attr=mail 
eduPersonPrimaryAffiliation supannAffectation supannCivilite ustlDepartement 
displayName
Oct 10 17:00:02 anubis slapd[14439]: conn=56335 op=1 SEARCH RESULT tag=101 
err=0 nentries=1 text=
Oct 10 17:00:02 anubis slapd[14439]: conn=56335 op=2 UNBIND
etc


Thanks a lot for your help !
Now, It's ok for my ldap's administrator, I wait for a database server 
in production (I hope soon...) and I put in production next week or 
following !!!....
Brigitte

Le 10/10/2011 20:07, Chris Hyzer a écrit :
> Ok, I fixed the problem (which is 1.6 specific) about the groupsave.  Also I 
> expanded this out so we can see where the problem is.  Note, if you know 
> where the subject comes from, and it is by id or identifier, you can change 
> SubjectFinder.findByIdOrIdentifier() to something else to make it more 
> efficient.  E.g.
>
> findByIdAndSource(id, sourceId, false);
>
> so look here:
>
> https://spaces.internet2.edu/display/Grouper/Grouper+LDAP+GSH+example
>
> And change the top part per your env, e.g.
>
> gsh 1% groupName = "lille1:utilitaires:ustlroletemoin";
> gsh 2% searchDn = "ou=people,dc=univ-lille1,dc=fr";
> gsh 3% searchFilter = "(ustlRole=*)";
> gsh 4% attributeName = "uid";
>
> Run each command below and let me know where the problem is exactly and what 
> your output and logs are...
>
> ############
> The question about the start queries and errors, you can set this in the 
> sources.xml
>
> findSubjectByIdentifiedOnCheckConfig to false
> findSubjectByStringOnCheckConfig to false
>
> to turn it off
>
> or you can set:
>
> subjectIdentifierToFindOnCheckConfig to someId that exists
> stringToFindOnCheckConfig to someId that exists
>
> To do the test, but it will not throw error :)
>
> Thanks,
> Chris
>
>
> -----Original Message-----
> From: Wallaert-Taquet Brigitte 
> [mailto:]
> Sent: Monday, October 10, 2011 5:18 AM
> To: Chris Hyzer
> Cc: Tom Zeller; 
>
> Subject: Re: [grouper-users] sources.xml ldap and script groupershell
>
> Hello,
>
> I have to modify 2 lines to have results :
> 1-
>       FROM
>             group = new
> GroupSave(grouperSession).assignName("lille1:utilitaires:ustlroletemoin").assignCreateParentStemsIfNotExist(true).save();
>       TO
>             group =
> GroupFinder.findByAttribute(grouperSession,"ustlRole","ustlRoleTemoin",true);
>
> because these errors :
> 2011-10-10 10:54:16,183: [main] INFO  EventLog.info(141) -  -
> [e29b74e53e0b424ba95163e5fef85585,'GrouperSystem','application']
> session: start (4ms)
> 2011-10-10 10:54:16,231: [main] ERROR ShellHelper.eval(76) -  - Sourced
> file: inline evaluation of: ``group = new
> GroupSave(grouperSession).assignName("lille1:utilitaires:ustlroletem . .
> . '' : Method Invocation save
> edu.internet2.middleware.grouper.exception.GroupAddAlreadyExistsException:
> group already exists with name: 'lille1:utilitaires:ustlroletemoin',
> stem name: lille1:utilitaires, group extension: ustlroletemoin, group
> dExtension: ustlroletemoin, uuid: null, typeOfGroup: null,
> Problem in HibernateSession: HibernateSession: isNew: false, isReadonly:
> false, grouperTransactionType: READ_WRITE_NEW,
> Problem in HibernateSession: HibernateSession: isNew: true, isReadonly:
> false, grouperTransactionType: READ_WRITE_NEW
>           at edu.internet2.middleware.grouper.Stem$5.callback(Stem.java:1760)
>           at
> edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:500)
>           at
> edu.internet2.middleware.grouper.Stem.internal_addChildGroup(Stem.java:1740)
>           at
> edu.internet2.middleware.grouper.Stem.internal_addChildGroup(Stem.java:1713)
>           at
> edu.internet2.middleware.grouper.Stem.internal_addChildGroup(Stem.java:1690)
>           at
> edu.internet2.middleware.grouper.GroupSave$1$1.callback(GroupSave.java:347)
>           at
> edu.internet2.middleware.grouper.GrouperSession.callbackGrouperSession(GrouperSession.java:644)
>           at
> edu.internet2.middleware.grouper.GroupSave$1.callback(GroupSave.java:251)
>           at
> edu.internet2.middleware.grouper.internal.dao.hib3.Hib3TransactionDAO$1.callback(Hib3TransactionDAO.java:51)
>           at
> edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:500)
>           at
> edu.internet2.middleware.grouper.internal.dao.hib3.Hib3TransactionDAO.transactionCallback(Hib3TransactionDAO.java:41)
>           at
> edu.internet2.middleware.grouper.hibernate.GrouperTransaction.callbackGrouperTransaction(GrouperTransaction.java:72)
>           at
> edu.internet2.middleware.grouper.hibernate.GrouperTransaction.callbackGrouperTransaction(GrouperTransaction.java:91)
>           at
> edu.internet2.middleware.grouper.GroupSave.save(GroupSave.java:244)
>           at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>           at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>           at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>           at java.lang.reflect.Method.invoke(Method.java:616)
>           at bsh.Reflect.invokeMethod(Unknown Source)
>           at bsh.Reflect.invokeObjectMethod(Unknown Source)
>           at bsh.BSHPrimarySuffix.doName(Unknown Source)
>           at bsh.BSHPrimarySuffix.doSuffix(Unknown Source)
>           at bsh.BSHPrimaryExpression.eval(Unknown Source)
>           at bsh.BSHPrimaryExpression.eval(Unknown Source)
>           at bsh.BSHAssignment.eval(Unknown Source)
>           at bsh.Interpreter.eval(Unknown Source)
>           at bsh.Interpreter.eval(Unknown Source)
>           at bsh.Interpreter.eval(Unknown Source)
>           at
> edu.internet2.middleware.grouper.app.gsh.ShellHelper.eval(ShellHelper.java:48)
>           at
> edu.internet2.middleware.grouper.app.gsh.GrouperShell.run(GrouperShell.java:404)
>           at
> edu.internet2.middleware.grouper.app.gsh.GrouperShell.grouperShellHelper(GrouperShell.java:216)
>           at
> edu.internet2.middleware.grouper.app.gsh.GrouperShell.main(GrouperShell.java:146)
>           at
> edu.internet2.middleware.grouper.app.gsh.GrouperShellWrapper.main(GrouperShellWrapper.java:16)
> )
>
> 2-
>
>     FROM get("pennname")
>     TO   get("uid")
> perhaps a mistake ?
>
> because error :
>
> 2011-10-10 11:11:56,604: [main] ERROR ShellHelper.eval(76) -  - Sourced
> file: inline evaluation of: ``while (results.hasNext())
> {group.addMember(SubjectFinder.findByIdOrIdentifier(re . . . ''
> java.lang.NullPointerException: Attempt to invoke method get on null value
>           at bsh.Reflect.resolveExpectedJavaMethod(Unknown Source)
>           at bsh.Reflect.invokeObjectMethod(Unknown Source)
>           at bsh.BSHPrimarySuffix.doName(Unknown Source)
>           at bsh.BSHPrimarySuffix.doSuffix(Unknown Source)
>           at bsh.BSHPrimaryExpression.eval(Unknown Source)
>           at bsh.BSHPrimaryExpression.eval(Unknown Source)
>           at bsh.BSHArguments.getArguments(Unknown Source)
>           at bsh.BSHMethodInvocation.eval(Unknown Source)
>           at bsh.BSHPrimaryExpression.eval(Unknown Source)
>           at bsh.BSHPrimaryExpression.eval(Unknown Source)
>           at bsh.BSHArguments.getArguments(Unknown Source)
>           at bsh.BSHMethodInvocation.eval(Unknown Source)
>           at bsh.BSHPrimaryExpression.eval(Unknown Source)
>           at bsh.BSHPrimaryExpression.eval(Unknown Source)
>           at bsh.BSHBlock.evalBlock(Unknown Source)
>           at bsh.BSHBlock.eval(Unknown Source)
>           at bsh.BSHBlock.eval(Unknown Source)
>           at bsh.BSHWhileStatement.eval(Unknown Source)
>           at bsh.Interpreter.eval(Unknown Source)
>           at bsh.Interpreter.eval(Unknown Source)
>           at bsh.Interpreter.eval(Unknown Source)
>           at
> edu.internet2.middleware.grouper.app.gsh.ShellHelper.eval(ShellHelper.java:48)
>           at
> edu.internet2.middleware.grouper.app.gsh.GrouperShell.run(GrouperShell.java:404)
>           at
> edu.internet2.middleware.grouper.app.gsh.GrouperShell.grouperShellHelper(GrouperShell.java:216)
>           at
> edu.internet2.middleware.grouper.app.gsh.GrouperShell.main(GrouperShell.java:146)
>           at
> edu.internet2.middleware.grouper.app.gsh.GrouperShellWrapper.main(GrouperShellWrapper.java:16)
>
> But my problem is always the same : my ldap's administrator doesn't
> agree because of 2 asks...
>
> So, I think to modify my sources.xml for goupershell script and to
> re-modify temporarly when I need the deploy again Grouper-API, in
> waiting to put Grouper 2.1 and use Grouper Loader LDAP.
>
> Another Question : there are 3 searchs with (uid=grouperTest....) at
> each connexion to ldap : is it necessary ?
>           2011-10-10 10:40:26,971: [main] DEBUG
> JNDISourceAdapter.getLdapResults(403) -  - searchType: searchSubject
> filter: (&  (uid=grouperTestSubjectByIdOnStartupASDFGHJ)
> (objectClass=supannPerson)) base: ou=people,dc=univ-lille1,dc=fr scope:
> SUBTREE_SCOPE
> 2011-10-10 10:40:26,975: [main] DEBUG
> JNDISourceAdapter.getLdapResults(403) -  - searchType:
> searchSubjectByIdentifier filter: (&
> (mail=grouperTestSubjectByIdentifierOnStartupASDFGHJ)
> (objectClass=supannPerson)) base: ou=people,dc=univ-lille1,dc=fr scope:
> SUBTREE_SCOPE
> 2011-10-10 10:40:26,979: [main] DEBUG
> JNDISourceAdapter.getLdapResults(403) -  - searchType: search filter: (&
> (|(uid=grouperTestStringOnStartupASDFGHJ)(displayName=*grouperTestStringOnStartupASDFGHJ*)))
> base: ou=people,dc=univ-lille1,dc=fr scope: SUBTREE_SCOPE
>
> Thanks !
> Brigitte
>
> Le 08/10/2011 20:37, Chris Hyzer a écrit :
> > I tried in 1.6 and it works fine.  One tweak though, change this line
> >
> > FROM:
> > while (results.hasNext()) 
> > {group.addMember(SubjectFinder.findByIdOrIdentifier(results.next().getAttributes().get("uid").get(),
> >  true));}
> >
> > TO:
> > while (results.hasNext()) 
> > {group.addMember(SubjectFinder.findByIdOrIdentifier(results.next().getAttributes().get("pennname").get(),
> >  true), false);}
> >
> > This will fail if a subject isn't resolvable, but will not fail if the 
> > subject is already in the group.
> >
> > Thanks,
> > Chris
> >
> > -----Original Message-----
> > From: Brigitte Wallaert Taquet 
> > [mailto:]
> > Sent: Saturday, October 08, 2011 2:48 AM
> > To: Chris Hyzer
> > Cc: Tom Zeller; 
> >
> > Subject: RE: [grouper-users] sources.xml ldap and script groupershell
> >
> > Hello,
> >
> > Sorry, I don't view this message, I'll try that. I am in 1.6.3.
> >
> > Thanks !
> > Quoting Chris 
> > Hyzer<>:
> >
> > > Are you on 2.0?
> > >
> > > Here is a script that will do what you want, and doesn't depend on
> > > sources.xml findAll()...
> > >
> > > First add these properties to the grouper.properties file:
> > >
> > > ldap.url = ldaps://someldap.school.edu:636
> > > ldap.baseDn = dc=school,dc=edu
> > > ldap.user = uid=user,ou=people,dc=school,dc=edu
> > > ldap.pass = xxxxxx
> > >
> > > Then, add the vt-ldap.jar to the classpath (lib dir?)
> > >
> > > http://code.google.com/p/vt-middleware/downloads/detail?name=vt-ldap-3.3.4-dist.zip&can=2&q=
> > >
> > > Here is a GSH script to add membership from an ldap filter to a
> > > group (note, make sure there are no newlines that are not below
> > > (only after semicolon)).  Also note, you could turn this into Java
> > > for more flexibility too...  :)
> > >
> > >
> > > grouperSession = GrouperSession.startRootSession();
> > >
> > > group = new
> > > GroupSave(grouperSession).assignName("test:testGroup").assignCreateParentStemsIfNotExist(true).save();
> > >
> > > ldapConfig = new
> > > edu.vt.middleware.ldap.LdapConfig(edu.internet2.middleware.grouper.cfg.GrouperConfig.getProperty("ldap.url"),
> > > edu.internet2.middleware.grouper.cfg.GrouperConfig.getProperty("ldap.baseDn"));
> > > ldapConfig.setBindDn(edu.internet2.middleware.grouper.cfg.GrouperConfig.getProperty("ldap.user"));
> > > ldapConfig.setBindCredential(edu.internet2.middleware.grouper.cfg.GrouperConfig.getProperty("ldap.pass"));
> > >
> > > factory = new edu.vt.middleware.ldap.pool.DefaultLdapFactory(ldapConfig);
> > >
> > > pool = new edu.vt.middleware.ldap.pool.BlockingLdapPool(factory);
> > >
> > > ldap = pool.checkOut();
> > >
> > > results = ldap.search("ou=people,dc=school,dc=edu", new
> > > edu.vt.middleware.ldap.SearchFilter("(|(uid=jsmith)(uid=tjones))"),
> > > new String[]{"uid"});
> > >
> > > while (results.hasNext())
> > > {group.addMember(SubjectFinder.findByIdOrIdentifier(results.next().getAttributes().get("uid").get(),
> > > true));}
> > >
> > > pool.checkIn(ldap);
> > >
> > >
> > > # check with this:
> > > getMembers("test:testGroup");
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: 
> > >
> > > [mailto:]
> > >  On Behalf Of Chris Hyzer
> > > Sent: Friday, October 07, 2011 11:31 PM
> > > To: Brigitte Wallaert Taquet
> > > Cc: Tom Zeller; 
> > >
> > > Subject: RE: [grouper-users] sources.xml ldap and script groupershell
> > >
> > > Yes, you can have a different sources.xml for gsh and ui.  In
> > > Grouper 2.1 there is a loader from ldap:
> > >
> > > https://spaces.internet2.edu/display/Grouper/Grouper+-+Loader+LDAP
> > >
> > > I assume you cant wait a few months for this :)  Or maybe you can
> > > get something working now, and then migrate to the loader later and
> > > switch your sources.xml back.
> > >
> > > Another option is I can give you some code from 2.1 in a jar that
> > > you could run temporarily until you upgrade to 2.1...
> > >
> > > Thanks,
> > > Chris
> > >
> > > -----Original Message-----
> > > From: Brigitte Wallaert Taquet 
> > > [mailto:]
> > > Sent: Friday, October 07, 2011 2:42 PM
> > > To: Chris Hyzer
> > > Cc: Tom Zeller; 
> > >
> > > Subject: RE: [grouper-users] sources.xml ldap and script groupershell
> > >
> > > Hello,
> > >
> > > Yes, I need for my script GrouperShell (folder-grouper/bin/gsh.sh
> > > myscript.gsh) to search all subjects for whom the attribute ldap
> > > "ustlRole" is valued (ustlRole=*) and then attach these subjects to a
> > > group named "ustlRoletemoin".
> > >
> > > Initially, I ask first ldap with dircontext and after, I transform
> > > each identifier received (uid) in subject Grouper (with findAll on
> > > identifier "uid") but my administrator ldap says that I ask 2 times
> > > for the same things so he doesn't agree.
> > >
> > > So, I think use only findAll in my groupershell script to search on
> > > attribute ustlRole=* instead on uid and displayName. But for LiteUi, I
> > > need that the search for a subject (findAll too it seems ?) continue
> > > to use uid and displayName attributes to do the search.
> > >
> > > I hope it is more clearly ?
> > >
> > > So, if I understand what you say, it's perhaps possible if I can have
> > > 2 sources.xml différents ; one for my script and one for LiteUi ? but
> > > how ? Now, I do that but I have to modify sources.xml if I need to
> > > deploy again Grouper-API for Grouper-UI, it is not very good I think !
> > >
> > > Thanks for your help.
> > >
> > > Cordialement
> > > Brigitte
> > >
> > >
> > >
> > > Quoting Chris 
> > > Hyzer<>:
> > >
> > > > Not sure I completely understand, but findAll cannot have a
> > > > parameter of where it is being called from, it is consistent.  But
> > > > you can have a slightly different sources.xml in your UI as opposed
> > > > to WS or something else.  Can you explain what you need in a
> > > > different way?  :)
> > > >
> > > > Thanks,
> > > > Chris
> > > >
> > > > -----Original Message-----
> > > > From: Wallaert-Taquet Brigitte 
> > > > [mailto:]
> > > > Sent: Friday, October 07, 2011 5:54 AM
> > > > To: Chris Hyzer; Tom Zeller
> > > > Cc: 
> > > >
> > > > Subject: Re: [grouper-users] sources.xml ldap and script groupershell
> > > >
> > > > Hello,
> > > >
> > > > Thanks a lot for your indications : effectively, the logs show me the
> > > > asks which was ok and finally, the administrator's ldap find that the
> > > > attribute "ustlRole" was protected and my server wasn't the rights to
> > > > bind with connection ldap anonyme.
> > > >
> > > > Are you using GrouperJndiSourceAdapter in sources.xml ?
> > > >
> > > >     <source
> > > > adapterClass="edu.internet2.middleware.grouper.subj.GrouperJndiSourceAdapter"
> > > > ...>
> > > >
> > > > Yes
> > > >
> > > > Now I have problem because if I modify the search in sources.xml for my
> > > > groupershell, the LiteUI will not find the subject correctly (because
> > > > ask on ustlRole attribute instead on uid/displayName attributes).
> > > > So, do you think it is possible to parameter somewhere for
> > > > SubjectFinder.findAll use another search defined in sources.xml (for
> > > > example, I can define another searchType named "searchUstlRole" in my
> > > > sources.xml) or to parameter LiteUI and AdminUi to use another searchType ?
> > > >
> > > > Thanks a lot !
> > > > Brigitte
> > > >
> > > >
> > > > Le 06/10/2011 05:34, Chris Hyzer a écrit :
> > > > > Search is used in SubjectFinder.findAll e.g. on the UI when you do
> > > > > a search for a subject...
> > > > >
> > > > > Thanks,
> > > > > Chris
> > > > >
> > > > > -----Original Message-----
> > > > > From: 
> > > > >
> > > > > [mailto:]
> > > > >  On Behalf Of
> > > > > Wallaert-Taquet Brigitte
> > > > > Sent: Wednesday, October 05, 2011 5:32 AM
> > > > > To: 
> > > > >
> > > > > Subject: [grouper-users] sources.xml ldap and script groupershell
> > > > >
> > > > > Hello,
> > > > >
> > > > > I make a script that have to populate a group (named "ustlroletemoin")
> > > > > for people LDAP for whom attribute ldap "ustlRole" have a value.
> > > > >
> > > > > So, I modify sources.xml for the search asks on ustlRole attribute (I
> > > > > don't know exactly when this search is used ?):
> > > > >
> > > > > <search>
> > > > > <searchType>search</searchType>
> > > > > <param>
> > > > > <param-name>filter</param-name>
> > > > > <param-value>
> > > > >                     (ustlRole=%TERM%)
> > > > > <--------------------------------------------------
> > > > > </param-value>
> > > > > </param>
> > > > > <param>
> > > > > <param-name>scope</param-name>
> > > > > <param-value>
> > > > >                     SUBTREE_SCOPE
> > > > > </param-value>
> > > > > </param>
> > > > > <param>
> > > > > <param-name>base</param-name>
> > > > > <param-value>
> > > > >                     ou=people,dc=univ-lille1,dc=fr
> > > > > </param-value>
> > > > > </param>
> > > > > </search>
> > > > >
> > > > > But I don't find a method that ask with this search ? It seems that
> > > > > SubjectFinder.findAll asks only on searchSubject in sources.xml ?
> > > > >
> > > > > Before to put "*" (!), I try this and I have no results although there
> > > > > are persons who have ustlRole=IntranetTest-users :
> > > > >
> > > > > for (Subject sujet : SubjectFinder.findAll("IntranetTest-users")) {
> > > > > groupetemoin.addMember(sujet,false); nbreusersajoutes+=1; } }
> > > > >
> > > > > result : entree and nbreuserajoutes=0
> > > > >
> > > > > Thanks a lot !
> > > > --
> > > > Brigitte Wallaert-Taquet
> > > > Ingénieure d'études
> > > > Chargée d'étude
> > > > Espace collaboratif de Documents
> > > > Université Lille1
> > > > Sciences et Technologies


--
Brigitte Wallaert-Taquet
Ingénieure d'études
Chargée d'étude
Espace collaboratif de Documents
Université Lille1
Sciences et Technologies