Grouper Users - Open Discussion List

[Mark Cairney <>]

Hi Dave,

This is Grouper trying to create the group initially and failing. When bulkSyncing
Having checked the UI the group has 338756 members(!) and is our "student applicant" group.
The next-biggest group (which did provision successfully) is our alumni group which has 232434 members.
What's the largest known group being provisioned with Grouper by the way?

From what you and Tom are saying it's the sheer size of the operation/ building the LDIF which is the issue which at least means that if the alumni group grows it shouldn't hit this issue unless it suddenly doubles in size as from here on in it's primarily small changes via the changelog consumer.

Perhaps unsurprisingly the only reference to this error message I'd found via Googling was the function call within the JNDI "BerEncoder.java" file.

If there is a way to split it into smaller operations (even temporarily to push it through) that would be great. Otherwise my plans of action have been to create the group manually based on the old version of the group from our Grouper 1.5 provisioned OU or to assemble it from a query of the group from the grouper DB. I think we are planning a purge on this Applicant group anyway so we may have to wait for that to happen and get the membership down to a manageable level.

On 06/03/2015 18:09, David Langenberg wrote:

Hi Mark,

What is the context of this operation?  Is it creating the group, adding a membership, or something else?  What is it trying to add?  I've never seen this before, but this error is being thrown by JNDI itself, so whatever you're doing it's got a very large piece of data in it.

Dave

On Wed, Mar 4, 2015 at 6:53 AM, Mark Cairney <> wrote:

Hi all,

We've progressed our Grouper installation into production but have been hitting the following error when attempting to provision a particular group with both bulkSync and sync.

It looks like a buffer overflow somewhere. On the assumption that we're not the only people using Grouper with similarly large groups has anyone else seen this behaviour and is there any further tuning we can do to resolve it?

We do have another group (ALUM) which is slightly smaller at around 250,000 members which was provisioned successfully. Another worry is what would happen if this other group were to grow to a similar size to the APPS group.

Finally on an unrelated note we're probably getting close to being in a state to do some sort of write-up of our deployment for the Grouper wiki. What's the procedure for getting this sorted out?

Kind regards,

Mark


2015-03-04 13:00:31,326: [main] WARN AbstractLdap.operationRetry(1105) -  - Error performing LDAP operation, retrying (attempt 0)
javax.naming.CommunicationException: SEQUENCE too long [Root exception is com.sun.jndi.ldap.Ber$EncodeException: SEQUENCE too long]; remaining name 'cn=APP,ou=affiliations,ou=grouper2,dc=authorise,dc=ed,dc=ac,dc=uk'
        at com.sun.jndi.ldap.LdapCtx.c_createSubcontext(LdapCtx.java:845)
        at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_createSubcontext(ComponentDirContext.java:341)
        at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.createSubcontext(PartialCompositeDirContext.java:268)
        at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.createSubcontext(PartialCompositeDirContext.java:256)
        at edu.vt.middleware.ldap.AbstractLdap.create(AbstractLdap.java:882)
        at edu.vt.middleware.ldap.Ldap.create(Ldap.java:673)
        at edu.internet2.middleware.psp.ldap.LdapSpmlTarget.execute(LdapSpmlTarget.java:257)
        at edu.internet2.middleware.psp.spml.provider.BaseSpmlProvider.execute(BaseSpmlProvider.java:177)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:483)
        at edu.internet2.middleware.psp.spml.provider.BaseSpmlProvider.execute(BaseSpmlProvider.java:123)
        at edu.internet2.middleware.psp.Psp.execute(Psp.java:456)
        at edu.internet2.middleware.psp.spml.provider.BaseSpmlProvider.execute(BaseSpmlProvider.java:177)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:483)
        at edu.internet2.middleware.psp.spml.provider.BaseSpmlProvider.execute(BaseSpmlProvider.java:123)
        at edu.internet2.middleware.psp.Psp.execute(Psp.java:1502)
        at edu.internet2.middleware.psp.Psp.execute(Psp.java:1441)
        at edu.internet2.middleware.psp.Psp.execute(Psp.java:879)
        at edu.internet2.middleware.psp.Psp.execute(Psp.java:800)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:483)
        at edu.internet2.middleware.psp.spml.provider.BaseSpmlProvider.execute(BaseSpmlProvider.java:123)
        at edu.internet2.middleware.psp.PspCLI.run(PspCLI.java:138)
        at edu.internet2.middleware.psp.PspCLI.main(PspCLI.java:84)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:483)
        at edu.internet2.middleware.grouper.app.gsh.GrouperShell.handleSpecialCase(GrouperShell.java:204)
        at edu.internet2.middleware.grouper.app.gsh.GrouperShell.main(GrouperShell.java:144)
        at edu.internet2.middleware.grouper.app.gsh.GrouperShellWrapper.main(GrouperShellWrapper.java:31)
Caused by: com.sun.jndi.ldap.Ber$EncodeException: SEQUENCE too long
        at com.sun.jndi.ldap.BerEncoder.endSeq(BerEncoder.java:179)
        at com.sun.jndi.ldap.LdapClient.encodeAttribute(LdapClient.java:987)
        at com.sun.jndi.ldap.LdapClient.add(LdapClient.java:1032)
        at com.sun.jndi.ldap.LdapCtx.c_createSubcontext(LdapCtx.java:808)
        ... 37 more
2015-03-04 13:00:32,603: [main] ERROR BaseSpmlProvider.execute(188) -  - Target 'ldap' - Add AddResponse[pso=<null>,status=failure,error=customError,errorMessages={SEQUENCE too long},requestID=2015/03/04-11:34:40.199]
2015-03-04 13:00:32,603: [main] ERROR BaseSpmlProvider.execute(190) -  - Target 'ldap' - Add XML:
<addResponse xmlns='urn:oasis:names:tc:SPML:2:0' status='failure' requestID='2015/03/04-11:34:40.199' error='customError'>
  <errorMessage>SEQUENCE too long</errorMessage>
</addResponse>

2015-03-04 13:00:32,604: [main] ERROR BaseSpmlProvider.execute(188) -  - Target 'psp' - Add AddResponse[pso=<null>,status=failure,error=customError,errorMessages={SEQUENCE too long},requestID=2015/03/04-11:34:40.199]
2015-03-04 13:00:32,604: [main] ERROR BaseSpmlProvider.execute(190) -  - Target 'psp' - Add XML:
<addResponse xmlns='urn:oasis:names:tc:SPML:2:0' status='failure' requestID='2015/03/04-11:34:40.199' error='customError'>

--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

--

David Langenberg

Identity & Access Management

The University of Chicago

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.