Grouper Users - Open Discussion List

[Peter DiCamillo <>]

I've found a solution for speeding up dumping the registry, and have the 
time down to less than 5 hours.  It's not a solution that would help 
other sites.  It takes advantage of the specific organization of our 
course groups, which use a set of 12 groups for each section of a 
course.  The set of groups is created in a structure such that much less 
information is needed to recreate them than the export normally program 
saves.  The course groups make such a big difference because they're 
about 88% of our groups (472K.)

Thanks for the offer to help.  I'm about to start looking at load times, 
and if that becomes a problem I'll get back to you.  It seems much more 
worthwhile to investigate any load problems, since they would be with 
version 1.6.1 rather than 1.3.

Peter

Tom Zeller wrote:
> Peter,
>
> Do you want to try and figure out performance issues ? or do you have
> enough to get you upgraded ?
>
> TomZ
>
> On Fri, Aug 27, 2010 at 3:31 PM, Peter DiCamillo
> <>
>  wrote:
> > Thanks for the memory suggestion.  I was already using "-Xmx2048M
> > -Xms2048M".
> >
> > We have 537K groups and 1.35 million direct memberships in those groups.
> >  Since I'm not familiar with the database schema, I ran a small program to
> > get the counts.  It traversed the group tree, and counted memberships using
> > the size of group.getImmediateMemberships.  Just running that took 70
> > minutes.
> >
> > Peter
> >
> > Chris Hyzer wrote:
> > > Did you add more memory to gsh?  Might be worth a shot...
> > >
> > > Chris
> > >
> > > -----Original Message-----
> > > From: 
> > >
> > >  
> > > [mailto:]
> > >  On Behalf Of Tom
> > > Zeller
> > > Sent: Friday, August 27, 2010 10:27 AM
> > > To: Peter DiCamillo
> > > Cc: 
> > >
> > > Subject: Re: [grouper-users] upgrading from 1.4.2 to 1.6.1
> > >
> > > > A related problem I encountered is that I'm unable to dump the entire
> > > > registry.  (That's why I was testing with -name.)  We have a very large
> > > > number of groups, and I did a rough calculation that it would take 32
> > > > hours
> > > > to dump them all.
> > > Roughly how many groups ? memberships ?
> > >
> > > At the time of our 1.3 upgrade, Memphis had ~32k groups and ~1.24
> > > million rows in grouper_memberships; exporting usually took less than
> > > half an hour.