Grouper Users - Open Discussion List

[Tom Zeller <>]

In addition to importing stems separately, it might also be
interesting to import first without memberships or permissions, then a
second time with memberships and permissions.

To ignore memberships and privileges, you would do something like

 bin/gsh.sh -xmlimport GrouperSystem export.xml import.properties

import.properties :

 import.data.lists=ignore
 import.data.privileges=ignore

And, of course, there's running with a profiler.

TomZ

On Thu, Dec 17, 2009 at 3:48 PM, Chris Hyzer 
<>
 wrote:
> Is this a 32bit machine or 64?  2gig might be the max that Java can use.
>
> We are planning to rewrite the import/export, and I hope to improve 
> performance in the rewrite.  In the meantime, maybe if this is not a 64 bit 
> machine, you could try from another machine which is 64 bit.  Or maybe you 
> could export twice and partition the stems so that you have two files (or 
> more) of half the size (or less).  Kind of a pain I know, but just 
> brainstorming here.
>
> Thanks,
> chris
>
> -----Original Message-----
> From: Colin Hudler 
> [mailto:]
> Sent: Thursday, December 17, 2009 10:18 AM
> To: 
> 
> Subject: Re: [grouper-users] gsh xmlimport memory usage
>
> The backup file is 256M.  Memberships are 1507480, in 90908 groups.  The
> subject source is
> edu.internet2.middleware.grouper.subj.GrouperJndiSourceAdapter, but
> maybe I will change it and compare.
>
> Tom Zeller wrote:
>> What is the filesize of backup.xml ? How many memberships do you have ?
>>
>> I've been able to xmlimport with Xmx=1024m, and we have ~1 million
>> memberships (select count(*) from grouper_memberships), and our xml
>> export is ~175MB.
>>
>> What source adapters are you using ?
>>
>> This doesn't answer your question, of course.
>>
>> TomZ
>>
>> On Thu, Dec 17, 2009 at 9:00 AM, Colin Hudler 
>> <>
>>  wrote:
>>> Hi,
>>>
>>>  Congrats on the fantastic new release, nice stuff.  I am doing upgrade
>>> rehearsals under Method 1,
>>>
>>> "./bin/gsh.sh -xmlimport GrouperSystem backup.xml"
>>>
>>> But it always uses all of the memory.  I increased it to 4GB, but it still
>>> actually commits that much.  Before moving on to method 2, I thought I'd
>>> ask, is there anything I can do reduce it?
>>>
>>> Obviously, this is the actual error:
>>>
>>> java.lang.OutOfMemoryError: Java heap space
>>>
>>
>