Grouper Developers Forum

["Gettes, Michael" <>]

hi Bert - I replied to your private Slack query about all this as well.  If 
my estimates on the size of a cached entry are correct (I supposed 600bytes) 
then 500K (which is what I am wanting to use) would be only 300MB of cache.  
If my math is right - then I don’t believe putting this on disk is worth it - 
just keep it all in memory.  If you do this kind of caching for each LDAP 
server defined for Grouper - and 10 LDAP servers - then it would be 3GB of 
caching.  I would still say to keep this in memory.

math and opinions are my own.

/mrg

> On Jun 4, 2019, at 1:22 PM, Bee-Lindgren, Bert 
> <> wrote:
> 
> PSPNG takes a Collections-intensive approach to processing identities. In 
> other words, there are methods to fetch hundreds of thousands of objects 
> and return them as a single Collection... obviously sucking lots of memory.
> 
> I'm testing more of a Streaming approach (processing results as they come 
> in rather than collecting them in a collection) as well as holding the 
> remaining, necessary, large structures in btrees so they can be on disk.
> 
> A couple questions:
> 1) Does the Loader or other parts of grouper do anything with 3-500k'ish 
> lists of identities with less memory impact? Any tricks?
> 
> 2) Does GrouperCache support hybrid (some memory, mostly disk) caching? I 
> can see that ehCache seems to have max-memory and max-disk options, but 
> didn't see if GrouperCache supported them.
> 
> 3) If I find btrees particularly useful for either short- or long-term 
> tasks, most btree libraries (like btree4j) seem to not be in maven central. 
> One (MapDB ) is in maven central... How would I add a non-maven-central or 
> a maven-central jar to PSPNG's dependencies?
> 
> 
> 
> Thanks,
>   Bert