Skip to Content.
Sympa Menu

grouper-dev - [grouper-dev] Reducing PSPNG's memory footprint

Subject: Grouper Developers Forum

List archive

[grouper-dev] Reducing PSPNG's memory footprint


Chronological Thread 
  • From: "Bee-Lindgren, Bert" <>
  • To: "" <>
  • Subject: [grouper-dev] Reducing PSPNG's memory footprint
  • Date: Tue, 4 Jun 2019 17:22:50 +0000

PSPNG takes a Collections-intensive approach to processing identities. In other words, there are methods to fetch hundreds of thousands of objects and return them as a single Collection... obviously sucking lots of memory.

I'm testing more of a Streaming approach (processing results as they come in rather than collecting them in a collection) as well as holding the remaining, necessary, large structures in btrees so they can be on disk.

A couple questions:
1) Does the Loader or other parts of grouper do anything with 3-500k'ish lists of identities with less memory impact? Any tricks?

2) Does GrouperCache support hybrid (some memory, mostly disk) caching? I can see that ehCache seems to have max-memory and max-disk options, but didn't see if GrouperCache supported them.

3) If I find btrees particularly useful for either short- or long-term tasks, most btree libraries (like btree4j) seem to not be in maven central. One (MapDB ) is in maven central... How would I add a non-maven-central or a maven-central jar to PSPNG's dependencies?



Thanks,
  Bert




Archive powered by MHonArc 2.6.19.

Top of Page