Grouper Users - Open Discussion List

["Bee-Lindgren, Bert" <>]

PSPNG Patches 7 & 8 make several improvements which are summarized below. You can also find more details in the following Jiras:

GRP-2192: Fix misplaced placeholders for a few log statements

GRP-2208 - Improvements made to: Caching, Unresolvable-subject handling, Integration tests (particularly Active Directory), and Bug Fixes

Improvements made to: Caching, Unresolvable-subject handling, Integration tests (particularly Active Directory), and Bug Fixes

Caching (more configurable, uses less memory)
-Cache configuration moved to (generic) Provisioner class from LdapProvisioner
(warnings and backwards compatibility in place for ldapUserCacheSize and
ldapUserCacheTime)
-Sharing GrouperGroup, GrouperSubject and TargetSystemUser caches
between Full and Incremental sync systems. TargetSystemGroup caches
are separately cached by Full and Incremental provisioners because
FullSync and Incremental provisioners might fetch different information
about Groups
-Separate cache sizes for Grouper and TargetSystem information
-Map cache-size=0 to cache-size=1 because size=0 means Unlimited caching
within EhCache

Unresolvable subjects: Handled gracefully

Integration Tests:
-Active Directory tests (needs external AD server)
-Support LDAP target separate from Subject Source
--ldap parameters from grouper-loader.properties
-Fix some missing objectclasses
-Better list comparisons that report diffs when integ tests fail
-add pre_test and post_test hooks for (external) AD cleanup
-Better test-failure messages

GrouperLoaderLog:
-Provide updates while quartz-initiated full-syncs are underway
--JobStatistics's date externalized so intermediate stats could collect information about work done so far

FullSync setup:
-Set activeProvisioner earlier in FullSync thread and just log a warning
when LdapObjects are created when the ActiveProvisioner is not defined
-Refresh GrouperSession earlier in process so messaging acks/requeues work

FullSync processing:
-Force set comparisons to use correct case sensitivity by forcing both sets
being compared to have correct sensitivity. (Java doesn't use consistent
set semantics when comparing sets:
https://bugs.openjdk.java.net/browse/JDK-6394757
https://bugs.openjdk.java.net/browse/JDK-8180409
-Only sync groups that are selected for provisioning
-Repeat FullSyncs when they make changes to make sure that the FullSync
didn't clobber a realtime change

• This required Provisioner.fullSync to return whether changes occurred
  -Do ldap modification right away (instead of coalescing them) because
  the FullSync handles a single group and, therefore, no coalescing across
  multiple groups is possible

Logging:
-Ldap-logging utilitiy methods in LdapEntry for consistent INFO summaries
and DEBUG details
-Moving several INFO messages to DEBUG
-Warnings about coalesced-provisioning failures greatly simplified
-Several small message/level improvements

Cache-Efficiency stats: (not complete)
-Add getNativeMemorySize_bytes() to TargetSystemUsers and Groups to
be able to compare how much memory TargetSystemUsers and TargetSystemGroup
objects take vs how much data came from the TargetSystem. Eg, compare
how much an LdapEntry data structure takes vs the equivalent LDIF text of
the Entry