Grouper Users - Open Discussion List

[Jeffrey Williams <>]

Carey,

We're sending all of the container logs(httpd, shibd, tomcat, and grouper) to Splunk via a volume folder to the host which is read by Splunk's universal forwarder. From there, we use Splunk to filter in what we want.  

In addition to the changelogs, loaders, and PSPNG logs, we also look at:

•  LdapGroupUserConverter 
• ldaptive
• grouper rules (logged at WARN)
• a few Grouper connectors that are in development(e.g. Box, Duo, and Google).  
  

In terms of log levels, We keep the root and base middleware to ERROR and WARN, then drop specific middlewares to INFO as needed.  

We're not doing any specific loggers for Splunk, but I did have my Splunk admin set it up to where we could separate and share the connector logs with those individual service owners.   

We haven't done anything specific to auditing yet, but when we do, I can see us beginning with attestation.  

We're also in the process in moving from the appliance image to the master, which by default handles logging via named pipe out to the Docker daemon.  You can run your container with a Splunk driver which sends the logs straight from the container to the http event collector(or configure it how you had it before).  

Has anyone changed their log forwarding method from a forwarder to an event collector?  If so, how much retooling was needed on Splunk to handle that sort of change?

On Fri, Sep 14, 2018 at 11:17 AM Carl Waldbieser <> wrote:

Carey,

We are basically shipping our changelogger and provisioner logs to Splunk.  This includes the events that are sent/received/processed for each service.  Those logs are useful for alerting if a provisioner runs into trouble.  We have standard web logs, and grouper_error logs going in, but I don't find those as useful for day-to-operations.

Thanks,
Carl Waldbieser
ITS Identity Management
Lafayette College

----- Original Message -----
From: "Carey M. Black" <>
To: "grouper-users" <>
Sent: Friday, 14 September, 2018 09:57:03
Subject: [grouper-users] RE: Grouper and Splunk

All,

Poking an old thread.
So far I have received no comments/strategies.  ( Should I assume that no one is doing anything like this? )

--
Carey Matthew
Office of the Chief Information Officer (OCIO)
Identity and Access Management - Security Engineer-Lead
614-292-6079 Office


-----Original Message-----
From: Black, Carey M.
Sent: Saturday, May 26, 2018 11:29 PM
To:
Subject: Grouper and Splunk

All,

Before I start to reinvent the wheel....

For those of you who use Grouper and Splunk....  (or any other SEIM tool )
        How/What data are you exporting from Grouper to Splunk?
                "Just" feeding it some of the standard log4J UI logs? Loader Logs? WS Logs? PSP/NG Logs?
                        Did you setup any specific log levels/classes specifically for Splunk visibility?
        Do you try to send Grouper audit data? ( "User audit" and/or "PIT audit" ? )
        Do you try to "limit"/"shape" the details that are headed to Splunk or just dumping it all?


Part of me thinks I should try to capture only this data and get it into Splunk:
        Membership changes in:
                External System of Record group changes ( loaded from: loader jobs, script integrations, etc... )
                Grouper System of Record Groups (Think:  manual groups maintained in Grouper, like includes/excludes )
                Access Policy groups ( any group "used by an external system")

        And not send data about membership changes all of the "group math" / intermediate role up groups between the SOR's and the Access Policies.
                And then parts of me thinks knowing who changed what about the group math structure would also be good to have logged too. ( Just not the membership changes for those groups.)

However, doing exactly that would take some work to identify/maintain the "right groups" and could be subject to "Opps, missed that group" problems too.
        ( Maybe use a custom change log consumer to directly emit the "Splunk" data in a "Splunk format" ?)

Anyone what to share their strategy?

--
Carey Matthew

--

Jeffrey Williams 

Identity Engineer
Identity & Access Services
https://its.uncg.edu