Shibboleth Developers

["Dergenski, Todd A." <>]

Current active user count would be the only real one.  At the moment, we 
created a JMX nagios plug-in to dig it out of terracotta.  Having it log out 
or exposing through /Status would simplify things.

Unless you are going to be running internal counts for various actions/stats 
just log the event as it happens.  Failed or Success with some kind of 
identifier can be easily turned into stats with cat | grep -o | sort | uniq 
-c |sort -n 

Other than that, as long as we have an identifier to link lines together, 
perl can put just about anything together.  

Todd Dergenski
Old Dominion University
Senior Security Administrator
4700 Elkhorn Ave - Room 4300
Norfolk, Va, 23529 USA

(757) 683-4301



-----Original Message-----
From: Chad La Joie 
[mailto:]
 
Sent: Thursday, May 20, 2010 10:55 AM
To: 

Subject: Re: [Shib-Dev] IdP v3 Metrics: What do you need?

Is there anything you currently have to dig out of the process log that
you'd like to show up in some sort of machine parsable log akin to the
audit log?

On 5/20/10 10:52 AM, Dergenski, Todd A. wrote:
> The logs you currently dump out are pretty good.  We were able to build a 
> shell script to dig out:
> Service Provider Usage Report - Raw count of logins per SP.
> Top Most Logged in Users - Top 10 Users.
> Top 10 Clients - Top 10 IPs
> Usage by Hour - Raw count. Hour breakdown of logins.  Useful for finding 
> appropriate outage windows.
> Usage By Day - Raw count. Day by day breakdown of logins.
> Number of Unique Logins - Unique users actually logging in.
> Number of Unique Logins per Service Provider - Unique users per SP.
> Number of Unique Logins per Service Provider By Day - Unique Users per SP 
> per Day.
> Number of Websites Logged in Per User Average - Gives a rough idea of who 
> is using Shibboleth as SSO and who just uses it for a single page.
> Auth type count - count of which methods are being used.
> 
> Wishes:
> Expose the client IP address to the MDC logger.  Not specifically related 
> to reporting, but for log analysis and consumption by many SIEMs (or 
> whatever the current term is) this makes life a lot simpler.  SessionID can 
> be used to link them, but many security oriented SIEMs don't care for the 
> IP only being logged once.
> 
> User Agent logging. Building reports that show non IE browsers are growing, 
> helps me in making sure that FF/Chrome are supported.
> 
> Monitoring/Logging - Current session count. Active authenticated users.  I 
> can get this out of terracotta at the moment, but something that can be 
> occasionally logged or exposed to /Status
> 
> Status of Data sources.  Which ones have had to fail over to?  Connection 
> counts for pooled resources.  Logged occasionally, but preferably exposed 
> through /Status.  At the moment, can be partially mined out of idp-process 
> log but something a little more request friendly.  I would want to build a 
> Nagios plug-in to report back to our operations staff.
> 
> Count of failed login attempts because of attribute retrieval. Can probably 
> be dig out of the existing logs.  
> 
> Not exactly Shibboleth itself, but which auth server failed/authenticated 
> the user.
> 
> Exceptions thrown. count by type.  We have a standalone report for this 
> already, but something integrated would be nice.
> 
> 
> 
> 
> Todd Dergenski
> Old Dominion University
> Senior Security Administrator
> 4700 Elkhorn Ave - Room 4300
> Norfolk, Va, 23529 USA
> 
> (757) 683-4301
> 
> 
> 
> -----Original Message-----
> From: Chad La Joie 
> [mailto:]
>  
> Sent: Wednesday, May 19, 2010 9:16 PM
> To: 
> 
> Subject: [Shib-Dev] IdP v3 Metrics: What do you need?
> 
> One focus of IdP v3 is on production-related enhancements.  One such
> enhancement is getting the IdP to produce more raw data from which
> people may generate reports.  So, that leads to the question, what data
> do you need?
> 
> Note, I will not be adding a reporting system to the IdP.  This means
> that there won't be any aggregate data (e.g. # of failed authentications
> in the last 30 minutes), just raw data (e.g. failed authentication at
> 2010-05-19T21:14:00).
> 
> So, chime in.  What type of data would you like to have access to, for
> reporting, monitoring, etc., that you don't today?
> 

-- 
Chad La Joie
www.itumi.biz
trusted identities, delivered