Shibboleth Developers

["Dergenski, Todd A." <>]

The logs you currently dump out are pretty good.  We were able to build a 
shell script to dig out:
Service Provider Usage Report - Raw count of logins per SP.
Top Most Logged in Users - Top 10 Users.
Top 10 Clients - Top 10 IPs
Usage by Hour - Raw count. Hour breakdown of logins.  Useful for finding 
appropriate outage windows.
Usage By Day - Raw count. Day by day breakdown of logins.
Number of Unique Logins - Unique users actually logging in.
Number of Unique Logins per Service Provider - Unique users per SP.
Number of Unique Logins per Service Provider By Day - Unique Users per SP per 
Day.
Number of Websites Logged in Per User Average - Gives a rough idea of who is 
using Shibboleth as SSO and who just uses it for a single page.
Auth type count - count of which methods are being used.

Wishes:
Expose the client IP address to the MDC logger.  Not specifically related to 
reporting, but for log analysis and consumption by many SIEMs (or whatever 
the current term is) this makes life a lot simpler.  SessionID can be used to 
link them, but many security oriented SIEMs don't care for the IP only being 
logged once.

User Agent logging. Building reports that show non IE browsers are growing, 
helps me in making sure that FF/Chrome are supported.

Monitoring/Logging - Current session count. Active authenticated users.  I 
can get this out of terracotta at the moment, but something that can be 
occasionally logged or exposed to /Status

Status of Data sources.  Which ones have had to fail over to?  Connection 
counts for pooled resources.  Logged occasionally, but preferably exposed 
through /Status.  At the moment, can be partially mined out of idp-process 
log but something a little more request friendly.  I would want to build a 
Nagios plug-in to report back to our operations staff.

Count of failed login attempts because of attribute retrieval. Can probably 
be dig out of the existing logs.  

Not exactly Shibboleth itself, but which auth server failed/authenticated the 
user.

Exceptions thrown. count by type.  We have a standalone report for this 
already, but something integrated would be nice.




Todd Dergenski
Old Dominion University
Senior Security Administrator
4700 Elkhorn Ave - Room 4300
Norfolk, Va, 23529 USA

(757) 683-4301



-----Original Message-----
From: Chad La Joie 
[mailto:]
 
Sent: Wednesday, May 19, 2010 9:16 PM
To: 

Subject: [Shib-Dev] IdP v3 Metrics: What do you need?

One focus of IdP v3 is on production-related enhancements.  One such
enhancement is getting the IdP to produce more raw data from which
people may generate reports.  So, that leads to the question, what data
do you need?

Note, I will not be adding a reporting system to the IdP.  This means
that there won't be any aggregate data (e.g. # of failed authentications
in the last 30 minutes), just raw data (e.g. failed authentication at
2010-05-19T21:14:00).

So, chime in.  What type of data would you like to have access to, for
reporting, monitoring, etc., that you don't today?

-- 
Chad La Joie
www.itumi.biz
trusted identities, delivered