Skip to Content.
Sympa Menu

perfsonar-dev - Re: [pS-dev] (SYS)LOG messages generated by the pS services?

Subject: perfsonar development work

List archive

Re: [pS-dev] (SYS)LOG messages generated by the pS services?


Chronological Thread 
  • From: "Luchesar V. ILIEV" <>
  • To: "" <>
  • Cc: Stijn Melis <>, Jochen Reinwand <>, Fausto Vetter <>, Martin Swany <>, , Nicolas Simar <>,
  • Subject: Re: [pS-dev] (SYS)LOG messages generated by the pS services?
  • Date: Thu, 24 Apr 2008 14:56:48 +0300
  • Disposition-notification-to: "Luchesar V. ILIEV" <>
  • Openpgp: id=9A1FEEFF; url=https://cert.acad.bg/pgp-keys/
Hi guys,

I fully concur too. Just like Jochen, I also have strong preference towards the typical syslog format of the entries. Fausto's suggestion thus seems very nice.

I'd just like to ask a couple of questions:

1) Do we have a common agreement on the SERVICE_TYPEs abbreviations?

2) RESULT should be fairly straightforward, I guess, but how about ACTION -- I'm afraid I don't understand what is going to be recorded there exactly. To put it another way -- are the ACTIONs going to be fairly standard (in fact: enforced as standard?)

Cheers,
Luchesar


Stijn Melis wrote:
Hi,

this format presented by Fausto looks very good to me also. It is easy to read, and it clearly states what needs to be said.

cheers,

Stijn

Jochen Reinwand wrote:
Hi,

OK for me! I do not really like CSV also ;)
It's really an advantage to have readable, "normal" syslog entries. Not everybody wants to set up a centralised parsing after installing only a simple perfSONAR service.

Jochen

On Thursday 24 April 2008 10:49, Fausto Vetter wrote:
Hi,

I like the idea of having a format, but I don't like the idea of just
CSV (comma, semi-comma, etc) format. It becomes easy to parse but not
easy to read. It would be better for example:

[pS-XML-LS] success.ls.remove 'Lookup info registered with key
http://stout.pc.cis.udel.edu:8080/services/RRDMA has been removed!'

where:

[SERVICE_TYPE] RESULT.ACTION 'DESCRIPTION'

Other fields like date & time and source are inherent to syslog format.

What do you think?

Fausto

Jochen Reinwand wrote:
Hi all,

I don't think that the EGEE guidelines are "wrong". They are not even
unusable for us! But I have the feeling that these key/value pairs tend
to grow to a standardisation horror. Parsing is easy! I'm quite sure
Fausto is thinking about analysing the syslog via Perl. Parsing key/value
pairs in Perl is like writting "Hello world!" in C. Eh, OK it's even
easier...
But I fear that we will get into a lot of different key/value pairs that
will be very hard to interpret (the necessary second step after parsing).
I'm forseeing log lines like these:

my-service-error=12345 xyz-stack=pop \
time=3-quarters-to-4-hours-after-local-midnight

Using a fixed format will perhaps prevent such a misuse. I see something
like a decision tree:

system;Cannot create pid file

perfSONAR;success.ls.remove;Lookup info registered with key \
http://stout.pc.cis.udel.edu:8080/services/RRDMA has been removed!

hades;database;No data found

Here "system" means related to perfSONAR independent errors comming from
typical system interaction (e.g. daemon creating a pid file). With
"perfSONAR" prefix typical perfSONAR/NM-WG status messages are logged.
The idea here: Write to syslog what the user sees in the response. The
last example is a "background system" related error. Most perfSONAR
services use some tools or databases in the background. It is most likely
usefull to log errors in this subsystems this way.

Once again: This can also be done using key/value pairs, but I fear to
much freedom here...

greetings,
Jochen

On Wednesday 23 April 2008 20:26, Martin Swany wrote:
Hi all,

I think that the overhead of the best practices format is justified by
the gains that Brian mentioned. Also note that the log format is
analogous to the NM-WG format in that it has a hierarchical
event names (eventType). I think there are some obvious wins
that come from having the same namespaces used in the log
messages and in the result codes.

best,
martin

On Apr 23, 2008, at 12:45 PM, Fausto Vetter wrote:
Hi,

Thats a good point. It needs to have a grammar otherwise it becomes
hard to parse and get some useful information from it. Putting in a
database is a step forward to go in some time, thats why we are
gonna setup a syslog server.

Cheers,
Fausto

Brian Tierney wrote:
Luchesar:

One of the main advantages of a 'best practice'-like format is that
if you think there might be a time you'd want to put these logs
into a database, then the logs must have some grammar, and not be
totally freeform.

Im not sure there is an immediate use case for putting perfSONAR
logs into a database, but it might be good to make sure that the
logs all follow the same grammar in case such a use case arises in
the future.


One possible use case I can think of is the following.

Imagine that every perfSONAR service generated a syslog message
logging each client connection. If the syslog messages where
collected and stored in a database, it would be very each to make
queries such as 'tell me the 10 top clients of perfSONAR services
last week'.

Clearly a log message that looked like something like this:

event=org.perfSONAR.MA.connect client=A.B.C.D request=RetrieveData

would be much easier to parse and insert into a database that one
that might look like this:

'received request for data from client x.y.w.z'

Something to consider.

Cheers.

Luchesar V. ILIEV wrote:
Hi Brian, Jochen, all,

First of all, Brian, thank you very much for providing a pointer
to this very interesting document. It'll certainly be helpful in
our efforts within perfSONAR.

At the same time, I have to agree with Jochen on most of his
points. Indeed, one of the reasons to use syslog is exactly the
fact that it already does provide standard approach to some of the
issues.

The "key/value" pairing also does not appeal very much to me
(although I'm sure it is chosen for very good reasons in the
context of the Grid Troubleshooting project.) On the other hand,
the fixed positioning of the data is something that will depend
very much on what information is there going to be.

* * *

For that latter reason, I'd like to kindly ask once more the
developers of the pS services to provide me with at least a few
representative examples of messages that could be generated during
the operation of their service. Otherwise I'll have to base my
proposal on purely theoretical considerations -- perhaps not the
best way to go.

* * *

Brian, Jochen, once again many thanks for your feedback.

Cheers,
Luchesar

Jochen Reinwand wrote:
Hi Brian, hi all,

I think this Best Practice is perhaps too complex for us. Things
like timestamp and level are already part of the syslog
functionality anyway.
For perfSONAR we already have error codes, that can be used more
or less directly also for syslogging. I would really recommend
this! This way the syslog receives the same messages and codes as
the user does via the perfSONAR response.
There are some message that do not necessarily issue a perfSONAR
error response. I think these are all typical UNIX system error
like "File not found", "Execution failed" and so on. These should
be send to syslog in a very simple form to distinguish them from
the perfSONAR errors.
I'm not sure whether a key value pair style string is really
handy. Most tools don't use such a format for syslogging. A
string with fixed positions with fixed meanings is normally
easier to handle and makes it easier to enforces a common
discipline.

regards,
Jochen

On Sunday 20 April 2008 17:40, Brian Tierney wrote:
Hi all:

I am working on a Grid Troubleshooting project where for logging
we
agreed on a syslog format where everything is 'name=value' pairs.
We felt that this was the best compromise of easy to parse and
easy to
read.

We also said that each log line should have a required
'event=eventname', where eventname should be similar to a java
class name.

I recommend that perfSONAR logs use this format as well.

For more information.

http://www.cedps.net/index.php/LoggingBestPractices

This format is the same as the EGEE "Middleware Security Audit
Logging
Guidelines": http://edms.cern.ch/document/793208

Cheers.

Luchesar V. ILIEV wrote:
Hi guys,

This mail is addressed to all pS service developers. As you
know, last
week in Zagreb we discussed the issues of monitoring the
deployed pS
services to ensure they are operating as intended, and really
serving
the end-users' requests up to the users' satisfaction.

As a next step in this process (having used so far ICMP, HTTP
GET, and
perfSONAR EchoRequest/EchoResponse messages,) in order to
achieve more
in-depth monitoring, which could, in turn, provide a more true-
to-life
view on the functioning of the services, we've decided in
Zagreb to
utilize the well-known and widely-used mechanism of sysloging.

I've the task to prepare a proposal for a common format of the
syslog
messages, or (depending on your feedback) at least some
guidelines,
which would ensure having consistency amongst the different
service's
messages' general structure.

Therefore, I'd like to ask you to provide me either with a full
list of
the possible messages that the service you're working on could
eventually write to the log, or provide me with information on
the
structure of the (different types of these) messages.

It could be helpful to have also some indication about the
severity of
each message (from your point of view, of course): e.g., debug,
information, warning, error, critical failure, etc. As decided in
Zagreb, you have to synchronize this with the Service Desk --
if you
could do this beforehand, it would be great. If not, I would be
OK with
your own (from service developer's point of view) classification.

I'll appreciate if you could send me your feedback before the
end of
next business week: that is, 25th of April inclusive.

If there's something unclear in my e-mail and specifically
about my
request, please do contact me.

Many thanks in advance,

Cheers,
Luchesar


Attachment: signature.asc
Description: OpenPGP digital signature


Archive powered by MHonArc 2.6.16.

Top of Page