perfsonar development work

[Guilherme Fernandes <>]

Nicolas Simar wrote:

> Hi,
Hi Nicolas

> an interesting story I got about measurement cron scheduling and 
> changes daylight saving time. It was reported by EGEE, they have 
> measurements probes running in different countries on different 
> timezones (which is our case basically).
>
> In those different timezone, the change to daylight saving isn't done 
> on the same day.
>
> As all the scheduling were done with cron, running with local time, 
> they ended-up with overlapping measurements. Depending on the type of 
> measurement you do, this can have some impact or not at all.
>
> This type of behavior will directly hit us. We need to take that into 
> account.
>
> Have we got mechanisms to prevent this behavior?
The problem I see in this is that we could want both behaviours. 
Currently Java uses the clock set in the machine, so the schedule will 
run on local time also, but this might be the wanted behaviour (e.g. I 
want to measure my network latency at 2 PM, this would be independent of 
whether I'm in daylight savings or not).

As you said, this is a problem when we have overlapping measurements 
(after daylight savings). The best solution I can think of is either to 
have a trigger that defines the passed schedule time information as UTC 
(or not) or a trigger defining the measurement should (or not) suffer 
daylight changes.

Regards,

Guilherme

> EGEE has been planning for a while to work on a scheduler not 
> dependent on the local cron which would have prevented that behavior. 
> I can put you in touch with them if you wish.
>
> Best regards,
> Nicolas
>
> Guilherme Fernandes wrote:
>
> > Roman Lapacz wrote:
> >
> > >  wrote:
> > >
> > > > Hi Guys,
> > >
> > >
> > >
> > >
> > > Hi Guilherme
> > Hi Roman, sorry for the late response..
> >
> > > > Before the release process started I was developing a cron-like 
> > > > scheduler for the CL-MP.
> > >
> > >
> > >
> > >
> > > Did you take a look at the scheduler implemented by Maciej. It can 
> > > be found in the base and, according to Maciej, it's generic and 
> > > ready to be taken when scheduling is needed. He used it to implement 
> > > LS registration and LS clean up stuff.
> > Yes, I had already looked at the code Maciej had implemented
> > (SimpleScheduler) before I started my implementation. To extend it would
> > be the same as making the actual component I made, but I didn't really
> > make it generic enough to be used by other services yet. I actually use
> > the SimpleScheduler to run the Cron Component every minute.
> >
> > I just found the implementation Loukik developed of a more complex
> > Scheduler that uses a XML database, haven't looked at it yet though,
> > maybe it's similar to mine. Jeff had already pointed this out, but at
> > the time the work on the Cron component was already almost done.
> >
> > > > The logical structure of the code is done, I now need to define the 
> > > > schemas for this communication. I have attached some examples of 
> > > > what I was thinking for the schemas, these are just sketches, I 
> > > > would really appreciate your opinions.
> > > >
> > > > Some background: So CL-MP currently does on-demand measurements 
> > > > using command-line tools. This new component allows the CL-MP to 
> > > > schedule measurements using a structure very similar to unix cron. 
> > > > The user will request CL-MP to add the measurement he wants 
> > > > executed (in the requestCron.xml example it's an owamp measurement) 
> > > > and the scheduling information. The scheduled measurements will be 
> > > > identified by an entryId (these will be unique in some way).
> > >
> > >
> > >
> > >
> > > I think the key could be the right element here. The service would 
> > > return unique key to identify a measurement.
> > Yes, I agree we could use the key element in this case.
> >
> > > > The measurements results are stored in a SQL MA. The user should be 
> > > > able to choose to which MA he wants the data sent.
> > > >
> > > > In the examples I used different namespaces for each action, these 
> > > > could be put inside the same namespace 'cron' and identified by a 
> > > > 'action' parameter. I've initially put the MA IP address as an 
> > > > attribute of the <nmwg:messsage> element, I think this was 
> > > > suggested by Jeff before, but this isn't actually currently 
> > > > supported by the nmwg schema. So maybe it should go also as a 
> > > > parameter in the cron metadata.
> > >
> > >
> > >
> > >
> > > I think the attribute in the message element is not a good place for 
> > > MA's address. What if your request would have 2 sub-requests (2 data 
> > > triggers) and results of them would have to be stored in separate 
> > > MAs. I prefer to have it in some metadata element directly related 
> > > with a measurement.
> > You're right, although it doesn't currently supports more than 1 request
> > in the same scheduler, this could be a wanted functionality in the 
> > future.
> >
> > > > <nmwg:message type="MeasurementRequest" id="msg1"
> > > >     ma="150.162.248.15:8080" 
> > > > xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/";
> > > >     xmlns:nmwgt="http://ggf.org/ns/nmwg/topology/2.0/";
> > > >     xmlns:owamp="http://ggf.org/ns/nmwg/tools/owamp/2.0/";
> > > >     xmlns:cron="http://ggf.org/ns/nmwg/tools/cron/modify/2.0/";>
> > > >
> > > >     <nmwg:metadata id="owamp">
> > > >         <owamp:subject id="sub2">
> > > >             <nmwgt:endPointPair>
> > > >                 <nmwgt:src type="ipv4" value="1.1.1.1" />
> > > >                 <nmwgt:dst type="ipv4" value="2.2.2.2" />
> > > >             </nmwgt:endPointPair>
> > > >         </owamp:subject>
> > > >         
> > > > <nmwg:eventType>http://ggf.org/ns/nmwg/tools/owamp/2.0</nmwg:eventType> 
> > > >
> > > >         <owamp:parameters id="param2">
> > > >             <nmwg:parameter name="direction">to</nmwg:parameter>
> > > >             <nmwg:parameter name="count">5</nmwg:parameter>
> > > >             <nmwg:parameter 
> > > > name="portRange">2500-2999</nmwg:parameter>
> > > >         </owamp:parameters>
> > > >     </nmwg:metadata>
> > > >
> > > >     <nmwg:metadata id="cron" metadataIdRef="owamp">
> > > >   
> > >
> > >
> > >
> > >
> > > My understanding of nmwg chaining is that the metadataIdRef 
> > > attribute in metadata element is used for base chaining. So only 
> > > when we want to divide metadata block in order to reuse their parts. 
> > > In this case if I decide that the metadata block with id="cron" will 
> > > not be used elsewhere I could create one metadata element with 
> > > contents of metadata id=owamp" and metadata id="cron". This of 
> > > course would be wrong. That is why other chaining approach should be 
> > > used:
> > This is exactly the type of thing I wanted to know, I don't really have
> > a deeper understanding of the nwmg schema, you're probably right. Maybe
> > someone else wants to comment on this? Jason, Jeff?
> >
> > Thanks,
> >
> > Guilherme
> >
> > >    <nmwg:metadata id="cron">
> > >        <cron:subject id="iusub2" metadataIdRef="owamp"/>
> > >        <cron:parameters id="param1">
> > >            <nmwg:parameter name="minutes">20-30</nmwg:parameter>
> > >            <nmwg:parameter name="hours">*</nmwg:parameter>
> > >            <nmwg:parameter name="dom">*</nmwg:parameter>
> > >            <nmwg:parameter name="months">*</nmwg:parameter>
> > >            <nmwg:parameter name="dow">*</nmwg:parameter>
> > >        </cron:parameters>
> > >        <nmwg:eventType>
> > >            http://ggf.org/ns/nmwg/tools/cron/modify/2.0
> > >        </nmwg:eventType>
> > >    </nmwg:metadata>
> > >
> > >
> > > Jason, Jeff, what's your view on this?
> > >
> > >
> > >
> > > >         <nmwg:eventType>
> > > >             http://ggf.org/ns/nmwg/tools/cron/modify/2.0
> > > >         </nmwg:eventType>
> > > >         <cron:parameters id="param1">
> > > >             <!-- if entryId is missing, considered new entry -->
> > > >             <nmwg:parameter name="entryId">1</nmwg:parameter> 
> > > >             <nmwg:parameter name="minutes">20-30</nmwg:parameter>
> > > >             <nmwg:parameter name="hours">*</nmwg:parameter>
> > > >             <nmwg:parameter name="dom">*</nmwg:parameter>
> > > >             <nmwg:parameter name="months">*</nmwg:parameter>
> > > >             <nmwg:parameter name="dow">*</nmwg:parameter>
> > > >         </cron:parameters>
> > > >     </nmwg:metadata>
> > > >
> > > >     <nmwg:data id="data" metadataIdRef="cron" />
> > > >
> > > > </nmwg:message>
> > > >   
> > > > ------------------------------------------------------------------------ 
> > > >
> > > >
> > > > <nmwg:message type="MeasurementRequest" id="msg1"
> > > >     xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/";
> > > >     xmlns:cron="http://ggf.org/ns/nmwg/tools/cron/list/2.0/";>
> > > >
> > > >     <nmwg:metadata id="meta1">
> > > >         <schedule:subject id="sub1"></schedule:subject>
> > > >         
> > > > <nmwg:eventType>http://ggf.org/ns/nmwg/tools/cron/list/2.0</nmwg:eventType> 
> > > >
> > > >         <cron:parameters id="param1">
> > > >             <nmwg:parameter name="entries">all</nmwg:parameter>
> > > >         </cron:parameters>
> > > >     </nmwg:metadata>
> > > >   
> > >
> > >
> > >
> > >
> > > As I wrote the idea of key could fit here
> > >
> > >
> > >     <nmwg:metadata id="meta1">
> > >          <nmwg:key>
> > >            <nmwg:parameters id="param1">
> > >              ...
> > >            </nmwg:parameters>
> > >          </nmwg:key>
> > >     </nmwg:metadata>
> > >
> > >     <nmwg:data id="1" metadataIdRef="meta1" />
> > >
> > >
> > >
> > >
> > >
> > > >     <nmwg:data id="1" metadataIdRef="meta1" />
> > > >
> > > > </nmwg:message>
> > > >   
> > > > ------------------------------------------------------------------------ 
> > > >
> > > >
> > > > <nmwg:message type="MeasurementRequest" id="msg1"
> > > >     xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/";
> > > >     xmlns:cron="http://ggf.org/ns/nmwg/tools/cron/remove/2.0/";>
> > > >
> > > >     <nmwg:metadata id="meta1">
> > > >         <nmwg:eventType>
> > > >             http://ggf.org/ns/nmwg/tools/cron/remove/2.0
> > > >         </nmwg:eventType>
> > > >         <cron:parameters id="param1">
> > > >             <nmwg:parameter name="entryID">1</nmwg:parameter>
> > > >         </cron:parameters>
> > > >     </nmwg:metadata>
> > > >
> > > >     <nmwg:data id="1" metadataIdRef="meta1" />
> > > >
> > > > </nmwg:message>
> > > >   
> > >
> > >
> > >
> > >
> > > The same here: key.
> > >
> > >
> > >
> > > Roman