perfsonar development work

[Nicolas Simar <>]

Jochen Reinwand wrote:
> Hi all,
>
> some things really need to be clarified here. Things that are related to the 
> LHC commitment and that need to be clarified before(!) the Berkeley meeting.
>
> Two reasons for this:
>
> 1. Here in Erlangen everyone has her/his own special areas she/he is "our 
> specialist". If there is something about a specific topic (especially if some 
> "joint development" with other participants is planed) we should really have 
> the right person in the meeting. For LHC related issues we have at least: 
> Verena (BWCTL, schemas, perfSONAR service), Roland (perfSONAR-UI plug-in), 
> and myself (Hades, perfSONAR service).
>
> 2. For Hades a lot of things, and I really mean a LOT of things need to be 
> done if some special requirements have to be met for the LHC commitment. But 
> at the moment there is quite a lot of obscurity about the actual 
> requirements. Time for choosing the "right" developments to be done is 
> running out! I rather would start development before the Berkeley meeting, 
> not afterwards ;-)

Seems that the current solution suggested to LHC is the one of an 
applicance. (Hades for the time being).
But the T1 want the possibility to communicate with the T2-T3 and get 
themselves organised. So in a second phase, we need to ensure the 
capability to interface in a way or another with the T2s for OWD 
measurements. I hope this does help a little bit. You can talk wiht 
Loukik if you wish to get more info, he was at the meeting.
They need mostly regular scheduled and a bit of on-demand.

It seems that for the time being the appliance is the solution to work 
on, with some hooks to allow regular tests on other measurements points 
hosted by T2-T3. (but more in a second phase). I'll be able to know more 
when I'll be back as HAns and Marian who attended the meeting were on 
holiday (So didn't get much feedback)

> The Hades requirements are the greatest problem, because there are completely 
> different scenarios:
>
> 1. Only Hades is used in a fully managed service. No problem! Everythings 
> ready. You can have it tomorrow.
>
> 2. As before only full managed Hades service, but on-demand measurements 
> should be possible. For on-demand measurements we are planing to use the 
> OWAMP protocol. We are really sure, that this is the best solution since 
> OWAMP is an accepted standard and being compatible to it is much better than 
> developing something completely different. This can be achieved in many 
> different ways, but I don't want to bore you with technical details. Would 
> not be too much development work for us. A simple but perhaps even sufficient 
> workaround is to just start an OWAMP daemon on the Hades machines and ignore 
> the interference issues.
>
> 3. In the previous scenario Hades will not be installed on machines that are 
> not managed by us. Only we install and configure the machines. For other 
> (unmanaged) machines only on-demand is possible via OWAMP. If we want to 
> enable Hades for other machines (additionally to the machines from the 
> managed service) a lot of issues arise. For Example we have to build some 
> sort of installable software release. That's a lot of work! Also Hades needs 
> one(!) central server for managing all machines and for the data collection. 
> Access is necessary via SSH. If the machine does not provide this, it cannot 
> take part in the measurement network. Our (Erlangen) opinion is that we 
> should avoid this scenario by all means. This will not be good publicity for 
> perfSONAR and the whole project...
>
> 4. Hades is not managed in any way by us. We are just giving them (who ever 
> "they" are) our software. That would mean we have to put a lot of effort in 
> release management, documentation and support. I doubt that "they" will have 
> anything usable since end of this year. Especially if "they" are ordinary 
> administrators from different administration domains not trusting one 
> another ;-)
>
> 5. There is the idea that we can use Hades in parallel to the AMI scheduler 
> doing scheduled one-way delay measurements using both system. I totally agree 
> with Jeff that this is most likely not what we want, because we end up with 
> two different scheduling systems and MA services. Especially the users will 
> have to deal with two different systems! The two systems will _not_ look like 
> one integrated service for the users. Once again I will not bore you with the 
> technical details. But just ask for further information if you don't believe 
> Jeff and me ;-)

I think I start understanding the picture ;-)

> On the software side there is no great problem in making it possible to run 
> the two systems (Hades and AMI) in parallel. It would be pretty much the same 
> from our side as with 2. above.
>
> 6. Don't use Hades at all, only AMI/OWAMP. There would not be much development 
> on our side ;-)
> Besides the OWAMP MP, I assume.
>
> For all these scenarios I don't really see the need for a "joint development" 
> during the Berkeley meeting. A decision has to be made on which way to go. 
> But this decision should be made before the Berkeley meeting, as I stated 
> above. Also I doubt that we (Erlangen and Jeff) are the right persons to make 
> this decision. I also doubt that a decision only on the measurement 
> methodology would make it clear which system to choose.
>
> We really need this decision asap, because at the moment we are more or less 
> "blocked". The above scenarios require different development work to be done 
> and we don't want to work on things that will turn out to be more or less 
> useless. Instead we should have done something completely different for which 
> we don't have enough time left then.
>
>
> For BWCTL a "joint development" would not make sense, too. The on-demand 
> system (BWCTL MP) and the scheduling system (AMI) are completely separate.
>
> The only issue that might possibly be discussed in Berkeley are the schemas. 
> But the schema issue is not that urgent and it might be more useful to 
> discuss it in Sevilla having more people available. Also it might be useful 
> to have Nina joining the discussion. At least she's the person who will have 
> to deal with the schemas after they are implemented by the services.
>
>
> Not really my business, but just an idea: Kick the release management from the 
> agenda. That's definitively a JRA1 meeting topic. A lot more persons involved 
> should be available in Sevilla. Most of the people available in Berkeley are 
> not directly involved in the release management.
>
>
> regards,
> Jochen (and the team in Erlangen)

Szymon will follow-up on the agenda creation.
I can't tell you much more at this stage about the direction to be taken 
for Hades.

Thanks.
Nicolas

> On Friday 10 August 2007 10:36, Nicolas Simar wrote:
>
> > Eric L. Boyd wrote:
> >
> > > Gang,
> > >
> > > So, I agree one key topic is to review the LHC commitment (which means
> > > participation by all the developers building things we promised LHC). As
> > > important is to lay out a roadmap for the next 12 months for LHC support
> > > going forward.
> >
> > I think we need for LHC:
> >
> > - to have a clear understanding about the layout of the T0-T1 centers,
> > where exactly to place the different services and what information they
> > need to provide. [JM, EB, NS, LK, JB, Erlangen]
> > (e.g. for T0 and T1 sites, the utilisation, errors and discard does
> > cover the last routers, all the internal routers, etc Where to place the
> > BWCTL tools: as close to the end-machines, behind the firewall, as close
> > to the circuit ends. They all have pros and cons).
> > What MA to deploy and where? (1/2 day)
> >
> > - List clearly the questions/information to be answered to the T0-T1
> > sites + start answering them (2 hours) [JM, EB, NS, LK, JB, Erlangen]
> >
> > -  Understand how to present the data and how to link what is seen as L2
> > and at L3. (1/2 day) [MY/MH, JB, AH/DS remote(?), NJ, JM]
> >
> > - clarify the different scenarios for Hades-owamp and the solution to
> > provide. (1 to 2 hours) [JB, Erlangen]
> >
> > - Progress/finalise the BWCTL schema (1/2 day) [JB, Erlangen, MS, JZ]
> >
> > - Progress/finalise the Hades measurement methodology and the schema
> > (1/2 day) [JM, JB, Erlangen]
> >
> > - test inter-op Hades/OWAMP solutions + on-demand scheduler if not done
> > before. (1/2 day)
> >
> > - Visualising the data [NJ, DS/AH remote, JB, SLAC?]
> >
> > - what's needed to be part of a perfsonar release for the ping MP and
> > the L2 status MP: [MH/MY, SLAC, Release Management team]
> >
> > - As the RRD MA and the visualisation is pretty much on track, it is not
> > foreseen to have them as discussion topics (unless otherwise specified).
> >
> > - Support contracts [NS, EB, ...]
> >
> >
> > Other:
> > - LS I would like to have Maciej, Jason and Martin to give an status of
> > the mLS and the topics that needs addressing, so that we can define if
> > we need to have a session on that topic.
> >
> > -  Easing the perfSONAR web-service installation [Maxim, release
> > management team, ?] : where to go and best practices to adopt
> >
> > - perfsonar 3.0 release: status and next steps.
> >
> > - perfsonar.net [LK, EB, NS]
> >
> > - Review the collaboration and information processes
> >
> > As there are some non-overlapping topics, for more efficiency, some of
> > this work will be organised in parallel session.
> >
> >
> > So for the attendance (EU) we got:
> > - MY or MH [LZR L2 status]
> > - LK (?)
> > - One member of Release Management team (LK to define)
> > - Erlangen [BWCTL, Hades, visualisation]
> > - NS and/or ST
> > - PSNC (tbc)
> >
> > We can send 6-7 people there. Could you please Loukik Erlangen, LZR and
> > PSNC confirm the names of who can go there?
> >
> > I am not keen on having this particular meeting open to anybody!
> > This is a developer working meeting and not a dissemination thing.
> >
> >
> > You are welcome to comment on those topics/add other ones of direct
> > interest to you so that you can make this trip worthwhile.
> >
> > Cheers,
> > Nicolas
> >
> >
> > > A second topic is to review development and deployment plans for
> > > individual networks.
> > >
> > > Others?
> > >
> > > --Eric
> > >
> > > Mark Yampolskiy wrote:
> > >
> > > > Hi Nicolas, all,
> > > >
> > > > is participation of M&M by this meeting expected?
> > > >
> > > > From our point of view, it is crucial that the tools effectively
> > > > support the needs of operations. Thus, the interoperability between
> > > > and integration of tools for LHCOPN operations is a very critical
> > > > issue. Furthermore all these tools should be treated as a bundle and
> > > > discussion about possible new futures should take it into account and
> > > > target needs of operations and of end-users (LHC).
> > > >
> > > > Cheers,
> > > > M&M
> > > >
> > > > Nicolas Simar schrieb:
> > > >
> > > > > It is in a month time. We need to identify who needs to go there this
> > > > > week as the flight prices increases, otherwise, we will have to
> > > > > cancel the meeting.
> > > > >
> > > > > Nicolas
> > > > >
> > > > > Szymon Trocha wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > I would like to start collecting items for the next meeting in
> > > > > > Berkeley
> > > > > > http://wiki.perfsonar.net/jra1-wiki/index.php/Berkeley_Dev_Sep_11_2007
> > > > > >
> > > > > > The initial idea is that it will coordinate efforts to build and
> > > > > > deploy measurement tools for the LHC. I think based on the scope of
> > > > > > discussion and topics we will identify the people who should attend
> > > > > > the meeting.
> > > > > >
> > > > > > So please send me your proposals of specific topics and problems to
> > > > > > discuss.
> > > > > >
> > > > > > Regards,

--
Nicolas
______________________________________________________________________

Nicolas Simar
Network Engineer

DANTE - www.dante.net

Tel - BE: +32 (0) 4 366 93 49
Tel - UK: +44 (0)1223 371 300
Mobile: +44 (0) 7740 176 883

City House, 126-130 Hills Road
Cambridge CB2 1PQ
UK
_____________________________________________________________________