perfsonar development work

[Loukik Kudarimoti <>]

Nicolas previously mentioned that we should try to achieve a balance in 
the emails that we send to perfsonar-user as the users won't have enough 
time to read them. I agree. I think this thread is too detailed. Hence I 
am moving it to perfsonar-dev.


Jason Zurawski wrote:
> > 1) ./perfSONAR pre-install
> >
> > This will ask if the user already has ant, tomcat, axis and exist.
> > * If the user says 'yes' to any of them, it will ask the path where 
> > it is located.
> > * If the user says 'no' to any of them, it will ask if the user wants 
> > to install them manually or download them automatically and install it.
> > ** If the user chooses 'Manually', the script will ask the user to 
> > run the same script again after installing the dependency. It will 
> > also suggest download sites where the user can find the software for 
> > downloading
> > ** If the user chooses  'Automatically' the script will  know a list 
> > of sites where the software is available and it will download the 
> > software and install it.
> >
> > The script will also prompt the user for the port number to run 
> > tomcat on. eXist can run on tomcat itself and hence on the same port 
> > as tomcat.
>
>
> I am for the creation of some sort of repository (apt/yum) that we 
> maintain and contains our pre-installation software (preferably 
> packaged by someone else such as the individual linux vendors or the 
> original authors).
I would prefer to use URLs for applications such as exist and tomcat. 
But I do not have a strong inclination towards it. Is there a reason why 
you would want to keep a copy of these applications on ps servers?

>   I don't however think that having the pS installer itself install 
> the software is in our best interest.  The installer then requires 
> administrator priv. to access system folders to install items like 
> java or tomcat (as well as configure iptables or other firewall 
> software) where now it can be installed by an end user albeit needing 
> one or two manual system level tweaks.

I am not suggesting to install java manually. The list says tomcat, 
axis, ant and exist. About tomcat, why is access to system folders required?
If the user wants us to install it automatically, we install it in the 
user's folder. Otherwise, the user installs it manually. We won't be 
able to do everything but we can achieve a basic installation in a 
simpler manner I think.


> I also think this complicates the issue further by forcing us to spend 
> a lot of time supporting and maintaining the nuances of other software 
> packages (especially how we intend to handle their individual 
> requirements during our own installation process).  If the supporting 
> software is going to be a major barrier to widespread adoption, we 
> definitely should consider moving to a more inclusive development model.
Dependencies are a headache. No doubt. I think discussion on reducing 
dependencies should be in a different thread.
> > 2) ./perfSONAR configure
> > The script configures the service (i.e. put the xml file in exist, 
> > log properties, etc)
> >
> > 3) ./perfSONAR install
> > At this point the script knows where all the packages are located. It 
> > simply installs perfSONAR (i.e. copies all necessary jar files to 
> > tomcat) and deploys the service
> >
> > 4) ./perfSONAR test
> > This script helps in testing the service. The test scripts will have 
> > the capability to test each part of the service. So, if the user 
> > comes back to us with support questions, a quick and complete 
> > diagnosis of the installation can be made. The test will ask the user 
> > what he/she is interested in testing.
>
>
> We handle points 2 though 4 right now, and although not perfect we are 
> starting to get enough feedback to start making this process better.

> I am all for ease, but we are not going to please everyone all the 
> time.  We could build rpm/apt bundles of pS with 'defaut' settings but 
> in my opinion we are offering a rather personalized system that 
> requires lots of questions to be effective (such as the initial setup 
> of the various config files or the XML messages used for testing).  If 
> we offer a 'one size fits most' release we can of course just rely on 
> sane defaults but then the documentation to manually configure will 
> become an issue vs. questions during the install.
>
> > Just by typing ./perfSONAR, the script would do all the above in 
> > sequence. There would then be no classpath configuration (as ant 
> > takes care of that)
>
>
> Ant (as well as  the current deployment and test scripts) will 
> temporarily take care of the CLASSPATH, but the variables must still 
> be set to use the service afterwards.

If we use ant for compiling, deploying and testing, the user never has 
to deal with the CLASSPATH. Of course for special cases such as RRD 
tool, the LD_Library_path needs to be set. But its far easier to set 
just that than to set both that and the classpath.

> I agree this is a huge obstacle for some people, and a royal pain, but 
> if pS is to work system wide then env. variables need to be set in a 
> system wide location by admin.  If we move towards a 'one step' 
> installation, then either the root user must perform the entire the 
> process (to ensure all of the special operations will go on without 
> issue) or we avoid touching sensitive areas by using only user level 
> operations (the route we choose).
>
>
> > The fact is, we have a lot of ant targets which can already do a 
> > majority of the above tasks. In perfSONAR 1.0, it was decided not to 
> > use ant but instead go with replacement perl scripts. While I 
> > understand the easiness of using perl scripts, I really think it 
> > needs to be a combination of both (i.e., perl scripts are simply 
> > making use of ant targets underneath once ant is installed)
>
>
> It still doesn't take away the requirement of a working version of 
> java and in turn a working version of ant, which may or may not be 
> available on all systems through package management (especially the v. 
> 5 release of java).  For debian/ubuntu especially the choice of java 
> is blackdown (not in v 5) or manual rpm conversion through alien as 
> available options; this already is a barrier when it comes to 
> installing pS.  
Right, we solve it by using ant which downloads from a configured list 
of URLs.
(Ofcourse, if the URL is broken, we might need to take measures but this 
seems to be a much easier way to deal with installing applications).

> The general thought process for the current setup was that every *nix 
> system constructed in the last 5 years can use sh and perl by default, 
> so why not install packages through that mechanism.
I totally agree that we should use perl where possible because it could 
be easier for certain users. If perl calls ant targets underneath, we 
will be solving quite a few problems right?

Loukik.
> If we were to go the route of some sort of hybrid sh/perl/java/ant 
> system we are going to run into many of the same problem (permissions, 
> still needing to set env variables) but we also involve additional 
> pieces of software that may not be configured correctly before the 
> process even gets under way.
>
>
> -jason