perfsonar development work

[Jason Zurawski <>]

> > > 1) ./perfSONAR pre-install
> > >
> > > This will ask if the user already has ant, tomcat, axis and exist.
> > > * If the user says 'yes' to any of them, it will ask the path where 
> > > it is located.
> > > * If the user says 'no' to any of them, it will ask if the user 
> > > wants to install them manually or download them automatically and 
> > > install it.
> > > ** If the user chooses 'Manually', the script will ask the user to 
> > > run the same script again after installing the dependency. It will 
> > > also suggest download sites where the user can find the software for 
> > > downloading
> > > ** If the user chooses  'Automatically' the script will  know a list 
> > > of sites where the software is available and it will download the 
> > > software and install it.
> > >
> > > The script will also prompt the user for the port number to run 
> > > tomcat on. eXist can run on tomcat itself and hence on the same port 
> > > as tomcat.
> >
> >
> > I am for the creation of some sort of repository (apt/yum) that we 
> > maintain and contains our pre-installation software (preferably 
> > packaged by someone else such as the individual linux vendors or the 
> > original authors).
> I would prefer to use URLs for applications such as exist and tomcat. 
> But I do not have a strong inclination towards it. Is there a reason 
> why you would want to keep a copy of these applications on ps servers?


URLs change (they did since the first writing of the readme to the 
second), and isn't the issue at hand trying to make the entire process a 
little more pleasant?  If the user has the option of 'apt-get install 
tomcat' over read a document, insert url, download package and configure 
package what is better?  It doesn't have to be a pS server, there are 
many 'outside' repositories that would do the job but by default the 
rhel and other standard repositories won't give access to all of the 
pre-reqs we are using.


> >   I don't however think that having the pS installer itself install 
> > the software is in our best interest.  The installer then requires 
> > administrator priv. to access system folders to install items like 
> > java or tomcat (as well as configure iptables or other firewall 
> > software) where now it can be installed by an end user albeit needing 
> > one or two manual system level tweaks.
>
> I am not suggesting to install java manually. The list says tomcat, 
> axis, ant and exist. About tomcat, why is access to system folders 
> required?
> If the user wants us to install it automatically, we install it in the 
> user's folder. Otherwise, the user installs it manually. We won't be 
> able to do everything but we can achieve a basic installation in a 
> simpler manner I think.


If you are going to install tomcat/axis/ant/exist then java is required 
and therefore it's installation becomes a manual process.  Because we 
are using the newest version this almost always forces someone to go 
outside of the packages distributed with the distro looking for it.  All 
our systems protect the /usr and /opt tree by default, so when I install 
the pre-reqs I must sudo/su as needed.  I have never had a problem with 
running the pS installer as a regular user once the pre-reqs are in the 
priv. areas however.  If a user really wants to install this in their 
home, great, but that means only that user is running a service that 
should be for all.  Ideally a sysadmin with root will be doing the 
default but why should we mandate our installer be an admin only process 
when it doesn't need to be?




> > > 2) ./perfSONAR configure
> > > The script configures the service (i.e. put the xml file in exist, 
> > > log properties, etc)
> > >
> > > 3) ./perfSONAR install
> > > At this point the script knows where all the packages are located. 
> > > It simply installs perfSONAR (i.e. copies all necessary jar files to 
> > > tomcat) and deploys the service
> > >
> > > 4) ./perfSONAR test
> > > This script helps in testing the service. The test scripts will have 
> > > the capability to test each part of the service. So, if the user 
> > > comes back to us with support questions, a quick and complete 
> > > diagnosis of the installation can be made. The test will ask the 
> > > user what he/she is interested in testing.
> >
> >
> > We handle points 2 though 4 right now, and although not perfect we 
> > are starting to get enough feedback to start making this process better.
>
> > I am all for ease, but we are not going to please everyone all the 
> > time.  We could build rpm/apt bundles of pS with 'defaut' settings 
> > but in my opinion we are offering a rather personalized system that 
> > requires lots of questions to be effective (such as the initial setup 
> > of the various config files or the XML messages used for testing).  
> > If we offer a 'one size fits most' release we can of course just rely 
> > on sane defaults but then the documentation to manually configure 
> > will become an issue vs. questions during the install.
> >
> > > Just by typing ./perfSONAR, the script would do all the above in 
> > > sequence. There would then be no classpath configuration (as ant 
> > > takes care of that)
> >
> >
> > Ant (as well as  the current deployment and test scripts) will 
> > temporarily take care of the CLASSPATH, but the variables must still 
> > be set to use the service afterwards.
>
> If we use ant for compiling, deploying and testing, the user never has 
> to deal with the CLASSPATH. Of course for special cases such as RRD 
> tool, the LD_Library_path needs to be set. But its far easier to set 
> just that than to set both that and the classpath.


So when the user makes handmade requests they must create and configure 
new ant targets?  Ant scripts must be used by init.d to start pS on 
reboot (also requiring some admin level configuration)?  I don't see the 
practicality of allowing ant to be the keeper of the CLASSPATH when the 
entire point of having environmental variables is to allow everyone to 
use them.  Frankly there are many java applications that our target 
audience is more than likely already using (globus for instance) that 
require mucking with these variables, so a manual editing does not seem 
out of the question.



> > I agree this is a huge obstacle for some people, and a royal pain, 
> > but if pS is to work system wide then env. variables need to be set 
> > in a system wide location by admin.  If we move towards a 'one step' 
> > installation, then either the root user must perform the entire the 
> > process (to ensure all of the special operations will go on without 
> > issue) or we avoid touching sensitive areas by using only user level 
> > operations (the route we choose).
> >
> >
> > > The fact is, we have a lot of ant targets which can already do a 
> > > majority of the above tasks. In perfSONAR 1.0, it was decided not to 
> > > use ant but instead go with replacement perl scripts. While I 
> > > understand the easiness of using perl scripts, I really think it 
> > > needs to be a combination of both (i.e., perl scripts are simply 
> > > making use of ant targets underneath once ant is installed)
> >
> >
> > It still doesn't take away the requirement of a working version of 
> > java and in turn a working version of ant, which may or may not be 
> > available on all systems through package management (especially the 
> > v. 5 release of java).  For debian/ubuntu especially the choice of 
> > java is blackdown (not in v 5) or manual rpm conversion through alien 
> > as available options; this already is a barrier when it comes to 
> > installing pS.  
> Right, we solve it by using ant which downloads from a configured list 
> of URLs.
> (Ofcourse, if the URL is broken, we might need to take measures but 
> this seems to be a much easier way to deal with installing applications).


You seem to be missing the point here, before the ant scripts work you 
need ant, and before installing ant you need java.  Just by changing the 
installer into an automated ant script does not negate these 
requirements or really take away from much of the workload of installing 
the pre-reqs. 


> > The general thought process for the current setup was that every *nix 
> > system constructed in the last 5 years can use sh and perl by 
> > default, so why not install packages through that mechanism.
>
> > If we were to go the route of some sort of hybrid sh/perl/java/ant 
> > system we are going to run into many of the same problem 
> > (permissions, still needing to set env variables) but we also involve 
> > additional pieces of software that may not be configured correctly 
> > before the process even gets under way.
> I totally agree that we should use perl where possible because it 
> could be easier for certain users. If perl calls ant targets 
> underneath, we will be solving quite a few problems right? 


I don't disagree it would be easier, but you are still basing your 
argument around the availability of a fully functioning java and ant 
setup, which for many systems is the largest part of the pre-req 
barrier.  Most apt/yum repos, even the system defaults, offer tomcat, 
ant, and axis.  For RH distros especially, java is not available b/c of 
license issues and either gcj is substituted or java must be downloaded 
from other sources (from the Sun site or if someone has taken the time 
to package the latest version).

-jason