PROOF Installation Notes



The following notes describes how to get started with PROOF (Parallel ROOT Facility) as of October 10, 2005, on a Linux machine with gcc version 3.2.3 (ROOT head version 5.05/01).

The current version of this document describes how to set up the PROOF master server and slave nodes on the same machine as the client as a first test. To continue with a cluster installation, consult the
$ROOTSYS/README/README.PROOF document.

For latest information, please consult $ROOTSYS/README/README.PROOF that comes with the ROOT installation.


1. ROOT Installation From Source


Follow the installation instructions on the ROOT homepage, or try the following.
Grab root from the cvs server, configure [in this case for linux] and build it:

> setenv CVSROOT :pserver:cvs@root.cern.ch:2401/user/cvs
> cvs -d $CVSROOT login
Logging in to :pserver:cvs@root.cern.ch:2401/user/cvs
CVS password: [standard password]
> cvs -d $CVSROOT co root
[for the head version]
> cd root
> ./configure linux
[to enable thread support use '--enable-thread' (requires the system library libpthread) - useful for large clusters during job submission]
> make

Set the ROOTSYS (to your install directory) and LD_LIBRARY_PATH (to $ROOTSYS/lib) environment variables in your login script, and add $ROOTSYS/bin to your PATH.

Make sure that the rootd and proofd daemons have been built:
> ls $ROOTSYS/bin/rootd
-rwxr-xr-x   1 meeee ya   576035 Oct 5 18:15 /home/meeee/root/bin/rootd*
> ls $ROOTSYS/bin/proofd
-rwxr-xr-x   1 meeee ya   556488 Oct 5 18:16 /home/meeee/root/bin/proofd*

Copy the root installation to /usr/local/root or /opt/root. On a cluster, each node should have a root installation in either of these directories.

Source the login script or start a new terminal and try to start root to make sure it works properly.


2. PROOF Preliminaries



Make sure that your machine has a central installation of ROOT in /usr/local/root or /opt/root.

Add the PROOF server and the ROOT file server to /etc/services:



If you are using xinetd (the extended Internet services daemon), add the following lines to /etc/xinetd.conf (assuming an existing root installation in /usr/local/root):



As superuser, kill the current xinetd daemon and restart it with the new configuration:

  > killall xinetd
  > xinetd -stayalive -pidfile /var/run/xinetd.pid

Optional Settings
For PROOF syslog messages, add the following line to /etc/syslog.conf on the master server (assuming a root installation in /usr/local/root):



In case the /usr/local/root/proof/log directory does not exist, create it:

  > mkdir /usr/local/root/proof/log

Also create an empty log file:

  > touch /usr/local/root/proof/log/proof.log

In the /etc/syslog.conf file, add local5 and local6 to the already existing /var/log/messages line (note that your messages line might contain more than what is shown below):



Now we are ready to stop and restart the system logging with the new configuration:

  > killall syslogd
  > syslogd -m 0 (e.g. with switched off timestamp logging)


3. PROOF Configuration



We can now configure the PROOF master and workers starting from an example file ($ROOTSYS/proof/etc/proof.conf.sample):



An example file for a single "test" machine (with an SMP core/hyperthreaded or dual processor) with one master and two slaves/workers looks like:



Copy your config file to $ROOTSYS/proof/etc/proof.conf.

For setting up a cluster, please consult the $ROOTSYS/README/README.PROOF document.


4. First PROOF Session



Start up ROOT, connect to your PROOF master and print some information:

  > gROOT->Proof("mypc.cern.ch")



  > gProof->Print()



(Using option "a" in the Print() method will give more information about the workers).

Commands can be executed with the Exec() method. E.g. if you want to see the current directories [on the workers]:

  > gProof->Exec(".!pwd")



The workers can be switched on and off with the SetParallel(n) method:

  > gProof->SetParallel(0)
  > gProof->SetParallel(2) (in the case of two workers)

Try to switch off the workers and check which is the current directory!


5. FAQ - Frequently Asked Questions



1. Authenticity can't be established.
2. Failed to setup connection with PROOF master server.

1. Authenticity can't be established.
"I'm using my normal AFS password to login to the PROOF server, but it fails to authenticate me!"



The main problem is that SSH does not propagate the password. Solutions to this problem can be found in the roottalk list. The simplest solution is to create a ROOT password file in your $HOME directory on each machine (including the master). The file must be called .rootdpass and contain the crypt hash of a password you have chosen. You can use the crypt script under proof/utils:

  > $ROOTSYS/proof/utils/crypt <secret_passwd> > $HOME/.rootdpass
  > chmod 0600 $HOME/.rootdpass

You will be prompted to enter the "secret_passwd" when you execute gROOT->Proof(...) and it should be propagated correctly to all the stages.


2. Failed to setup connection with PROOF master server.
"It worked yesterday, but today it fails to setup a connection with the PROOF master server!"



Reason: ?
(Temporary) Solution: kill the xinetd process (as root user), restart it and try again:

  > killall xinetd
  > xinetd -stayalive -pidfile /var/run/xinetd.pid
  > ...



Paul NILSSON
Last modified: Tue Oct 11 14:03:51 CEST 2005