Xml-qstat-1.0

From GridWiki
Jump to: navigation, search

xml-qstat 'pure java' documentation and installation notes

This is a temporary wiki page for alpha and beta testers of the new xml-qstat code from http://xml-qstat.org

We now have a pure-Java version and need to make changes to our installation directions, documentation and packaging methods to reflect the new, improved (and far simpler!) codebase.

I'm going to document the rough steps involved in getting the "new" xml-qstat to work, hopefully others can expand and point out problems and inaccuracies here. This wiki page will be used to create the final installation instructions that will ship with the package.

Prerequisites

  • Familiarity with the "old" directions at http://xml-qstat.org/install.html so you can see what is different
  • You need your JAVA_HOME environment variable set in order for Cocoon to start
  • The Grid Engine "settings.sh|csh" file must have been sourced, SGE binaries must be in your path. Cocoon will inherit your path and currently the sitemap.xml file will call 'qstat' without a full path defined. You must be able to successfully run the 'qstat -f' command yourself from the command line if there is going to be any hope that Cocoon will be able to do so!

Tested Platforms

Tested and found to work on the following platforms:

  • Pre-release seed of Apple OS X "Leopard" Server with Java 1.5.0_13, Cocoon 2.1.10 and SGE 6.1u2
  • Centos 5 Linux on x86 with Java 1.5.0_13, Cocoon 2.1.10 and SGE 6.1u2
  • Centos 5 Linux on x86 with Java 1.5.0_13, Cocoon 2.1.10 and SGE 6.0u11
  • Centos 5 Linux on x86 with Java 1.5.0_13, Cocoon 2.1.10 and SGE 6.0u10

Step by step

Step 1 - Download the latest xml-qstat code:

svn checkout svn://xml-qstat.org/xml-qstat/trunk

Step 2 - Rename the 'trunk' directory to something more sensible

zareh:tmp dag$ mv trunk xmlqstat
zareh:tmp dag$ ls -l xmlqstat/
total 24
-rw-r--r--  1 dag  wheel   569 Oct 23 17:15 Changelog
-rw-r--r--  1 dag  wheel  3295 Oct 23 17:15 build.xml
drwxr-xr-x  4 dag  wheel   136 Oct 23 17:15 java
-rw-r--r--  1 dag  wheel    82 Oct 23 17:15 manifest.mf
drwxr-xr-x  3 dag  wheel   238 Oct 23 17:15 nbproject
drwxr-xr-x  4 dag  wheel   306 Oct 23 17:15 scripts
drwxr-xr-x  8 dag  wheel   306 Oct 23 17:15 xmlqstat

Step 3 - Download the latest Cocoon application server, install it into the xmlqstat folder (ideally)

cd xmlqstat/
wget http://www.gtlib.gatech.edu/pub/apache/cocoon/cocoon-2.1.10-src.tar.gz
gzcat cocoon-2.1.10-src.tar.gz | tar xvf -

zareh:xmlqstat dag$ ls -l
total 24
-rw-r--r--  1 dag  wheel   569 Oct 23 17:15 Changelog
-rw-r--r--  1 dag  wheel  3295 Oct 23 17:15 build.xml
drwxr-xr-x  7 dag  admin   952 Oct 23 16:12 cocoon-2.1.10
drwxr-xr-x  4 dag  wheel   136 Oct 23 17:15 java
-rw-r--r--  1 dag  wheel    82 Oct 23 17:15 manifest.mf
drwxr-xr-x  3 dag  wheel   238 Oct 23 17:15 nbproject
drwxr-xr-x  4 dag  wheel   306 Oct 23 17:15 scripts
drwxr-xr-x  8 dag  wheel   306 Oct 23 17:15 xmlqstat

Step 4 - Build the Cocoon web application server

cd cocoon-2.1.10
./build.sh

Step 5 - Create the Cocoon mount-table.xml file The file should be located at cocoon-2.1.10/mount-table.xml and assuming that cocoon is in the same directory as the xmlqstat files then the mount-table.xml file would have these contents:

<mount-table>
 
  <!-- Map the browser URI "/xmlqstat" to our webapp folder -->
  <mount uri-prefix="xmlqstat/" src="../../../xmlqstat/"/>
    
</mount-table>


Step 5 - Obtain the xml-qstat.jar file The checked out files from the SVN trunk/ contain the raw source code for the jar file along with the necessary Netbeans project files needed to successfully build a local copy of the jar yourself. It is unlikely that we would make end-users do this so we should be testing with a pre-built jar file to see how it works out. Note: See bottom of this page for instructions on how to build the jar locally ...

wget http://gridengine.info/files/xml-qstat.jar

Step 6 - Install the xml-qstat.jar file into the Cocoon web application server Note that the "build" directory will not exist until you have successfully built the app server (Step 4)

cp xml-qstat.jar ./cocoon-2.1.10/build/webapp/WEB-INF/lib/

Step 7 - Start Cocoon

zareh:cocoon-2.1.10 dag$ ./cocoon.sh 
./cocoon.sh: executing default action 'servlet', use -h to see other actions
./cocoon.sh: using ./build/webapp as the webapp directory
-------------------- Loading --------------------
Processing repository: ./tools/jetty/lib
Adding jar: ./tools/jetty/lib/jetty-4.2.23.jar
Adding jar: ./tools/jetty/lib/servlet-2.3.jar
Processing repository: ./lib/endorsed
Adding jar: ./lib/endorsed/jakarta-bcel-20040329.jar
Adding jar: ./lib/endorsed/jakarta-regexp-1.4.jar
Adding jar: ./lib/endorsed/xalan-2.7.0.jar
Adding jar: ./lib/endorsed/xercesImpl-2.9.0.jar
Adding jar: ./lib/endorsed/xml-apis-1.3.04.jar
-------------------- Executing -----------------
Main Class: org.mortbay.jetty.Server
16:14:28.042 EVENT  Checking Resource aliases
16:14:28.390 EVENT  Starting Jetty/4.2.23
16:14:30.139 EVENT  Started WebApplicationContext[/,./build/webapp]
16:14:34.650 WARN!! Alias request of 'file:/opt/trunk/dist/xml-qstat.jar' for 'file:/opt/trunk/cocoon-2.1.10/build/webapp/WEB-INF/lib/xml-qstat.jar'
16:14:34.651 WARN!! Alias request of 'file:/opt/trunk/dist/xml-qstat.jar' for 'file:/opt/trunk/cocoon-2.1.10/build/webapp/WEB-INF/lib/xml-qstat.jar'
trying to register database
trying to register database
WARN! *** JMS block is installed but jms client library not found. ***
- For the jms block to work you must install and start a JMS server and place the client jar in WEB-INF/lib.
- The default server, OpenJMS is configured in cocoon.xconf but is not bundled with Cocoon.
16:14:49.364 EVENT  Started SocketListener on 0.0.0.0:8888
16:14:49.365 EVENT  Started org.mortbay.jetty.Server@b0bdc8


At this point the embedded webserver within Apache Cocoon will be operating on port 8888, if you want to hide this behind a real apache webserver, surf on over to the "old" installation directions located at http://xml-qstat.org/install.html and read the section on "Cocoon behind Apache Proxies".


Step 8 - Test

At this point you should be able to test by pointing your browser at the following url:

http://<your server>:8888/xmlqstat/qstat.html



Building the xml-qstat.jar archive locally

The following shell script allowed me to build my own jar file without having to use the complex Netbeans IDE environment.

The script is now present in the SVN trunk/ so it should be already available on your system.

#!/bin/sh

# Chris Dagdigian (dag@sonsorol.org)
#
# This script can be used to compile and package the xml-qstat.jar file required
# by Apache Cocoon for direct XML queries to the SGE Qmaster. 
#
# Instructions:
#
#  1. Run this script, fix as needed until it runs properly to completion
#
#  2. Copy (or symlink) the resulting xml-qstat.jar file to your Cocoon
#     build/webapp/WEB-INF/lib/ directory (before starting cocoon)
#


## YOU WILL NEED TO EDIT THIS BLOCK TO MATCH YOUR LOCAL INSTALL
MyCocoonBase="/opt/trunk/cocoon-2.1.10/build"

MYCLASSPATH="$MyCocoonBase/cocoon/cocoon.jar:\
$MyCocoonBase/webapp/WEB-INF/lib/commons-lang-2.2.jar:\
$MyCocoonBase/webapp/WEB-INF/lib/avalon-framework-api-4.3.jar:\
$MyCocoonBase/webapp/WEB-INF/lib/avalon-framework-impl-4.3.jar:\
$MyCocoonBase/webapp/WEB-INF/lib/xml-apis-1.3.04.jar:\
$MyCocoonBase/webapp/WEB-INF/lib/excalibur-pool-api-2.1.jar:\
$MyCocoonBase/webapp/WEB-INF/lib/excalibur-datasource-2.1.jar:\
$MyCocoonBase/webapp/WEB-INF/lib/excalibur-xmlutil-2.1.jar:\
$MyCocoonBase/webapp/WEB-INF/lib/excalibur-sourceresolve-2.1.jar\
"

## Cocoon 2.1.11 note
## When trying this script under cocoon 2.1.11 it failed because the filenames
## of the .jar files mentioned above were a bit out of date. I had to make these
## changes:
##  -- commons-lang-2.2.jar became commons-lang.2.3.jar
##  -- excalibur-sourceresolve-2.1.jar became excalibur-sourceresolve-2.2.3.jar
##
## If you have trouble running this script, carefully compare the .jar files listed
## above against what is actually in the cocoon build/webapp/WEB-INF/lib/ directory




## -- should not need to make changes below but if so the code should be pretty clear -- ##

echo "[STATUS] Entering the xmlqstat generator directory"
cd java/org/xmlqstat/generator/

echo "[STATUS] Will try to run this command:"
echo "javac -classpath $MYCLASSPATH CommandGenerator.java "

if  javac -classpath $MYCLASSPATH CommandGenerator.java; then
   echo "\n[STATUS] The javac command seems to have worked. Continuing ..."
 else
   echo "\n[STATUS] ** The javac command failed. No point in continuing..."
   exit 1
fi

echo "\n[STATUS] Java compiled into .class files; now we need to make a clean .jar archive ..."

cd ../../../../

## We need to make a copy of our java/org/xmlqstat/generator/ tree that is free
## from ".svn/*" files and also does not contain the .java file. From that 
## clean directory we can make a nice jar archive

if [ -d jarbuilder ]; then
   echo "\n[STATUS] Moving existing jarbuilder folder to jarbuilder.old"
   mv -f jarbuilder jarbuilder.old
fi

echo "\n[STATUS] Making a clean jarbuilder directory"
mkdir jarbuilder
cd ./jarbuilder/

echo "\n[STATUS] Making the org/xmlqstat/generator directory structure"
mkdir -p org/xmlqstat/generator

echo "\n[STATUS] copying compiled .class files into new directory structure"
cp -v ../java/org/xmlqstat/generator/*.class ./org/xmlqstat/generator/


echo "\n[STATUS] Making our .jar file"
jar cvf xml-qstat.jar -C . .

echo "\n[STATUS] Done! Look for a 'xml-qstat.jar' file in the jarbuilder/ folder."