Diff for "TEST12" - NetarchiveSuite

Differences between revisions 41 and 42

Release test Wayback

This test tests the setup of a Wayback installation, automatic indexing of new harvests, and the cooperation between Wayback and the NetarchiveSuite archive, via the NetarchiveResourceStore. It doesn't test Wayback's functionality, which are tested by archive.org.

This test is configured for use in the test environment in Denmark only. Note that there is only a single server running wayback so only one tester should test wayback functionality at any given time.

Items	Status 1	Status 2	Status 3	Notes	Known open bugs	Bugs tested	New bugs found	Previous bugs
[:It43StartSuite :1. Start Suite ]
[:It43Check the Indexer is Running :2. Check Indexer ]
[:It43Start a Small Harvest :3. Start Harvest ]
[:It43StartWayback :4. Start Wayback ]
[:It43Configure Proxy :5. Configure Proxy ]
[:It43Wait for Indexing :6. Wait for Indexing ]
[:It43Sort Index :7. Sort Index ]
[:It43BrowseInHarvest :8. Browse In Harvest ]
[:It43ShutdownSuite :1. Shutdown Suite ]

-  ⇤ ← Revision 41 as of 2010-04-26 12:54:01 → 
  Size: 5812
  Editor: TueLarsen
  Comment:
+   ← Revision 42 as of 2010-06-04 08:18:09 → ⇥
  Size: 1360
  Editor: ColinRosenthal
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 2:
-This test tests the setup of a Wayback installation and the cooperation between Wayback and the NetarchiveSuite archive, via the NetarchiveResourceStore. It doesn't test Wayback's functionality, which are tested by archive.org.
+This test tests the setup of a Wayback installation, automatic indexing of new harvests, and the cooperation between Wayback and the NetarchiveSuite archive, via the NetarchiveResourceStore. It doesn't test Wayback's functionality, which are tested by archive.org.
 Line 4:
-''Paths and name are chosen to make this test as easy as possible, feel free to change these to suit your system. ''
+This test is configured for use in the test environment in Denmark only. Note that there is only a single server running wayback so only one tester should test
wayback functionality at any given time.
-Line 6:
+Line 7:
-''The following commands (lines starting with '$' can be run directly from a terminal. *Remember to change the text in bold.''
 Line 8:
-=== 1. Generate a sample archive ===
Generate a sample archive using the [Quick_Start_Manual: the quick start manual] with ARC-files by making a selective harvest of the netarkivet.dk domain. Use the "Once_an_hour" schedule, and let it run 3 or 4 times.

Your ARC/WARC files are located here '''/home/test/QUICKSTART/bitarkiv/filedir/'''.

'''Remember''' to make these directories readable/writable for the user conducting the test.

=== 2. Download NetarchiveSuite from trunk (after code freeze) and make wayback war file: ===
''$ svn checkout --username '''developername''' https://gforge.statsbiblioteket.dk/svn/netarchivesuite/trunk/ ~/netarchivesuite_trunk/'' (Note: currently does not work using svn 1.6.6, but does work with svn 1.4.4)

{{{
cd ~/netarchivesuite_trunk/
wget http://downloads.sourceforge.net/project/archive-access/wayback/1.4.2/wayback-1.4.2.tar.gz?use_mirror=dfn -O wayback-1.4.2.tar.gz
tar xzf wayback-1.4.2.tar.gz
cd conf;
ln -s ../examples/wayback wayback
cd ..
ant jarfiles
ant -f wayback.build.xml warfile
}}}
Check that there are no errors.

=== 3. Download Apache Tomcat http://tomcat.apache.org/, binary version 6.XX is recommended. And prepare wayback webapp. ===
Download and unpack Tomcat to ~/tomcat/

''$ mv ~/netarchivesuite_trunk/ROOT.war ~/tomcat/apache-tomcat-6.0.26/webapps/ ; export TOMCAT_DIR=~/tomcat ;''

Make sure Tomcat loads ''~/netarchivesuite_trunk/conf/wayback/settings.xml''. This settings.xml file is copied to the WEB-INF directory in the ROOT.war generated above. This can be done my add the following line to ~/tomcat/apache-tomcat-6.0.26/bin/catalina.sh

CATALINA_OPTS='-Ddk.netarkivet.settings.file=$TOMCAT_DIR/apache-tomcat-6.0.26/webapps/ROOT/WEB-INF/settings.xml'

''$ cd ~/tomcat/apache-tomcat-6.0.26/bin/''

Edit catalina.sh and add the above line just below '''#!/bin/bash''' or '''#!/bin/sh'''

=== 4. Make CDX files and setup standalone archive. ===
{{{
cd ~/netarchivesuite_trunk
export ARCS=`ls /home/test/QUICKSTART/bitarkiv/filedir | grep -v metadata`
for arc in $ARCS
do
echo Adding index of $arc to file unsorted.cdx
./wayback/bin/arc-indexer /home/test/QUICKSTART/bitarkiv/filedir$arc >> unsorted.cdx
done
}}}
''$ mkdir /home/test/wayback_cdx/''

'''Remember''' to make /wayback/ readable/writable for the user conducting the test.

Sort cdx files by using the ''sort'' command in linux, and move the sorted file to /home/test/wayback_cdx/file.sorted.cdx.

{{{
export LANG=C
sort unsorted.cdx > /home/test/wayback_cdx/file.sorted.cdx
}}}
Verify, that your JMS broker is still running by using following command 'ps -ef | grep mq'

Check that there are no errors in jms error log

''$ mkdir ~/standalone''

''$ cd ~/netarchivesuite_trunk''

''$ ant releasezipball''

Check that there are no errors

''$ cp ~/netarchivesuite_trunk/NetarchiveSuite.zip ~/standalone''

''$ cp ~/netarchivesuite_trunk/conf/wayback/standalone_archive.xml ~/standalone/''

Change relevant settings in '''standalone/standalone_archive.xml''' like ''deployGlobal.thisPhysicalLocation.deployInstallDir'' and ''deployGlobal.thisPhysicalLocation.deployMachineUserName''.

Downlad ''RunNetarchiveSuite.sh'' from http://netarchive.dk/suite/Installation_Manual_3.10/AppendixC?action=AttachFile&do=get&target=RunNetarchiveSuite.sh and save it in '''~/standalone/'''.

''$ cd ~/standalone/''

''$ ./RunNetarchiveSuite.sh NetarchiveSuite.zip standalone_archive.xml STANDALONE''

Check that std. output and log files don't report any errors. Log files lies in '''~/standalone/STANDALONE/log/*'''.

=== 5. Start Tomcat and browse the archive ===
''$ cd ~/tomcat/''

''$ chmod goa+x *.sh ''

''$ ./bin/startup.sh''

''Check that there are reported no errors in '''~/tomcat/log/catalina.out'''. ''

''''Goto http://localhost:8080/wayback in a browser and browser some material the the ARC/CDX files. '''''''

'''''''If you get the following error:

{{{
Unable to find a javac compiler;
com.sun.tools.javac.Main is not on the classpath.
Perhaps JAVA_HOME does not point to the JDK
        at org.apache.tools.ant.taskdefs.compilers.CompilerAdapterFactory.getCompiler(CompilerAdapterFactory.java:105)
        at org.apache.tools.ant.taskdefs.Javac.compile(Javac.java:924)
        at org.apache.tools.ant.taskdefs.Javac.execute(Javac.java:757)
        at org.apache.jasper.compiler.Compiler.generateClass(Compiler.java:320)
        at org.apache.jasper.compiler.Compiler.compile(Compiler.java:370)
        at org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:473)
        at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:190)
}}}
add $JAVA_HOME/lib/tools.jar to the classpath explicitly:

{{{
export CLASSPATH=$JAVA_HOME/lib/tools.jar:$CLASSPATH
}}}
and then restart tomcat:

{{{
cd ~/tomcat; ./bin/shutdown.sh ; sleep 5 ; ./bin/startup.sh
}}}
=== 6. Clean up things ===
Stop Tomcat: ''$ cd ~/tomcat/'' ''$ ./bin/shutdown.sh''

Stop archive: ''$ cd ~/standalone/STANDALONE/''

''$ ./killall_NATIONAL_LIBRARY.sh''

''Remove installed application: ''

''$ rm -r ~/tomcat/''

''$ rm -r ~/standalone''

''$ rm -r ~/netarchivesuite_trunk''

''Optional remove ''/arcs'' and ''/wayback''. ''
+||<tablewidth="400px">'''Items ''' ||'''Status 1''' ||'''Status 2''' ||'''Status 3''' ||'''Notes ''' ||'''Known open bugs ''' ||'''Bugs tested ''' ||'''New bugs found ''' ||'''Previous bugs''' ||
||[:It43StartSuite :1. Start Suite ] || || || || || || || || ||
||[:It43Check the Indexer is Running :2. Check Indexer ] || || || || || || || || ||
||[:It43Start a Small Harvest :3. Start Harvest ] || || || || || || || || ||
||[:It43StartWayback :4. Start Wayback ] || || || || || || || || ||
||[:It43Configure Proxy :5. Configure Proxy ] || || || || || || || || ||
||[:It43Wait for Indexing :6. Wait for Indexing ] || || || || || || || || ||
||[:It43Sort Index :7. Sort Index ] || || || || || || || || ||
||[:It43BrowseInHarvest :8. Browse In Harvest ] || || || || || || || || ||
||[:It43ShutdownSuite :1. Shutdown Suite ] || || || || || || || || ||