Easy Installation of NetarchiveSuite

Action(edit)

• Verify that you have all the needed software installed by installing the QuickStart according to https://netarchive.dk/suite/Quick_Start_Manual e.g. in /home/test/netarchive by starting the Quickstart

• Shutdown the QuickStart according to the QuickStart Manual

• Download following attached files to e.g. /home/test/netarchive:

attachment:RunNetarchiveSuite.sh

attachment:deploy_example_one_machine.xml

The first script is a simple script for doing all the steps during deployment. It takes a NetarchiveSuite package ('.zip'), a configuration file (the second file), and a temporary installation directory as arguments (in the given order).

In the configuration file all the applications are placed on one machine (e.g. the current machine, localhost). This gives the same kind of instance as the QuickStart, though this is configured and installed through the Deploy module. If run directly it is installed and run from the directory /home/test/USER.

Below, you find other deploy examples. ( They have to be modfied to your environment)

E.g.

#stop the Quickstart
cd /home/test/netarchive/scripts/simple_harvest/killhard.sh
cd /home/test/netarchive
bash RunNetarchiveSuite.sh NetarchiveSuite.zip deploy_example_one_machine.xml USER/
#if you have not setup your ssh keygen correctly, you need to login some times before the installation finish successfully

The script creates a "USER" folder in e.g. /home/test , which contains e.g. methods for starting and stopping NetarchiveSuite and starts the whole NetarchiveSuite.

• Set your browser to proxy according to the QuickStart Manual - now on port 8070

• Choose the URL e.g. http://dia-test-int-01.kb.dk:8074/HarvestDefinition/

• You can now create, run and browse according to the QuickStart - or User Manual

Examples of deploy configuration files

In the following are two examples of configuration files for deploy. The first two requires adaptation to your own system before use.

attachment:deploy_example.xml

The instance with two replicas divided over two physical locations. Each physical locations contain several machines. Bitarchive machines, harvester machine and viewerproxy machine. Only one physical location has an administator machine, which contains the GUI application, the Bitarchive monitors and the arc repository.

attachment:deploy_example_single.xml

This is the instance with only one replica and one physical location. It is very close to the first example, just with one replica removed.

A running HW/SW setup example from June 2009 for Netarkivet.dk

http://netarchive.dk/suite/Installation_Manual_3.10?action=AttachFile&do=view&target=HW_SW_production_example.txt

How to add a harvester more on the same machine and set all to HIGHPRIORITY selective harvesting

Using e.g. deploy_example.xml

• Duplicate the existing harvester <applicationName> definition within <deployMachine>

In the new duplicate harvester config, change all following duplicate values to new unique values within <deployMachine>:

• <applicationInstanceId>

• <common><jmx><port> and <rmiPort>

• <heritrix><guiport> and <jmxPort>

• <serverDir>harvester_high_2</serverDir>

and set

• <queuePriority>HIGHPRIORITY</queuePriority>

e.g.:

• <applicationName name="dk.netarkivet.harvester.harvesting.HarvestControllerApplication">

  • <settings>

    • <common>

      • <applicationInstanceId>high2</applicationInstanceId> <jmx>

        • <port>8112</port> <rmiPort>8212</rmiPort>

        </jmx>

      </common> <harvester>

      • <harvesting>

        • <queuePriority>HIGHPRIORITY</queuePriority> <heritrix>

          • <guiPort>8192</guiPort> <!-- T: jmxPort to be modified by test (was 8093) --> <jmxPort>8193</jmxPort>

            • <jmxUsername>controlRole</jmxUsername> <jmxPassword>R_D</jmxPassword>

          </heritrix> <serverDir>harvester_high_2</serverDir>

        </harvesting>

      </harvester>

    </settings>

  </applicationName>