• UsingHarvestersOutsideKbandSB

Deploying NetarchiveSuite w/ application placed outside the KB and SB network

With the DeployApplication it is straight-forward to place the harvesters on a different physical location than on Kb and SB (location OUTSIDE). Just define location OUTSIDE as a physicalLocation in the deploy configuration file.

Note that the harvester-machines on location OUTSIDE need to have

• write-access to a FTP-server, that can be seen from the bitarchiv servers on both SB and KB.
• access to the JMSbroker on the Administrative machine in the netarkivet prod-environment.

Note that FTP or SFTP is required as the transport method, as it is not possible to push data with the current network grouping of the Bitarchive-machines at KB, and therefore can't use the PUSH-transport methods (http or https).

Deploying NetarchiveSuite on location OUTSIDE can be done as we do today when deploying on KB and SB:

• Run the deployApplication
• Generate installation-packages for the three locations KB, SB, and OUTSIDE.
• Copy the installation-packages to the respective locations

• CLO coordinates the installation and starting of NetarchiveSuite on the three locations

Template for configuring an OUTSIDE location in the deploy config file

<deployGlobal>
    <deployClassPath>lib/dk.netarkivet.harvester.jar</deployClassPath>
    <deployClassPath>lib/dk.netarkivet.archive.jar</deployClassPath>
    <deployClassPath>lib/dk.netarkivet.viewerproxy.jar</deployClassPath>
    <deployClassPath>lib/dk.netarkivet.monitor.jar</deployClassPath>
    <deployJavaOpt>-Xmx1536m</deployJavaOpt>
    <settings> 
    <!-- here default settings are written common for the whole NetarchiveSuite installation -->
    </settings>
<thisPhysicalLocation name="K">
..
</thisPhysicalLocation>
<thisPhysicalLocation name="S">
..
</thisPhysicalLocation>
     
<thisPhysicalLocation name="OUTSIDE">
        <!-- installation directories only used by deploy -->
        <deployInstallDir>/home/netarkiv</deployInstallDir>
        <deployMachineUserName>netarkiv</deployMachineUserName>

        <settings>
            <common>
                <remoteFile>
                    <serverName>FTP-SERVER-ACCESSABLE-FROM-OUTSIDE-LOCATION</serverName> 
                    <userName>...</userName> 
                    <userPassword>...</userPassword>
                </remoteFile>
                <mail>
                    <server>OUTSIDE LOCATION MAILSERVER</server> 
                </mail>
                <notifications>
                    <class>dk.netarkivet.common.utils.EMailNotifications</class>
                    <sender>prod@ADMIN-MACHINE</sender>
                    <receiver>clo@netarkivet.dk</receiver>
                </notifications>
                <useReplicaId>KB</useReplicaId> 
            </common>
        </settings>
 
        <deployMachine name="OUTSIDE-HARVESTING-MACHINE-NAME">
             <deployClassPath>lib/dk.netarkivet.harvester.jar</deployClassPath>
             <deployClassPath>lib/dk.netarkivet.archive.jar</deployClassPath>
             <deployClassPath>lib/dk.netarkivet.viewerproxy.jar</deployClassPath>
             <deployClassPath>lib/dk.netarkivet.monitor.jar</deployClassPath>
 
             <applicationName name="dk.netarkivet.harvester.harvesting.HarvestControllerApplication">
                 <settings>
                     <common>
                        <applicationInstanceId>high_8081</applicationInstanceId>            
                         <jmx>
                             <port>8190</port>
                             <rmiPort>8290</rmiPort>
                         </jmx>
                     </common>
                     <harvester>
                         <harvesting>
                             <queuePriority>HIGHPRIORITY</queuePriority>
                             <heritrix>
                                 <guiPort>8380</guiPort> 
                                 <jmxPort>8480</jmxPort>
                                 <jmxUsername>controlRole</jmxUsername>
                                 <jmxPassword>...</jmxPassword>
                             </heritrix>
                            <serverDir>harvester_high_8081</serverDir>
                         </harvesting>
                     </harvester>
                 </settings>
             </applicationName>
             <applicationName name="dk.netarkivet.harvester.harvesting.HarvestControllerApplication">
                 <settings>
                     <common>
                        <applicationInstanceId>high_8082</applicationInstanceId>            
                         <jmx>
                             <port>8191</port>
                             <rmiPort>8291</rmiPort>
                         </jmx>
                     </common>
                     <harvester>
                         <harvesting>
                             <queuePriority>HIGHPRIORITY</queuePriority>
                             <heritrix>
                                 <guiPort>8381</guiPort> 
                                 <jmxPort>8481</jmxPort>
                                 <jmxUsername>controlRole</jmxUsername>
                                 <jmxPassword>...</jmxPassword>
                             </heritrix>
                            <serverDir>harvester_high_8082</serverDir>
                         </harvesting>
                     </harvester>
                 </settings>
             </applicationName>
             <applicationName name="dk.netarkivet.harvester.harvesting.HarvestControllerApplication">
                 <settings>
                     <common>
                        <applicationInstanceId>high_8083</applicationInstanceId>
                         <jmx>
                             <port>8192</port>
                             <rmiPort>8292</rmiPort>
                         </jmx>
                     </common>
                     <harvester>
                         <harvesting>
                             <queuePriority>HIGHPRIORITY</queuePriority>
                             <heritrix>
                                 <guiPort>8382</guiPort> 
                                 <jmxPort>8482</jmxPort>
                                 <jmxUsername>controlRole</jmxUsername>
                                 <jmxPassword>...</jmxPassword>
                             </heritrix>
                            <serverDir>harvester_high_8083</serverDir>
                         </harvesting>
                     </harvester>
                 </settings>
             </applicationName>
        </deployMachine>
    </thisPhysicalLocation>