Configuration Basics - NetarchiveSuite Settings

edit

It is possible to control much of the behaviour of NetarchiveSuite tools and applications using settings. Some settings need to be updated for a distributed system to work, others work best with their default settings.

Below, the basics of settings and default settings are described. For description of how to tailor the configurations to the applications, please refer to the Installation Manual.

Setting basics

All NetarchiveSuite applications are based on the same type of configuration: Keys can be mapped to values, and the mappings can be set either in a settings file written in XML, or on the command line. If no value is specified for a given configuration key, a default value is used.

The keys are defined in a hierarchy. When naming the keys, we separate the levels in a key with dots, for instance:

    settings.common.http.port=8076

When describing the same keys in XML, we use the XML hierarchy:

<settings>
  <common>
    <http>
      <port>8076</port>
    </http>
  </common>
</settings>

Setting keys with multiple values

Some settings allow a list of values, rather than just one value. For instance:

<settings>
  <archive>
    <bitarchive>
      <baseFileDir>/mnt/storage1</baseFileDir>
      <baseFileDir>/mnt/storage2</baseFileDir>
    </bitarchive>
  </archive>
</settings>

It is only possible to specify multiple values using configuration files. This cannot be done on the command line.

If you specify more than one settings file, the first settings file to contain a value for the key specifies all values. Values from the settings files will not be merged.

As an example, consider the following two settings files:

settings1:

<settings>
  <archive>
    <bitarchive>
      <baseFileDir>/mnt/storage1</baseFileDir>
      <baseFileDir>/mnt/storage2</baseFileDir>
    </bitarchive>
  </archive>
</settings>

settings2:

<settings>
  <archive>
    <bitarchive>
      <baseFileDir>/mnt/storage3</baseFileDir>
      <baseFileDir>/mnt/storage4</baseFileDir>
    </bitarchive>
  </archive>
</settings>

The following command will give the value /mnt/storage5:

  java -Ddk.netarkivet.settings.file=settings1.xml:settings2.xml -Dsettings.archive.bitarchive.baseFileDir=/mnt/storage5 dk.netarkivet.common.webinterface.GUIApplication

The following command will give the values /mnt/storage1 and /mnt/storage2:

  java -Ddk.netarkivet.settings.file=settings1.xml:settings2.xml dk.netarkivet.common.webinterface.GUIApplication

The following command will give the values /mnt/storage3 and /mnt/storage4:

  java -Ddk.netarkivet.settings.file=settings2.xml:settings1.xml dk.netarkivet.common.webinterface.GUIApplication

Default Settings

The NetarchiveSuite package includes such XML setting files with default values for the settings that are used to initialize classes if they are not overwritten by separate settings files or on the command line (please refer to Installation Manual).

The NetarchiveSuite has five main levels under the top settings level:

• common
• harvester
• archive
• viewerproxy
• monitor

All settings are defined within these five main levels.

The NetarchiveSuite package includes default values for most defined settings. These are defined in XML setting files that are used to initialize classes, one for each main level and one for each plug-in. (TODO: Name the exceptions)

The meaning of the different settings are documented in the javadoc of the associated setting classes as listed below.

Common part

In the common part of the settings, we have general purpose settings (e.g. settings.common.tmpDir, settings.common.http.port), and settings, that allow us to select plug-ins and their associated arguments (e.g. settings.common.RemoteFile.class, settings.common.jms.broker, settings.common.arcrepositoryClient, and settings.common.indexClient.class).

Most default values for the common part can be found in dk/netarkivet/common/settings.xml and their documentation can be found in the javadoc of the related dk.netarkivet.common.CommonSettings.java class definition.

Futhermore, there are other dedicated(?) common default values for specific plug-in classes defined in the following setting files. All of these are referred to as part of the common part, but are defined with the plug-in itself. Please see section Plug-in Default Settings.

Harvester part

In the harvester part of the settings, we have settings configuring the harvesting process: scheduling, job splitting etc. Most of these settings are used by the scheduler in DefinitionsSiteSection of the GUIApplication

The default values for the harvester part can be found in dk/netarkivet/harvester/settings.xml and their documentation can be found in javadoc of the associated dk.netarkivet.harvester.HarvesterSettings.java class definition.

Archive part

In the archive part of the settings, we have settings related to archive-access (e.g. certain timeouts, replicas and their credentials are defined here). Also behaviour of the BitarchiveApplications is set here.

The default values for the archive part can be found in dk/netarkivet/archive/settings.xml and their documentation can be found in javadoc of the associated dk.netarkivet.archive.ArchiveSettings.java class definition.

Viewerproxy (Access) part

In the viewerproxy part of the settings, we have settings related to the user-access viewerproxy module (e.g. the main directory used for storing the Lucene index for the jobs being viewed)

The default values for the viewerproxy part can be found in dk/netarkivet/viewerproxy/settings.xml and their documentation can be found in javadoc of the associated dk.netarkivet.viewerproxy.ViewerProxySettings.java class definition.

Monitor part

In the monitor part of the settings, we have settings for the monitoring shown in the System State in the form of e.g. JMX user name and password and number of shown logged lines.

The default values for the monitor part can be found in dk/netarkivet/monitor/settings.xml and their documentation can be found in javadoc of the associated dk.netarkivet.monitor.MonitorSettings.java class definition.

Plug-in default settings

At the moment, the following plugins have associated default settings defined in the following classes, where their documentation can be found in the javadoc:

• EMailNotifications.java with defaults in dk.netarkivet.common.utils.EMailNotificationsSettings.xml

• FTPRemoteFile.java with defaults in dk.netarkivet.common.distribute.FTPRemoteFileSettings.xml

• HTTPRemoteFile.java with defaults in dk.netarkivet.common.distribute.HTTPRemoteFileSettings.xml

• HTTPSRemoteFile.java with defaults in dk.netarkivet.common.distribute.HTTPSRemoteFileSettings.xml

• JMSConnectionSunMQ.java with defaults in dk.netarkivet.common.distribute.JMSConnectionSunMQSettings.xml

• JMSArcRepositoryClient.java with defaults indk.netarkivet.archive.arcrepository.distribute.JMSArcRepositoryClientSettings.xml

• IndexRequestClient.java with defaults indk.netarkivet.archive.indexserver.distribute.IndexRequestClientSettings.xml