NOTE: This description is in the early stage of being written

Assignment - Rewrite deploy based on settings overwrite

Action(print,Printer friendly version) TableOfContents

References

Reference documents

• ["Glossary"]

Dependencies

• All the tasks below need to be done in the order they are specified.
• all of the tasks is required to be done in the same iteration, although they can be implemented and tested in parallel with other developments.

Terminology

• ...

Bugs

...

Feature Requests

[#1520] Deploy of more than one Bitapplication per server

Basic idea behind new deploy

History of deploy

In the previous NetarchiveSuite there have already existed a deploy module. However this module had the following inconviniences:

• It was tightly connected to the setup of the Danish intallation of NetarchiveSuite

• It was based on a configuration file consisting of XML where the connection between configuration settings and NetarchiveSuite settings were far from obvious.

• It was based on the assumption that all settings had to be specified in a settings file for NetarchiveSuite

Now since the assumption of specified in a settings file for NetarchiveSuite is no longer valid becuse of resent changes. The NetarchiveSuite has now been changed to have in-build default settings where only local overwrites of defaults has to be set for the individual applications.

Furthermore, resent analysis and assignment description (assignment B2) for the archive has revealed the most obvious connections to the setup of the Danish intallation of NetarchiveSuite. This is mainly based on the interpretation of Location, which have both meant "location of bitarchive to be used" and "physical location of this instance of this application". As part of the assignment B2 as well as part of this assignment is a change of naming of the old location settings into distinc terms of physical location and replica. This change will mean that the new deploy no longer needs to be connected to the Danish intallation of NetarchiveSuite.

Lastly, the definitions in the new configuration file will in the new deploy be named precisely the same way as settings for NetarchiveSuite in cases where there are a direct connection. Only few definitions will be for deploy only, and in these cases the definitions will follow the naming convention of being prefixed with "deploy_".

Override settings structure

The idea is that the new deploy will be based on the default settings in NetarchiveSuite. A configuration file is then used for specifications of where the different applications are placed and which settings overrides is needed for the applications.

The default settings can be overwritten at different levels in the configuration settings file. this is illustrated in the below figure:

attachment:layers.gif

Documentation

In existing documentation

Deployment Manual with reference to existing sections in Installation Manual

• Preparation
  • make it-config file
  • get netarchivesuite zip file
  • get and adjust configuration files
• deploy
  • how to
  • what is happening
• installation
  • how to (including modifications)
  • what is happening (incl. makedir)
• start and kill

Settings documentation should only consist in references to xml files in repository. Explation of settings overwrite structure and special deploy configuration settings (levels, install dir etc.)

New deploy documentation

The new deploy works in three steps:

1. prepare folders for deployment
2. Install folders on machines for all physical location
3. Start all installed applications

Preparation of folders for deployment is illustrated in the below figure

(consider whether NetarchiveSuite zip file should be given as parameter here and be placed in install-dir) attachment:deploy_step1.gif

Installation of folders on machines for all physical location is illustrated in the below figure

(consider whether NetarchiveSuite zip file should be as parameter in previous step, and then be taken from install dir here)

attachment:deploy_step2.gif

Start all installed applications is illustrated in the below figure attachment:deploy_step3.gif

Changes in NetarchiveSuite apart from deploy

System state GUI

The current columns "Organisation" and "Port" must be replaced by

• physicalLocation for "Location" (for all)

• replica for "Replica" (for BitArchiveMonitors and BitArchives only)

• http.port for "HTTP" (for GUIApplication and ViewerProxyApplication only)

• harvesting.queuePriority (for HarvestControllerApplication only)

Final look must be agreed with users. An idea could be introduce a hiding mechanism at the same level and in the same way as the "Show all" functionality

In order to pass data to System state GUI, the data must be made available via the SingleMbeanObject.

Split of location concept

Today "location" covers both physical location and bitarchive replica. This must change in order to make deploy more general.

The following settings are changed:

• locations -> replicas

• location ->

  • replicaId (added replicaType & replicaName)

  • deployApplicationInstanceId (for BitarchiveMonitorApplications)

• locations.batchLocation -> useReplicaId

• thisLocation ->

  • archive.thisReplicaId for bitarchive & monitor (Channels)

  • thisPhysicalLocation for others (SingleMbeanObject)

  • useReplicaId for getFile/get in archive.tool, viewerproxy & arcrepository?! Used for getFile/get tool

Channel name definitions

Must rely on thisReplica and applicationInstanceId instead of thisLocation and http.port

The following setting is new:

• applicationInstanceId (replacing use of port in channel names)

  • HarvestController

  • ViewerProxyApplication(may not be necessary if use of IP-address is enough and only 1 on each machine)

  • BitarchiveMonitorApplication (may not be necessary if we accept deploys naming instances from thisReplicaId)

Differentiate applications on instance id

Differentiate applications on instance id istead of http.port number or thisLocation. ...

The following setting is new:

• applicationInstanceId (replacing use of port in channel names)

  • HarvestController

  • ViewerProxyApplication(may not be necessary if use of IP-address is enough and only 1 on each machine)

  • BitarchiveMonitorApplication (may not be necessary if we accept deploys naming instances from thisReplicaId)

New definition of IT-config

explation of settings overwrite structure and special deploy configuration settings (levels, install dir etc.)

New special deploy settings

• <deployApplicationInstanceId>

• Instance identification (e.g. suffix for application scripts)

• <deployClassPath>

• Class paths to be added for an application (can have several)

• <deployGlobal>

• Global 1. level scope - overwrites setting defaults.

• <deployInstallDir>

• Installation directory for a deployMachine

• <deployJavaOpt>

• Java options for an application (can have several)

• <deployMachine>

• Machine 3. level scope - overwrites 1. and 2. level settings.

• <deployMachineUserName>

• User name for a deployMachine

Settings indirectly set

Named tags that result in implicit setting og settings:

• <thisPhysicalLocation name="PL"> in <deployGlobal> scope

• sets settings.common.thisPhysicalLocation to PL

• <applicationName name="AN"> in <physicalLocation> scope

• sets settings.common.applicationName to AN
• NOTE: that the applicationName is NOT USED today, it is automatically set at startup. However this is also one of the reasons that we cannot reload new setting for an application. Thus this new deploy will also help solving the reuoload issue at a later stage.

Rewrite deploy

Note that the newest version of the NetarchiveSuite code has eliminated use of the SideKick application. Therefore the special handling in starting and stopping this process is not necessary anymore.

Reuse deploy code from kb-doms (Royal Library Digital Object Managemnt System code):

• machine definition class (for different os)
• hirarchy modules at machine and application level

Reuse (and expand) SimpleXmlTree from the current NetarchiveSuite software.

parameter changes compares to old deploy:

• jmxremote.password must be generated from scratch instead of reading a file
• security.policy file must be given explicit as parameter
• log.prop file must be given explicit as parameter

• NetarchiveSuite-package (zip file) must be given explicit as a new parameter

• Settings file is no longer needed as parameter
• Environment is no longer needed as parameter, it is readen from setting environmentName

Use following design:

• check input
• read and set settings from global level
• make physical location objects with from global level in it-config overwritten by specific physical location settings
• for each physical location
  • read and set settings from physical location level
  • make machine objects specified for the location correspong to the mashine OS
  • for each machine
    • read and set settings from machine level
    • make application objects specified for the machine
    • for each application
      • read and set settings from applciation level
• use methods on objects in hierarchy to produce various scripts

When deployed is rewritten, the scripts for making multi-user test platform must also be updated

Order of implementation

• Location impl. + test could with big advantage be implemented first
• New deploy could be started as parallel activity, but will depend on
  • changes in location settings
  • (to some extend) introduction os applicationInstanceId setting