NetarchiveSuite|
Size: 35545
Comment:
|
Size: 35544
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 53: | Line 53: |
| this bug in the future by using the dk.netarkivet.archive.tools.CreateIndex script ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Fixed || |
this bug in the future by using the dk.netarkivet.archive.tools.CreateIndex script ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Fixed || |
Task list and timetable for iteration 37
Use IterationTaskListTemplate template to start new review pages
Status |
OK/Not Ok |
1. Highlights approved |
OK |
2. Assignment of tasks |
|
3. Task list and time table approved |
|
4. Implementation phase started |
|
5. Release test phase started |
|
6. Assignment phase for next iteration started |
|
7. Iteration 37 completed |
|
Highlights for Iteration
[http://kb-prod-udv-001.kb.dk/twiki/bin/edit/Netarkiv/SupportNetarchiveSuite Support] of released NetarchiveSuite (http://netarchive.dk/suite).
Enhance NetarchiveSuite wiki according to [:UpdateNetarchiveSuiteWiki:decided structure].
Implement bugs and feature request from [https://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/BugPrioFebruar2009 this prioritized list]
Implementation of some of the priority 4 feature request from [:TaskTableFromMay2009Workshop:task list]
- Enhancement of QA
- Enhancement of Batch support
Finalize the support of Wayback in the Netarchive.dk production site. See [:IntegrationOfWaybck:List of tasks] and [:AssignmentWaybackIntegration:Assignment] for Wayback Integration
- Migration of old Web materials to Netarchive.dk
- Start of task according to roadmap
Module Archive
- Enhanced support for Batch
Module Harvester
- ...
Module Access
- Support for Wayback
- Test of Nutchwax
Module Common
- ...
- Bug fixes according to updated prioritized bug list
- Iteration 37 is planned as a development release candidate.
Development procedure
Implementation according to [http://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/TemplateImplementationTask implementation methodology]
Implementation and release test mainly in [http://www.google.com/calendar/render?gsessionid=tjZgbhGt6eNBB1mrlNwt3A intensive period]
- Estimated ... for implementation.
- Estimated ... for release test.
- Estimated ... for assignemnt of tasks for iteration XX+1.
- Target release: ....
Table of tasks
Tasks for iteration 37. Updated 64. June 2009 |
Estimate md |
Main responsible |
Reviewer |
Remaining md at 4. June 2009 |
Comments |
Status |
||||
Implementation phase (task x-n) |
||||||||||
Open Source release + bugs and feature request |
Total x |
- |
- |
Total x |
|
- |
||||
Support of Open Source Release |
||||||||||
1. [http://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/SupportNetarchiveSuite Support] of released NetarchiveSuite |
2 |
All (Google calender) |
|
|
|
Ongoing |
||||
2. Implement translateprocess. Adjustment to Open Source partners. |
1 |
KFC |
ELZI |
|
|
.. |
||||
Bugs and Features requests |
||||||||||
Prioritized bugs according to [https://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/BugPrioFebruar2009 list from February 2009] and [:TaskTableFromMay2009Workshop:list] of priority 4 tasks. |
Total x |
- |
- |
SubTotal x |
|
- |
||||
Module Harvester: [https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1254 Bug 1254] Database connections to MySQL close down intermittently? |
2 |
Nicolas |
|
|
.. |
.. |
||||
||Module Archive: [https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1566 Bug 1566] Several Deduplicating processes started by error - only one should be possible ||<style="TEXT-ALIGN: center">0 ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">We consider this bug fixed, as there now is a method to avoid this bug in the future by using the dk.netarkivet.archive.tools.CreateIndex script ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Fixed ||
Module Common: [https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=555 Bug 555] JMS connections cannot reconnect |
? |
|
|
|
... |
.. |
||||
Module Harvester: [https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1172 Bug 1172] password protected domain was not harvested |
? |
|
|
|
|
.. |
||||
Module Harvester: [https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1336 Bug 1336] Harvester job dies suddenly |
? |
|
|
|
... |
.. |
||||
.. |
|
|
|
|
... |
.. |
||||
Module Common: [https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1218 Bug 1218] Exception while adding listeners to JMSConnection |
? |
|
|
|
... |
.. |
||||
Module Common: [https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1275 Bug 1299] Network I/O errors shuts down JMSConnection |
? |
|
|
|
|
.. |
||||
Module Common: [https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1645 Bug 1645] JMS connections very unstable. |
? |
|
|
|
.. |
.. |
||||
Module Harvester:: [https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1656 Bug 1656] WARNING: Aborting crawl because og inactivity. URLS's in queue:19". |
? |
|
|
|
. |
.. |
||||
Module None:: [https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1670 Bug 1670] Default timeout settings are set way too low in the default settings |
? |
|
|
|
. |
.. |
||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
Prioritized Feature Requests according to [https://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/BugPrioFebruar2009 list from February 2009] and [:TaskTableFromMay2009Workshop:list] of priority 4 tasks |
Total x |
- |
- |
SubTotal x |
|
- |
||||
-: [https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1298 Feature request 1298] Set JMXConnection timeout, if possible |
|
|
|
|
... |
.. |
||||
-: [https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1654 Feature request 1654] . second-level domains for .at in settings.xml |
|
Andreas |
|
|
|
.. |
||||
-: [https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1678 Feature request 1678] Make CDX-entries for the deduplicate entries in the crawl.log, and append to the other CDX-entries. |
|
|
|
|
|
.. |
||||
: [https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1014 Feature request 1014] No good way to mark a non-reported-stopped job as FAILED or DONE. |
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
Roadmap tasks <season>. |
Total x |
- |
- |
Total x |
|
- |
||||
Tasks from ... |
||||||||||
[:AssignmentWaybackIntegration:Task Access 2.1] Wayback Into Version Control |
1,5 |
CSR |
KFC |
|
|
.. |
||||
[:AssignmentWaybackIntegration:Task Access 2.2] Ant target for deployable wayback |
2 |
CSR |
JOLF |
|
|
.. |
||||
[:AssignmentWaybackIntegration:Task Access 2.3] Create a PROPER version of NetarchiveResourceStore |
5 |
HBK |
CSR |
|
|
.. |
||||
Assignment for enhanced QA tools |
2 |
KFC |
SVC |
|
|
.. |
||||
Finalize [:AssignmentHarvester2:Assigment] for Harvester for support of WARC format |
? |
SVC |
KFC |
|
|
.. |
||||
Finalize assignment for [:AssignmentGroupB2:Assignment group B.2.2] |
0,5 |
ELZI |
KFC |
|
|
.. |
||||
Implement [:AssignmentGroupB2:Assignment B.2.2a] - Generalise replica to include all checksum voters |
14? |
JOLF |
KFC |
|
|
.. |
||||
Implement [:AssignmentGroupB2:Assignment B.2.2b] - Store bit preservation information in a database |
8 |
|
|
|
|
.. |
||||
Implement [:AssignmentGroupB2:Assignment B.2.3] - Use segments in bitarchives |
6 |
|
|
|
|
.. |
||||
Implement [:AssignmentGroupB2:Assignment B.2.4] - Write BitPreservation scheduler |
5 |
|
|
|
|
.. |
||||
Implement [:AssignmentGroupB2:Assignment B.2.5] - Write BitPreservation webinterface |
6 |
|
|
|
|
.. |
||||
Finalize assignment for [:AssignmentGroupB4:Assignment group B.4.4] - Yet more better infrastructure |
2 |
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
Wayback/Nutchwax tasks independent of NetarchiveSuite code-freeze. |
Total x |
- |
- |
Total x |
|
- |
||||
Tasks from ... |
||||||||||
|
|
|
|
|
|
.. |
||||
|
5 |
|
|
|
|
.. |
||||
[:AssignmentWaybackIntegration:Task Access 2.4] Deduplicated CDX Indexing (Technical investigation) |
1 |
|
|
|
|
.. |
||||
Evaluation of NutchWax. |
2? |
|
|
|
|
.. |
||||
Technical decision on type of production HW for Wayback and Nutchwax. |
2? |
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
Converting old Web collections to Netarchive.dk. See [http://udvikling.kb.dk/cvsshadow/digiliv/ProjektDokumenter/omkostninger%20ved%20indsamling%20af%20gammelt%20materiale-3.doc proposal]. These task will be independent of NetarchiveSuite code-freeze. |
Total x |
- |
- |
Total x |
|
- |
||||
Tasks from ... |
||||||||||
Investigation in dataformat as well as methods |
? |
SVC |
HBK |
|
|
.. |
||||
Generic converter prototype |
? |
SVC |
HBK |
|
|
.. |
||||
Old KB Webarchive |
? |
SVC |
HBK |
|
|
.. |
||||
Old Webarchive harvested with ARC-Httrack |
? |
HBK |
SVC |
|
|
.. |
||||
Old Webarchive harvested with Wget |
? |
HBK |
SVC |
|
|
.. |
||||
Old Webarchive harvested with NedLib |
? |
SVC |
HBK |
|
|
.. |
||||
Old Webarchive from Niels Brugger in waf format |
? |
HBK |
JOLF |
|
|
.. |
||||
Old Webarchive from Kurt Vest Nielsen (Ingeniøren from 1995) |
? |
JOLF |
HBK |
|
|
.. |
||||
Webarchive from the library of The Danish Parliament |
? |
SVC |
HBK |
|
|
.. |
||||
Old Webarchives from Net-papers |
? |
SVC |
HBK |
|
|
.. |
||||
Digital publications of The Danish Law Gazette from the missing period |
? |
SVC |
HBK |
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
Common tasks calculated as implementation tasks |
Total x |
- |
- |
Total x |
|
- |
||||
Others |
Total x |
- |
- |
SubTotal x |
|
- |
||||
Setup of new KB test system |
|
TLR |
|
|
|
.. |
||||
Setup open Crucible server |
|
KFC |
|
|
|
.. |
||||
Prepare release test |
Total x |
- |
- |
SubTotal x |
|
- |
||||
Prepare [http://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/Iteration37ReleaseTest release test] |
|
|
|
|
|
.. |
||||
Available man-days for implementation phase |
Total x |
- |
- |
Total x |
|
- |
||||
Release test phase (task ...) |
||||||||||
Release test |
Total x |
- |
- |
Total 10 |
|
- |
||||
Execute [http://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/Iteration37ReleaseTest release test]. |
|
' |
|
|
|
.. |
||||
|
|
' |
|
|
|
.. |
||||
Release notes |
Total x |
- |
- |
Total 0,5 |
|
- |
||||
|
|
|
|
|
|
- |
||||
Available man-days for release test phase |
Total x |
- |
- |
Total 10 |
|
- |
||||
Assignment phase for next iteration (task ...) |
||||||||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
Available man-days for assigment phase |
Total x |
- |
- |
Total 22 |
|
- |
||||
Timetable
Timetable iteration 37. Updated 6. June 2009 |
Start time |
End time |
Responsible |
Baseline 2. June 2009 . Start time |
Baseline 2. June 2009 . End time |
1. Implementation of decided tasks |
2. June 2009 |
2. July 2009 |
|
dd-mm-yyyy |
dd-mm-yyyy |
2. Code freeze. Create the build for release test and notify when build is ready |
27. July 2009 |
|
KFC |
dd-mm-yyyy |
|
3. Release test |
27. July 2009 |
29. July 2009 |
TLR |
dd-mm-yyyy |
dd-mm-yyyy |
4. Code unfreeze |
30. July 2009 |
|
KFC |
dd-mm-yyyy |
|
5. Assignments, bug components and bug fixes |
30. July 2009 |
31 July 2009 |
|
dd-mm-yyyy |
dd-mm-yyyy' |