Task list and timetable for iteration 40
Status |
OK/Not Ok |
1. Highlights approved |
OK |
2. Assignment of tasks |
OK |
3. Task list and time table approved |
OK |
4. Implementation phase started |
OK |
5. Release test phase started |
OK |
6. Assignment phase for next iteration started |
OK |
7. Iteration 40 completed |
|
Highlights for Iteration
Support of released NetarchiveSuite (http://netarchive.dk/suite).
Enhance NetarchiveSuite wiki according to decided structure.
- Implement prioritized bugs and feature requests
- Enhancement of Batch support
- Review of WARC Tools Phase III SRS
Support of Wayback in the Netarchive.dk production site. See List of tasks and Assignment for Wayback Integration
- Migration of old Web materials to Netarchive.dk
- Iteration 40 is planned as a development release.
Development procedure
Implementation according to implementation methodology
Implementation and release test mainly in intensive period
- Target release: December 2009
Table of tasks
Tasks for iteration 40. Updated 15. December 2009 |
Estimate md |
Main responsible |
Reviewer |
Remaining md at 15. December 2009 |
Comments |
Status |
||||
Implementation phase (task x-n) |
||||||||||
Open Source release + bugs and feature request |
Total 3 |
- |
- |
Total 3 |
|
- |
||||
Support of Open Source Release |
||||||||||
1. Support of released NetarchiveSuite |
2 |
All (Google calender) |
|
2 |
|
Ongoing |
||||
2. Implement translateprocess. Adjustment to Open Source partners. |
1 |
CSR |
SVC |
|
|
- |
||||
Bugs and Features requests |
||||||||||
Prioritized bugs according to list of priority 4 and priority 3 tasks. |
Total 5 |
- |
- |
SubTotal 0 |
.. |
- |
||||
Priority 4 bugs |
||||||||||
3. Module Access: Bug 1758 UrlCanonicalizerFactory falls back to default value silently |
0,5 |
CSR |
SVC |
0 |
High Priority |
OK |
||||
4. Module Harvester: Bug 1777 Add event seeds only accepts a very short list of seeds |
3 |
SVC |
HBK |
0 |
High priority |
OK |
||||
Priority 3 bugs |
||||||||||
5. Module Access: Bug 1796 Configurations manual doesn't include anything about setting up wayback |
1 |
HBK |
SVC |
0 |
|
OK |
||||
6. Module Access: Bug 823 No index = Internal server error |
|
|
|
|
|
|
||||
7. Module Monitor: Bug 1756 JMX status page does not update when a new application is started on previously used JMX port |
|
|
|
|
|
|
||||
8. Module Archive: Bug 1782 Same datetime repeated many times, while logging batch checksum of files |
|
|
|
|
|
|
||||
9. Module Documentation: Bug 1779 Improve documentation of the additional tools |
|
|
|
|
|
|
||||
10. Module Archive: Bug 1764 Poor information on failed batch job |
|
|
|
|
|
|
||||
11. Module Documentation: Bug 1732 LocalArcRepositoryClient not documented |
|
|
|
|
|
|
||||
|
|
|
|
|
|
|||||
13. Module Archive: Bug 1708 bitpreservation logic offers "add to archive" for file that is not in either location |
|
|
|
|
|
|
||||
14. Module Deploy: Bug 1705 Make jmxremote.access writable before overwriting it (install script) |
|
JOLF |
SVC |
|
|
OK |
||||
15. Module Archive: Bug 1619 Potential NullPointer exception in RemoveAndGetFileMessage.getData() |
|
|
|
|
|
|
||||
16. Module Archive: Bug 1260 Too much and wrong feedback information on "Missing pages" |
|
|
|
|
|
|
||||
17. Module Monitor: Bug 1205 Security policy for unit tests contains hardcoded path to development environment |
|
|
|
|
|
|
||||
18. Module Archive: Bug 1193 Exceptions from FileBatchJob stop batch job processing |
|
|
|
|
|
|
||||
19. |
|
|
|
|
|
.. |
||||
Prioritized Feature Requests according to list of priority 4 and priority 3 tasks |
Total 21 |
- |
- |
SubTotal 21 |
|
- |
||||
Priority 4 Feature request |
||||||||||
20. Module harvester: Feature request 1116 Global crawlertraps |
6 |
CSR |
SVC |
0 |
Assignment . See also FR 1120 Review decides whether some of the tasks will be implemented. Backend. To be finished in Iteration 41. Decision: Only assignment. |
OK |
||||
21. Module harvester: Feature request 1696 Ingest domain seed URLs |
5 |
Nicolas |
SVC |
0 |
|
Postponed |
||||
22. Module harvester: Feature request 1688 Monitoring broad crawls. |
5 |
Nicolas |
SVC |
0 |
FR under discussion on mailing list. |
Postponed |
||||
23. Module archive: Feature request 1736 Monitoring batchjobs |
2 |
JOLF |
HBK |
0 |
New feature request for restructuring the Batch architecture has been made. This FR only deals with improved logging. |
OK. |
||||
24. Module monitor: Feature request 1757 Need a way to remove an application from lists of monitored applications |
2 |
HBK |
SVC |
0 |
High priority . |
OK |
||||
Priority 3 Feature request |
||||||||||
25.Module Harvester: Feature request 1774 Stop using the JMS queues for queuing snapshot harvests |
|
|
|
|
|
|
||||
26. Module Harvester: Feature request 1681 Add seed to DB via webservice (via Browser Extension/Rich Client) |
? |
Andreas |
|
|
|
Started Postponed |
||||
27. Module Harvester: Feature request 1682 Statistics (DB access, scripts, batch jobs ....) |
? |
Andreas |
|
|
|
? |
||||
28. Module Harvester: Feature request 1683 Util for regenerate admin.data file |
? |
Andreas |
|
|
|
? |
||||
29. Module Harvester: Feature request 1684 Activity when domain is to be crawled. One table for seed |
? |
Andreas |
|
|
|
? |
||||
30. Module Archive: Feature request 1743 When accessing Bitpreservation this takes really long time |
? |
Andreas |
|
|
|
? |
||||
31. Module Harvester: Feature request 1120 Crawlertrap info should be shareable between institutions |
? |
Andreas |
|
|
SVC will add comments to this FR. Might be an easy solution to share Crawlertraps by emailing files with crawler trap informations. |
Redundant (Copy of 20) |
||||
32. Module Harvester: Feature request 1066 Show whether seed URL existed |
? |
Andreas |
|
|
|
? |
||||
33. Module Archive: Feature request 1809 Write assignment for improving batchjob interface |
? |
JOLF |
|
|
|
.. |
||||
34a. Module Harvester: Feature request 1134 Filter job lists by category |
? |
Nicolas/Sara |
|
|
|
.. |
||||
34b. Module Harvester: Feature request 1668 Paginate and make sortable and searchable the list of jobs |
? |
Nicolas/Sara |
|
|
|
.. |
||||
Roadmap tasks |
Total 52? |
- |
- |
Total 8,5 |
|
- |
||||
Tasks from ... |
||||||||||
35. Review of WARC Tools Phase III SRS |
5 |
SVC |
HBK,CSR,TLR |
|
High priority |
OK |
||||
36. Assignment for enhanced QA tools |
2 |
|
|
|
|
Postponed |
||||
37. Finalize Assigment for Harvester for support of WARC format |
? |
|
|
|
|
|
||||
38. Finalize assignment for Assignment group B.2.2 |
0,5 |
|
|
|
|
|
||||
39. Implement Assignment B.2.2a - Generalise replica to include all checksum voters |
8 |
JOLF |
SVC |
0 |
High priority |
OK |
||||
40. Implement Assignment B.2.2b - Store bit preservation information in a database |
8 |
|
|
|
|
In Progress |
||||
41. Implement Assignment B.2.3 - Use segments in bitarchives |
6 |
|
|
|
|
|
||||
42. Implement Assignment B.2.4 - Write BitPreservation scheduler |
5 |
|
|
|
|
|
||||
43. Implement Assignment B.2.5 - Write BitPreservation webinterface |
6 |
|
|
|
|
|
||||
44. Finalize assignment for Assignment group B.4.4 - Yet more better infrastructure |
2 |
|
|
|
|
|
||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
Wayback/Nutchwax tasks independent of NetarchiveSuite code-freeze. |
Total x |
- |
- |
Total x |
|
- |
||||
Tasks from ... |
||||||||||
|
|
|
|
|
|
- |
||||
2? |
HBK |
CSR |
|
High priority |
OK |
|||||
|
|
|
|
|
|
In progress |
||||
|
|
|
|
|
|
.. |
||||
Converting old Web collections to Netarchive.dk. See proposal. These task will be independent of NetarchiveSuite code-freeze. |
Total x |
- |
- |
Total x |
|
- |
||||
Tasks from ... |
||||||||||
46. Investigation in dataformat as well as methods |
? |
SVC |
HBK |
|
|
In progress |
||||
47. Prepare ARC/WARC document for basis of decision to Steering group. Must old data be ingested into Neetarkivet.dk using ARC container or WARC container. |
? |
HBK |
SVC,TLR |
|
High priority |
In progress |
||||
48. Generic converter prototype |
? |
HBK |
SVC |
|
High priority |
.. Done Committed in CVS needs code review |
||||
49. Old KB Webarchive |
? |
SVC |
HBK |
|
|
In progress |
||||
50. Old Webarchive harvested with ARC-Httrack |
? |
HBK |
SVC |
|
|
.. Under dev. |
||||
51. Old Webarchive harvested with Wget |
? |
HBK |
SVC |
|
|
In progress |
||||
52. Old Webarchive harvested with NedLib |
? |
SVC |
HBK |
|
|
In progress |
||||
53. Old Webarchive from Niels Brugger in waf format |
? |
HBK |
JOLF |
|
|
.. Done Waiting for Code Review |
||||
54. Old Webarchive from Kurt Vest Nielsen (Ingeniøren from 1995) |
? |
JOLF |
HBK |
|
|
Postponed |
||||
55. Webarchive from the library of The Danish Parliament |
? |
SVC |
HBK |
|
|
Postponed |
||||
56. Old Webarchives from Net-papers |
? |
SVC |
HBK |
|
|
Postponed |
||||
57. Digital publications of The Danish Law Gazette from the missing period |
? |
SVC |
HBK |
|
|
Postponed |
||||
58. Old Webarchive from Niels Brugger collected by HTTrack |
? |
HBK |
SVC |
|
|
Postponed |
||||
59. Prepare ingest of extracted data from Internet Archive into Netarkivet.dk |
|
SVC |
HBK |
|
High priority.Output will be a document showing what has been made of choice and an instruction to the daily manager of Netarkivet.dk of how to ingest the data. |
In progress |
||||
60. Ingest received data from Internet Archive into Netarkivet.dk |
|
CLO |
HBK |
|
High priority |
Awaiting document from task 59. |
||||
Common tasks calculated as implementation tasks |
Total x |
- |
- |
Total x |
|
- |
||||
Others |
Total x |
- |
- |
SubTotal 2 |
|
- |
||||
61. Setup of new KB test system |
2 |
TLR |
SVC |
2 |
High priority |
In progress |
||||
62. Test of 64 bit version of KB-PROD-ADM |
2 |
TLR |
SVC |
2 |
High priority |
.. |
||||
63. Architectual consideration: Move harvesters close to the backbone of the research network. |
2 |
SVC |
HBK |
2 |
High priority |
.. |
||||
64. Create/execute a batch test script specified by 1 or 2 researches |
2 |
JOLF |
HBK |
2 |
|
.. |
||||
65. Prepare joint face to face meeting with UDV and Pligt/Natinal |
1 |
CHH |
CSR |
1 |
|
.. |
||||
66. Prepare action plan for development activities |
1 |
CHH |
CSR |
1 |
High priority |
.. |
||||
67. Status of Crawl-problems (Netarchive.dk) |
1 |
CHH |
CSR |
1 |
|
.. |
||||
Prepare release test |
Total x |
- |
- |
SubTotal 12 |
|
- |
||||
68. Prepare release test |
6 |
|
|
0 |
|
OK |
||||
Available man-days for implementation phase |
Total x |
- |
- |
Total x |
|
- |
||||
Release test phase (task ...) |
||||||||||
Release test |
Total x |
- |
- |
Total 12 |
|
- |
||||
69. Execute release test. |
12 |
TLR |
All |
12 |
|
In progress |
||||
|
|
' |
|
|
|
.. |
||||
Release notes |
Total x |
- |
- |
Total 0,5 |
|
- |
||||
70. Write release note |
0,5 |
SVC |
|
|
|
Awaiting end of code freeze |
||||
Available man-days for release test phase |
Total x |
- |
- |
Total 10 |
|
- |
||||
Assignment phase for next iteration (task ...) |
||||||||||
71. Component bug/feature fix/management |
|
QA |
|
|
|
.. |
||||
72. Define goals for Iteration 41 task list |
|
CHH |
|
|
|
.. |
||||
73. Presentation of goals and tasks for Iteration 41. Achieve a common understanding of the purpose of the iteration and each task on status meeting |
|
SVC |
|
|
|
.. |
||||
74. Assignment of tasks, bugs and feature request |
|
QA |
|
|
|
.. |
||||
75. Update release test procedure |
|
TLR |
|
|
|
.. |
||||
Available man-days for assigment phase |
Total x |
- |
- |
Total 22 |
|
- |
Timetable
Timetable iteration 40. Updated 13. November 2009 |
Start time |
End time |
Responsible |
Baseline 13. November 2009. Start time |
Baseline 13. November 2009. End time |
1. Implementation of decided tasks |
16. November 2009 |
11. December 2009 |
|
16. November 2009 |
11. December 2009 |
2. Code freeze. Create the build for release test and notify when build is ready |
14. December 2009 |
|
SVC |
14. December 2009 |
|
3. Release test |
14. December 2009 |
16. December 2009 |
TLR |
14. December 2009 |
16. December 2009 |
4. Code unfreeze |
17. December 2009 |
|
SVC |
17. December 2009 |
|
5. Assignments, bug components and bug fixes |
16. December 2009 |
18. December 2009 |
|
16. December 2009 |
18. December 2009 |
eration39TaskList here.