Task list and timetable for iteration 46
Status |
OK/Not Ok |
1. Highlights approved |
OK |
2. Assignment of tasks |
OK |
3. Task list and time table approved |
OK |
4. Implementation phase started |
OK |
5. Release test phase started |
OK |
6. Assignment phase for next iteration started |
OK |
7. Iteration 46 completed |
OK |
Highlights for Iteration
- Implement high priority FR from BNF, ONB and Netarkivet
Support of released NetarchiveSuite (http://netarchive.dk/suite).
- Fix prioritized bugs
Support of Wayback in the Netarchive.dk production site. See List of tasks and Assignment for Wayback Integration
- Migration of old Web materials to Netarchive.dk
- Iteration 46 is planned as a development release.
Development procedure
Implementation according to implementation methodology
Implementation and release test mainly in intensive period
- Target release: February 2011
Table of tasks
Tasks for iteration 46. Updated 1. March 2011 |
Estimate md |
Main responsible |
Reviewer |
Remaining md at 9. February 2011 |
Comments |
Status |
||||
Implementation phase (task x-n) |
||||||||||
Open Source release + bugs and feature request |
|
- |
- |
|
|
- |
||||
Support of Open Source Release |
||||||||||
1. Support of released NetarchiveSuite |
2 |
All (Google calender) |
|
2 |
|
Ongoing |
||||
2. Implement translateprocess. Adjustment to Open Source partners. |
1 |
CSR |
SVC |
|
|
Ongoing |
||||
3. Maintain French Translation files. |
1 |
Nicolas/Sara |
SVC |
|
|
Ongoing |
||||
4. Maintain Italian and german Translation files. |
1 |
Andreas |
SVC |
|
|
Ongoing |
||||
Bugs and Feature requests |
||||||||||
Prioritized bugs according to list of priority 4 and priority 3 tasks. |
|
- |
- |
SubTotal 0 |
.. |
- |
||||
Priority 4 bugs |
||||||||||
5. Module Access: Bug 1977: "Can not browse in ftp harvested URL's with the viewerproxy" |
|
SVC |
CSR |
|
|
Done |
||||
6. Module Archive: Bug 2098: Unresponsive bitapps |
|
JOLF |
CSR |
|
Awaiting info from production |
TGC has added more logging on SB FTB servers |
||||
7. Module Harvester: Bug 2111: Deadlock when processing a CrawlProgressMessage |
|
SVC |
Nicolas |
|
|
Done |
||||
8. |
|
|
|
|
|
|
||||
Priority 3 bugs |
||||||||||
9. Module harvester: Bug 688 hosts-report should be IDNA decoded when writing harvestInfo to the DB |
|
|
|
|
|
- |
||||
10. Module Access: Bug 823 No index = Internal server error |
|
|
|
|
|
|
||||
11. Module Monitor: Bug 1756 JMX status page does not update when a new application is started on previously used JMX port |
|
|
|
|
|
|
||||
12. Module Archive: Bug 1782 Same datetime repeated many times, while logging batch checksum of files |
|
|
|
|
|
|
||||
13. Module Archive: Bug 1708 bitpreservation logic offers "add to archive" for file that is not in either location |
|
|
|
|
|
|
||||
14. Module Archive: Bug 1193 Exceptions from FileBatchJob stop batch job processing |
|
|
|
|
|
|
||||
15. Module Archive: Bug 1260 Too much and wrong feedback information on "Missing pages" |
|
|
|
|
|
|
||||
16. Module Monitor: Bug 1205 Security policy for unit tests contains hardcoded path to development environment |
|
|
|
|
|
|
||||
17. Module Harvester: Bug 1851 Close connections with Heritrix on unfinished jobs |
|
?? |
|
|
Fixed and part of stable release 3.14.0 |
|
||||
18. Module Harvester:Bug 2019 Select of Harvest name may not work |
|
Nicolas |
|
|
Fixed and part of stable release 3.14.0 |
|
||||
19. Module Harvester: Bug 2073 Document postgres database |
|
MSS |
|
|
|
|
||||
20. Module Harvester: Bug 2074 Trim HarvestJobManager related documentation on the wiki |
|
MSS |
|
|
|
.. |
||||
21. Module Harvester: Bug 1968 Unable to run unit tests from ant |
|
MSS |
|
|
|
Done |
||||
22. Module Harvester: Bug 2005 Remove Common and Testutils test class references to other modules |
|
MSS |
|
|
|
|
||||
23 Module Harvester: Bug 2059 java.lang.OutOfMemoryError:Out of swap space? etc |
|
MSS |
|
|
|
Closed, fixed as part of FR 1774 |
||||
|
|
|
|
|
|
.. |
||||
Prioritized Feature Requests according to list of priority 4 and priority 3 tasks |
|
- |
- |
SubTotal 21 |
|
- |
||||
Priority 4 Feature request |
||||||||||
24 Module Access: Narc45: Researchers are not allowed to get access to E-books through Wayback |
|
CSR |
SVC |
|
|
Will be a rule in Wayback. Independent of code freeze. |
||||
25. Module Harvester:Feature request 1813 An extra resubmit button to make it visible which jobs have already been handled |
|
MSS |
JOLF |
|
Depends on whether feature still required after implementing FR 1928. CLO to decide. |
TBD whether or not to postpone this issue. |
||||
26. Module Harvester:Feature request 929 Documentation needed for how we split jobs (incl. maybe additional splitting modularity) |
2 |
SVC |
JOLF |
|
|
In progress. Independent of code freeze |
||||
27. Module Harvester: Feature request 2094 "To flick through the pages of OAI-harvest of e-books" |
|
CSR |
SVC |
|
|
Done |
||||
28. Module Harvester: Feature request 2093"Maximum time for a crawljob wanted for broad crawls" |
|
SVC |
MSS |
|
|
Done |
||||
29. Module Wayback: Update NetarchiveResourceStore to support ARC and WARC records.. |
|
SVC |
CSR |
|
|
Done |
||||
30. Module Harvester: Feature request 1774Stop using the JMS queues for queuing snapshot harvests . |
|
Nicolas |
MSS |
|
|
Postponed |
||||
31. Module Monitor: Feature request 2096 Monitor retired queues in the running jobs monitor. |
|
Nicolas |
MSS |
|
|
Done |
||||
|
|
|
|
|
|
|
||||
32. Module Harvester: Feature request 2097 Add additional tags to harvestInfo.xml. . |
|
Nicolas |
MSS |
|
|
Done |
||||
32a. Feature request 2083 Bundle PostgreSQL JDBC driver with netarchiveSuite distribution |
|
Nicolas |
MSS |
|
|
Done |
||||
32b. Feature request 2084 Allow use of queue-total-budget in place of QuotaEnforcer for crawl budget |
|
Nicolas |
MSS |
|
|
Done |
||||
32c. Feature request 2101Selective harvest details page: sort domain configurations and display the size of the list |
|
Nicolas |
MSS |
|
|
Done |
||||
|
|
|
|
|
|
|
||||
|
|
|
|
|
|
|
||||
Priority 3 Feature request |
||||||||||
|
|
|
|
|
|
- |
||||
33. Module harvester: Feature request 1969 Missing start and end date columns in Job status to search after |
|
Nicolas |
|
|
Fixed and part of stable release 3.14.0 |
|
||||
34. Module Harvester:Feature request 1681 Add seed to DB via webservice (via Browser Extension/Rich Client) |
|
Andreas |
|
|
|
- |
||||
35. Module Harvester:Feature request 1682 Statistics (DB access, scripts, batch jobs ....) |
|
Andreas |
|
|
|
- |
||||
36. Module Harvester:Feature request 1683 Util for regenerate admin.data file |
|
Andreas |
|
|
|
- |
||||
37. Module Harvester:Feature request 1684 Activity when domain is to be crawled. One table for seed |
|
Andreas |
|
|
|
- |
||||
38. Module Archive:Feature request 1743 When accessing Bitpreservation this takes really long time |
|
JOLF |
|
|
|
- |
||||
39. Module Harvester:Feature request 1120 Crawlertrap info should be shareable between institutions |
|
Andreas |
|
|
SVC will add comments to this FR. Might be an easy solution to share Crawlertraps by emailing files with crawler trap informations. |
Redundant (Copy of ??) |
||||
40. Module Harvester:Feature request 1066 Show whether seed URL existed |
|
Andreas |
|
|
|
- |
||||
|
|
|
|
|
|
|
||||
|
|
|
|
|
|
|
||||
|
|
|
|
|
|
|
||||
Roadmap tasks |
Total 52? |
- |
- |
Total 8,5 |
|
- |
||||
Tasks from ... |
||||||||||
41. QA: Assignment for enhanced QA tools |
2 |
SVC |
CSR |
|
High priority |
On going |
||||
42. WARC: Finalize Assigment for Harvester for support of WARC format |
? |
|
|
|
|
|
||||
43. Archive: Implement Assignment B.2.3 - Use segments in bitarchives |
6 |
|
|
|
|
|
||||
44. Archive: Implement Assignment B.2.4 - Write BitPreservation scheduler |
5 |
|
|
|
|
|
||||
45. Archive: Implement Assignment B.2.5 - Write BitPreservation webinterface |
6 |
|
|
|
|
|
||||
|
|
|
|
|
|
- |
||||
|
|
|
|
|
|
.. |
||||
Crawl-problems (Netarchive.dk) . |
Total x |
- |
- |
Total x |
|
- |
||||
Focus on following crawl-problems |
||||||||||
46. Support to Crawl-problems |
|
CSR |
SVC |
1 |
High priority |
On going |
||||
|
|
|
|
|
|
|
||||
|
|
|
|
|
|
|
||||
|
|
|
|
|
|
|
||||
Wayback/Nutchwax tasks independent of NetarchiveSuite code-freeze. |
Total x |
- |
- |
Total x |
|
- |
||||
Tasks from ... |
||||||||||
47. Wayback. Follow up after release to production |
|
CSR |
|
|
|
|
||||
|
|
|
|
|
|
- |
||||
|
|
|
|
|
|
.. |
||||
Converting old Web collections to Netarchive.dk. See proposal. These task will be independent of NetarchiveSuite code-freeze. |
Total x |
- |
- |
Total x |
|
- |
||||
Tasks from ... |
||||||||||
48. Old Web collection: Old KB Webarchive |
|
SVC |
JOLF |
|
High priority |
In progress |
||||
49. Old Web collection: Old Webarchive from Niels Brugger collected by HTTrack |
|
SB? |
SVC |
|
High priority |
Awaiting developer |
||||
50. Old Web collection: Prepare ingest of extracted data from Internet Archive into Netarkivet.dk |
|
SVC |
MSS |
|
High priority |
In progress |
||||
51. Old Web collection: Ingest received data from Internet Archive into Netarkivet.dk |
|
CLO |
SVC |
|
|
- |
||||
Common tasks calculated as implementation tasks |
Total x |
- |
- |
Total x |
|
- |
||||
Others |
Total x |
- |
- |
SubTotal 2 |
|
- |
||||
52. Batch: Create/execute a batch test script specified by 1 or 2 researchers |
2 |
JOLF |
TLR |
|
High priority |
Postponed |
||||
53. Infra-structure: Migration to SBforge |
|
MSS |
SVC |
|
|
In progress |
||||
54. Release test: Automate system test |
|
MSS |
TLR |
|
|
In progress |
||||
55. Unit-Test: Stabilize Unit test |
|
MSS |
SVC |
|
|
Done |
||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
|
|
|
|
|
|
.. |
||||
Prepare release test |
Total x |
- |
- |
SubTotal 12 |
|
- |
||||
56. Prepare release test |
6 |
TLR |
|
6 |
|
Done |
||||
Available man-days for implementation phase |
Total x |
- |
- |
Total x |
|
- |
||||
Release test phase (task ...) |
||||||||||
Release test |
Total x |
- |
- |
Total 12 |
|
- |
||||
57. Execute release test. |
12 |
TLR |
All |
12 |
|
In progress |
||||
|
|
' |
|
|
|
.. |
||||
Release notes |
Total x |
- |
- |
Total 0,5 |
|
- |
||||
58. Write release notes |
0,5 |
SVC |
|
|
|
- |
||||
Available man-days for release test phase |
Total x |
- |
- |
Total 10 |
|
- |
||||
Assignment phase for next iteration (task ...) |
||||||||||
59. Component bug/feature fix/management |
|
QA |
|
|
|
.. |
||||
60. Define goals for Iteration 47 task list |
|
CHH |
|
|
|
.. |
||||
61. Presentation of goals and tasks for Iteration 46. Achieve a common understanding of the purpose of the iteration and each task on status meeting |
|
SVC |
|
|
|
.. |
||||
62. Assignment of tasks, bugs and feature request |
|
QA |
|
|
|
.. |
||||
63. Update release test procedure |
|
TLR |
|
|
|
.. |
||||
Available man-days for assigment phase |
Total x |
- |
- |
Total 22 |
|
- |
Timetable
Timetable iteration 46. Updated 1. March 2011 |
Start time |
End time |
Responsible |
Baseline 8. November 2010. Start time |
Baseline 8. November 2010. End time |
1. Implementation of decided tasks |
8. November 2010 |
16. February 2011 |
|
8. November 2010 |
17. January 2011 |
2. Code freeze. Create the build for release test and notify when build is ready |
17. February 2011 |
|
SVC |
18. January 2011 |
|
3. Release test |
17. February 2011 |
28. February 2011 |
TLR |
18. January 2011 |
21. January 2011 |
4. Code unfreeze |
1. March 2011 |
|
CSR |
24. January 2011 |
|
5. Assignments, bug components and bug fixes |
28. February 2011 |
1. March 2011 |
|
19. January 2011 |
20. January 2011 |