== Task list and timetable for iteration 45 == ||<#cccccc style="text-align: center">'''Status''' ||<#cccccc>'''OK/Not Ok''' || ||1. Highlights approved ||OK || ||2. Assignment of tasks ||OK || ||3. Task list and time table approved ||OK || ||4. Implementation phase started ||OK || ||5. Release test phase started ||OK || ||6. Assignment phase for next iteration started || || ||7. Iteration 45 completed || || === Highlights for Iteration === * Implement high priority FR from BNF, ONB and Netarkivet * [[http://kb-prod-udv-001.kb.dk/twiki/bin/edit/Netarkiv/SupportNetarchiveSuite|Support]] of released !NetarchiveSuite (http://netarchive.dk/suite). * Fix prioritized bugs * Support of Wayback in the Netarchive.dk production site. See [[IntegrationOfWaybck|List of tasks]] and [[AssignmentWaybackIntegration|Assignment]] for Wayback Integration * Migration of old Web materials to Netarchive.dk * Iteration 45 is planned as a stable release. === Development procedure === * Implementation according to [[http://netarchive.dk/suite/Development|implementation methodology]] * Implementation and release test mainly in [[http://www.google.com/calendar/render?gsessionid=tjZgbhGt6eNBB1mrlNwt3A|intensive period]] * Target release: Primo October 2010 === Table of tasks === ||'''Tasks for iteration 45. Updated 8. September 2010''' ||<#cccccc style="text-align: center">'''Estimate md''' ||<#cccccc style="text-align: center">'''Main responsible''' ||<#cccccc style="text-align: center">'''Reviewer ''' ||<10% #cccccc style="text-align: center">'''Remaining md at 8. September 2010''' ||<20% #cccccc style="text-align: center">'''Comments''' ||<#cccccc style="text-align: center">'''Status''' || ||||||||||||||||||||||<#cccccc style="text-align: center">'''Implementation phase (task x-n)''' || ||<#cccccc style="text-align: center">'''Open Source release + bugs and feature request''' ||<#cccccc style="text-align: center"> ||<#cccccc style="text-align: center">'''-''' ||<#cccccc style="text-align: center">'''-''' ||<#cccccc style="text-align: center"> ||<#cccccc style="text-align: center"> ||<#cccccc style="text-align: center">'''-''' || ||||||||||||||||||||||<#cccccc style="text-align: center">'''Support of Open Source Release''' || ||1. [[http://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/SupportNetarchiveSuite|Support]] of released !NetarchiveSuite ||2 ||'''All (Google calender)''' || ||2 || ||<#cccccc style="text-align: center">Ongoing || ||2. Implement translateprocess. Adjustment to Open Source partners. ||1 ||CSR ||SVC || || ||<#cccccc style="text-align: center">Ongoing || ||3. Maintain French Translation files. ||1 ||Nicolas/Sara ||SVC || || ||<#cccccc style="text-align: center">Ongoing || ||4. Maintain German Translation files. ||1 ||Andreas ||SVC || || ||<#cccccc style="text-align: center">Ongoing || ||5. Maintain Italian Translation files ||1 ||SVC || || || ||<#cccccc style="text-align: center">Ongoing || ||||||||||||||||||||||<#cccccc style="text-align: center">'''Bugs and Feature requests''' || ||<#cccccc style="text-align: center">Prioritized bugs according to [[https://gforge.statsbiblioteket.dk/tracker/index.php?group_id=7&atid=105|list]] of priority 4 and priority 3 tasks. ||<#cccccc style="text-align: center"> ||<#cccccc style="text-align: center">'''-''' ||<#cccccc style="text-align: center">'''-''' ||<#cccccc style="text-align: center">'''!SubTotal 0''' ||<#cccccc style="text-align: center">.. ||<#cccccc style="text-align: center">'''-''' || ||||||||||||||||||||||<#cccccc style="text-align: center">'''Priority 5 bug''' || ||'''6 Module archive: '''''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1895|Bug 1895]]'' Running checksum gives Garbage Collector !OutOfMemoryError and schedule stops and ''Bug [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=2030|Bug 2030]]'' Message saturation of bitarchive applications result in OOM ''Bug [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=2009|Bug 2009]]'' OOM i BitarchiveMonitorServer makes us forget about sent messages ||1 ||JOLF ||SVC || ||Awaiting results of observations in production. Working on patch to version 3.12.1. ||<#cccccc style="text-align: center">Need info || ||||||||||||||||||||||<#cccccc style="text-align: center">'''Priority 4 bugs''' || ||'''7. Module Archive:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1962|Bug 1962]]'' Batchjob fails when !ArcRepository is overloaded || ||JOLF ||CSR || ||Awaiting results of observations after upgrade to 3.12.1 ||<#cccccc style="text-align: center">Need Info || ||'''8. Module harvester:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1972|Bug 1972]]'' Resubmit selected failed job does not do anything and gives a webpage error || ||Nicolas ||SVC || || ||<#cccccc style="text-align: center"> || ||'''9. Module Harvester'''''' : '''''Bug 2016 ''Still missing Heritrix reports || ||Nicolas ||SVC || ||Review in branch ||<#cccccc style="text-align: center"> || ||'''10. Module Monitor''' : ''Bug 2018'' Scheduling is not always working || ||Nicolas ||SVC || ||Review in branch ||<#cccccc style="text-align: center"> || ||||||||||||||||||||||<#cccccc style="text-align: center">'''Priority 3 bugs''' || ||'''11. Module Havester: '''''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1205|Bug 1851]]'' Close connexions with Heritrix on unfinished jobs || ||?? || || || ||<#cccccc style="text-align: center"> || ||'''12. Module Harvester:''' ''Bug 2019'' Select of Harvest name may not work || ||Nicolas || || || ||<#cccccc style="text-align: center"> || || || || || || || ||<#cccccc style="text-align: center"> || || || || || || || ||<#cccccc style="text-align: center">.. || ||<#cccccc style="text-align: center">Prioritized Feature Requests according to [[TaskTableFromMay2009Workshop|list]] of priority 4 and priority 3 tasks ||<#cccccc style="text-align: center"> ||<#cccccc style="text-align: center">'''-''' ||<#cccccc style="text-align: center">'''-''' ||<#cccccc style="text-align: center">'''!SubTotal 21''' ||<#cccccc style="text-align: center"> ||<#cccccc style="text-align: center">'''-''' || ||||||||||||||||||||||<#cccccc style="text-align: center">'''Priority 4 Feature request''' || ||'''13. Module Harvester: ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1928|Feature request 1928]]''''' Ability to easily resubmit a selection of failed jobs || ||Nicolas/Sara ||CSR || ||Implemented. See outstanding bug 1972 ||<#cccccc style="text-align: center">OK || ||'''14. Module Harvester:'''''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1813|Feature request 1813]]'' An extra resubmit button to make it visible which jobs have already been handled || ||MSS ||JOLF || ||Depends on whether feature still required after implementing FR 1928 ||<#cccccc style="text-align: center">TBD whether or not to postpone this issue. || ||'''15. Module harvester:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1963|Feature request 1963]]'' Make a !HarvestSchedulerApplication that runs the !HarvestScheduler || ||MSS ||JOLF || || ||<#cccccc style="text-align: center">Done || ||'''16. Module common/archive:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=2040|Feature request 2040]]'' Enable retrieval of WARCRecords from the bitarchives || ||SVC ||JOLF || ||Development under way in branch svc_warc ||<#cccccc style="text-align: center"> || || || || || || || ||<#cccccc style="text-align: center"> || ||||||||||||||||||||||<#cccccc style="text-align: center">Priority 3 Feature request || ||'''17. Module harvester: '''''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1956|Feature request 2011]] ''Harvest documentation : generate an optional arcfiles-report.txt || ||Nicolas || || || ||<#cccccc style="text-align: center">-''' ''' || ||'''18. Module harvester:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1956|Feature request 1969]]'' Missing start and end date columns in Job status to search after || ||Nicolas || || || ||<#cccccc style="text-align: center"> || ||'''19. Module Archive''':''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1743|Feature request 1743]]'' When accessing Bitpreservation this takes really long time || ||JOLF || || || ||<#cccccc style="text-align: center">- || || || || || || || ||<#cccccc style="text-align: center"> || || || || || || || ||<#cccccc style="text-align: center"> || ||<#cccccc style="text-align: center">Roadmap tasks ||<#cccccc style="text-align: center">Total 52? ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">Total 8,5 ||<#cccccc style="text-align: center"> ||<#cccccc style="text-align: center">- || ||||||||||||||||||||||<#cccccc style="text-align: center">Tasks from ... || ||'''20. QA:''' [[QaTools|Assignment for enhanced QA tools]] ||2 ||SVC ||CSR || ||High priority ||<#cccccc style="text-align: center">On going || ||'''21. WARC:''' Finalize [[AssignmentHarvester2|Assigment]] for Harvester for support of WARC format ||? || || || || ||<#cccccc style="text-align: center"> || ||'''22. Archive: '''Implement [[AssignmentGroupB2|Assignment B.2.3]] - Use segments in bitarchives ||6 || || || || ||<#cccccc style="text-align: center"> || ||'''23. Archive: '''Implement [[AssignmentGroupB2|Assignment B.2.4]] - Write !BitPreservation scheduler ||5 || || || || ||<#cccccc style="text-align: center"> || ||'''24. Archive: '''Implement [[AssignmentGroupB2|Assignment B.2.5]] - Write !BitPreservation webinterface ||6 || || || || ||<#cccccc style="text-align: center"> || || || || || || || ||<#cccccc style="text-align: center">- || || || || || || || ||<#cccccc style="text-align: center">.. || ||<#cccccc style="text-align: center">[[http://netarkivet.dk/netarkivet/index.php?title=Kendte_problemer|Crawl-problems]] (Netarchive.dk) . ||<#cccccc style="text-align: center">Total x ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">Total x ||<#cccccc style="text-align: center"> ||<#cccccc style="text-align: center">- || ||||||||||||||||||||||<#cccccc style="text-align: center">Focus on following crawl-problems || ||25. Support to [[http://netarkivet.dk/netarkivet/index.php?title=Kendte_problemer|Crawl-problems]] || ||CSR ||SVC ||1 ||High priority ||<#cccccc style="text-align: center">On going || ||26. Test of [[ConclusionLiwaAug2010|conclusion]] from LIWA project || ||MSS ||SVC || ||High priority ||<#cccccc style="text-align: center">Done || || || || || || || ||<#cccccc style="text-align: center"> || || || || || || || ||<#cccccc style="text-align: center"> || ||<#cccccc style="text-align: center">Wayback/Nutchwax tasks independent of !NetarchiveSuite code-freeze. ||<#cccccc style="text-align: center">Total x ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">Total x ||<#cccccc style="text-align: center"> ||<#cccccc style="text-align: center">- || ||||||||||||||||||||||<#cccccc style="text-align: center">Tasks from ... || ||'''27.Wayback: '''Documentation of Indexer/Aggregator ||2 ||CSR ||JOLF ||1 ||High priority ||<#cccccc style="text-align: center">Done (Additonal Tools + Config Manual). To be reviewed. || ||'''28. Wayback deploy: '''Complete Wayback deploy ||2 ||JOLF ||CSR ||2 ||High priority ||<#cccccc style="text-align: center">- || || || || || || || ||<#cccccc style="text-align: center">.. || ||<#cccccc style="text-align: center">Converting old Web collections to Netarchive.dk. See [[http://udvikling.kb.dk/cvsshadow/digiliv/ProjektDokumenter/omkostninger%20ved%20indsamling%20af%20gammelt%20materiale-3.doc|proposal]]. These task will be independent of !NetarchiveSuite code-freeze. ||<#cccccc style="text-align: center">Total x ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">Total x ||<#cccccc style="text-align: center"> ||<#cccccc style="text-align: center">- || ||||||||||||||||||||||<#cccccc style="text-align: center">Tasks from ... || ||'''29. Old Web collection:''' Old KB Webarchive || ||SVC ||JOLF || ||High priority ||<#cccccc style="text-align: center">In progress || ||'''30.''' '''Old Web collection: '''Old Webarchive from Niels Brugger collected by HTTrack || ||MSS ||SVC || ||High priority ||<#cccccc style="text-align: center">Awaiting staging server || ||'''31.''' '''Old Web collection: '''Prepare ingest of extracted data from Internet Archive into Netarkivet.dk || ||SVC ||MSS || ||High priority ||<#cccccc style="text-align: center">In progress || ||'''32.''' '''Old Web collection:''' Ingest received data from Internet Archive into Netarkivet.dk || ||CLO ||SVC || || ||<#cccccc style="text-align: center">- || ||<#cccccc style="text-align: center">Common tasks calculated as implementation tasks ||<#cccccc style="text-align: center">Total x ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">Total x ||<#cccccc style="text-align: center"> ||<#cccccc style="text-align: center">- || ||<#cccccc style="text-align: center">Others ||<#cccccc style="text-align: center">Total x ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">!SubTotal 2 ||<#cccccc style="text-align: center"> ||<#cccccc style="text-align: center">- || ||'''33. Batch:''' Create/execute a batch test script specified by 1 or 2 researchers ||2 ||JOLF ||TLR || || ||<#cccccc style="text-align: center"> || || || || || || || ||<#cccccc style="text-align: center">.. || || || || || || || ||<#cccccc style="text-align: center">.. || ||<#cccccc style="text-align: center">Prepare release test ||<#cccccc style="text-align: center">Total x ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">!SubTotal 12 ||<#cccccc style="text-align: center"> ||<#cccccc style="text-align: center">- || ||34. Prepare [[http://netarchive.dk/suite/Iteration45Releasetest|release test]] ||6 ||TLR || ||6 || ||<#cccccc style="text-align: center">OK || ||<#cccccc style="text-align: center">Available man-days for implementation phase ||<#cccccc style="text-align: center">Total x ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">Total x ||<#cccccc style="text-align: center"> ||<#cccccc style="text-align: center">- || ||||||||||||||||||||||<#cccccc style="text-align: center">Release test phase (task ...) || ||<#cccccc style="text-align: center">Release test ||<#cccccc style="text-align: center">Total x ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">Total 12 ||<#cccccc style="text-align: center"> ||<#cccccc style="text-align: center">- || ||35. Execute [[http://netarchive.dk/suite/Iteration45Releasetest|release test]]. ||12 ||TLR ||All ||12 || ||<#cccccc style="text-align: center">In progress || || || ||' || || || ||<#cccccc style="text-align: center">.. || ||<#cccccc style="text-align: center">Release notes ||<#cccccc style="text-align: center">Total x ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">Total 0,5 ||<#cccccc style="text-align: center"> ||<#cccccc style="text-align: center">- || ||36. Write release notes ||0,5 ||SVC || || || ||<#cccccc style="text-align: center">- || ||<#cccccc style="text-align: center">Available man-days for release test phase ||<#cccccc style="text-align: center">Total x ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">Total 10 ||<#cccccc style="text-align: center"> ||<#cccccc style="text-align: center">- || ||||||||||||||||||||||<#cccccc style="text-align: center">Assignment phase for next iteration (task ...) || ||37. Component bug/feature fix/management || ||QA || || || ||<#cccccc style="text-align: center">.. || ||38. Define goals for [[http://netarchive.dk/suite/Iteration46TaskList|Iteration 46 task list]] || ||CHH || || || ||<#cccccc style="text-align: center">.. || ||39. Presentation of goals and tasks for Iteration 46. Achieve a common understanding of the purpose of the iteration and each task on status meeting || ||SVC || || || ||<#cccccc style="text-align: center">.. || ||40. Assignment of tasks, bugs and feature request || ||QA || || || ||<#cccccc style="text-align: center">.. || ||41. Update release test procedure || ||TLR || || || ||<#cccccc style="text-align: center">.. || ||<#cccccc style="text-align: center">Available man-days for assigment phase ||<#cccccc style="text-align: center">Total x ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">- ||<#cccccc style="text-align: center">Total 22 ||<#cccccc style="text-align: center"> ||<#cccccc style="text-align: center">- || === Timetable === ||<#cccccc style="text-align: center">Timetable iteration 45. Updated 11. October 2010 ||<#cccccc style="vertical-align: top">Start time ||<#cccccc style="vertical-align: top">End time ||<#cccccc style="vertical-align: top">Responsible ||<#cccccc style="vertical-align: top">Baseline 3. September ''2010''. Start time ||<#cccccc style="vertical-align: top">''Baseline 3. September ''''2010'''.''''' End time ''' || ||1. Implementation of decided tasks ||8. September 2010 ||8. October 2010 || ||7. September 2010 ||28. September 2010 || ||2. Code freeze. Create the build for release test and notify when build is ready ||9. October 2010 || ||SVC ||29. September || || ||3. Release test ||11. October 2010 ||15. October 2010 ||TLR ||29. September ||1. October || ||4. Code unfreeze ||18. October 2010 || ||CSR ||4. October || || ||5. Assignments, bug components and bug fixes ||14. October 2010 ||15. October 2010 || ||30. September ||1. October ||