== Task list and timetable for iteration 36 == ||'''Status''' ||'''OK/Not Ok''' || ||1. Highlights approved ||'''OK''' || ||2. Assignment of tasks ||'''OK''' || ||3. Task list and time table approved ||'''OK''' || ||4. Implementation phase started ||'''OK''' || ||5. Release test phase started ||'''OK''' || ||6. Assignment phase for next iteration started ||'''OK''' || ||7. Iteration 36 completed ||'''OK''' || === Highlights for Iteration === * See [[https://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/GrovSkitse2008OktoberNovemberDecember|Proposal]] from October the 20th 2008 * [[http://kb-prod-udv-001.kb.dk/twiki/bin/edit/Netarkiv/SupportNetarchiveSuite|Support]] of released !NetarchiveSuite (http://netarchive.dk/suite). * Enhance !NetarchiveSuite wiki according to [[UpdateNetarchiveSuiteWiki|decided structure]]. * Start of task according to roadmap * Module '''Archive''' * Enhanced batch execution, including support for more than one jar file. * Module '''Harvester''' * Remove Sidekick * Module '''Access''' * First phase of Wayback integration * Module '''Common''' * Add deploy * Bug fixes according to [[https://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/BugPrioNovember2009|list from November the 10th 2008]] * Iteration 36 is planned as a stable release candidate. === Development procedure === * Implementation according to [[https://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/TemplateImplementationTask|implementation methodology]] * Intensive period of implementation and release test * Estimated ... for implementation. * Estimated ... for release test. * Estimated ... for assignment of tasks for iteration XX+1. * Target release:End of April 2009 === Table of tasks === ||'''Tasks for iteration 36. Updated 26-5-2009''' ||'''Estimate md''' ||'''Main responsible''' ||'''Other participants''' ||'''SVC/ md''' ||'''KFC/ md''' ||'''ELZI/ md''' ||'''CSR/ md''' ||'''TLR/ md''' ||'''Remaining md at 26-5-2009''' ||'''Status''' || ||||||||||||||||||||||'''Implementation phase (task x-n)''' || ||'''Open Source+ Infrastructure+ bugs and feature request''' ||'''Total 5''' ||'''-''' ||'''-''' ||'''SVC/ md''' ||'''KFC/ md''' ||'''ELZI/ md''' ||'''CSR''''''/ md''' ||'''TLR/ md''' ||'''Total 0''' ||'''-''' || ||||||||||||||||||||||'''Support of Open Source Release''' || ||1. [[http://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/SupportNetarchiveSuite|Support]] of released !NetarchiveSuite ||2 ||'''See [[http://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/PlanningRollen#PrimaryIBworkingDay|watchman list]]''' ||- || || || || || ||0 ||'''Ongoing ''' || ||2. Finalise the [[http://netarchive.dk/suite/Development#head-dafa3b77803d9a203646c210554254d8c77bcd49|bug reporting procedure]] for the Gforge bug tracker. See [[http://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/ReferatDagsordenIBMaj2008|summary]] from joint meeting. ||1 ||ELZI ||SVC || || || || || ||0 ||OK. || ||3. Finalise [[http://netarchive.dk/suite/BugInfGuide#head-a35e070a5e2e95218b5158fdfc3c12f7c5919e6e|the Process description of external patches]] ||2 ||ELZI ||SVC || || || || || ||0 ||See task 2. || ||Prioritized bugs according to [[https://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/BugPrioNovember2009|list from November the 10th 2008]]''' ''''''' ''' ||SubTotal 21,5 '''''' ||Main responsible '''''' ||Other participants '''''' ||SVC/ md '''''' ||KFC/ md '''''' ||ELZI/ md '''''' ||CSR/ md '''''' ||TLR/ md '''''' ||!SubTotal 0 '''''' ||- '''''' || ||4.'''''QA''''': [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1152|Bug 1152]] URLs with { or } not browsable. ||2 ||CSR ||SVC || || || || || ||0 ||OK. || ||5.'''''Harvester stability''': [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=934|Bug 934]] could not start harvester due to JMX problem (Adress already in use) '' ||2 ||SVC '''''' ||JOLF || || || || || ||0 ||OK. || ||6. '''''Harvester stability:''' [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1154|Bug 1154]] Timeout kills non finished job because harvester gets stuck? '' ||0 || || || || || || || ||0 ||'''Won't fix''' || ||7.'''''Harvester stability:''' [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1171|Bug 1171 ]]Heritrix initialization errors are not caught by JMXHeritrixController '' ||0 || || || || || || || ||0 ||'''Invalid''' || ||8.'''''Harvester stability:''''' [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1174|Bug 1174]] Poor error message on dead job . ||0 || || || || || || || ||0 ||Postponed Fixed when bug 1188 is fixed || ||9.'''''Harvester stability:''''' [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1188|Bug 1188]] Heritrix side exceptions on JMX calls are ignored. ||3 ||CSR ||SVC || || || || || ||0 ||Postponed || ||10.'''''Harvester stability:''''' [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1249|Bug 1249]] Heritrix puts itself into PAUSED mode if it encounteres an OOM exception. ||1 ||SVC ||CSR || || || || || ||0 ||Won't fix because it's not possible to reproduce the error. || ||11.'''''Harvester stability:''''' [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1336|Bug 1336]] Harvester job dies suddenly. ||1 ||CSR ||SVC || || || || || ||0 ||OK. || ||12.'''''Harvest configuration:''''' [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=688|Bug 688]] hosts-report should be IDNA decoded when writing harvestInfo to the DB. ||3 ||SVC ||CSR || || || || || ||0 ||Postponed Bug 596 and 597 will also be fixed. Started and will be tested by unit test only. || ||13.'''''Harvest configuration:''''' [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=928|Bug 928]] The guess of initial size of unharvested domains is very bad on harvests with a large object limit. ||0 || || || || || || || ||0 ||Postponed Fixed when bug 1281 is fixed || ||14.'''''Harvest configuration:''''' [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1226|Bug 1226]] Discrepancy between how our database are defined in the dev, and prod environments respectively. ||1 ||SVC ||CSR || || || || || ||0 ||OK. || ||15.'''''Harvest configuration:''''' [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1281|Bug 1281]] Current default value for errorFactorBestGuess is greater than errorFactorPrevResult. ||1 ||CSR ||SVC || || || || || ||0 ||Postponed || ||16.'''''Archive''''': [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=574|Bug 574]] When parsing checksum in ArcRepository we do strange things on wrong results. ||2 ||ELZI ||SVC || || || || || ||0 ||OK || ||17.'''''Archive''''': [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1193|Bug 1193]] Exceptions from FileBatchJob stop batch job processing. ||0,5 ||SVC ||JOLF || || || || || ||0 ||OK || ||18.'''''Documentation''''': [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1069|Bug 1069]] Set up proxy in Installation manual. ||1,5 ||SVC ||CSR || || || || || ||0 ||Postponed || ||19.'''''Documentation''''': [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1266|Bug 1266]] UserManual: missing screendump desc for new field in "Adding seeds to an event harvest". ||0,5 ||TLR || || || || || || ||0 ||OK. || ||20.'''''Documentation''''': [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1484|Bug 1484]] Document that we are not entirely platform independent in installation manual. ||0,5 ||SVC ||ELZI || || || || || ||0 ||Last review in Release test || ||21.'''''Dev help''''': [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=662|Bug 662]] Vi smider SQL exception info væk. ||2,5 ||CSR ||SVC || || || || || ||0 ||OK || ||22. '''''Information''''': [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1260|Bug 1260]] Too much and wrong feedback information on "Missing pages". ||0 || || || || || || || ||0 ||Postponed || ||23.'''''Information''''': [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1492|Bug 1492]] The "Missing Files" page repeats the "status" headline. ||1 ||ELZI ||SVC || || || || || ||0 ||OK || ||23x.'''''New'''''. [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1566|Bug 1566]] Several Deduplicating processes started by error - only one should be possible ||? ||SVC ||CSR || || || || || ||0 ||OK || ||23y.'''''New'''''. [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1605|Bug 1605]] Problems indexingen ||? ||JOLF ||SVC || || || || || ||0 ||Postponed. Needs more information || ||Prioritized Feature Requests according to [[https://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/BugPrioNovember2009|list from November the 10th 2008]] '''''' ||SubTotal 7 '''''' ||Main responsible '''''' ||Other participants '''''' ||SVC/ md '''''' ||KFC/ md '''''' ||ELZI/ md '''''' ||CSR/ md '''''' ||TLR/ md '''''' ||!SubTotal 0 '''''' ||- '''''' || ||24.Harvester monitoring: [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1497|Feature request 1497]] Button in !NetarchiveSuite. Insert button in !NetarchiveSuite UI to invoke Heritrix UI. ||0,5 ||SVC ||JOLF || || || || || ||0 ||OK || ||25.'''''QA''''': [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=623|Feature request 623]] We need to normalize URLs when browsing data. ||1 ||CSR ||ELZI || || || || || ||0 ||Postponed Trivial cases is solved by fixing bug 1152. Assignment for the remaining parts will be the task instead. || ||26.'''''QA''''': [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=680|Feature request 680]] Cannot browse harvested password protected material. ||2 ||SVC ||ELZI || || || || || ||0 ||Postponed Implementation postponed. Assigments is written instead. || ||27.'''''Documentation:''''' [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1389|Feature request 1389]] Installation Manual should document the need to set the maximum number of producers on JMS broker. ||0,5 ||SVC ||ELZI || || || || || ||0 ||Last review in Release test || ||28.'''''Dev help''':''''' '''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1101|Feature request 1101]] Should upgrade to using Java 1.6.''' ''' ||0 || || || || || || || ||0 ||Fixed || ||29.'''''Dev help:''''' [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1252|Feature request 1252]] Upgrade to Apache Derby 10.4.1.3. ||3 ||SVC || || || || || || ||0 ||OK || ||30.'''''Dev help''':''''' '''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1436|Feature request 1436]] Find replacement for com.sun.jndi.rmi.registry.RegistryContextFactory'''.''' ||1 ||CSR ||SVC || || || || || ||0 ||Postponed || ||31.'''''Information'': '''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1482|Feature request 1482]] Harvest information for job must report if there are problems in getting.''' ''' ||0 || || || || || || || ||0 ||This FR will be included in the batch assignment under the Archive assignment section || ||32.'''''Information'': '''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1483|Feature request 1483]] Alternative monitor plugin could be print of jmx url. ||0 || || || || || || || ||0 ||Fixed || ||33'''.''Information'': '''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1485|Feature request 1485]] File must be streamed instead on QA/QA-crawlloglines.jsp . ||0 || || || || || || || ||0 ||Fixed || ||33x'''. NEW ''Dev. help'': '''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1643|Feature request 1643]] Update Heritrix to version 1.14.3. ||0 || || || || || || || ||0 ||Fixed || ||[[Roadmap|Roadmap tasks]]. Stable release 3.8. '''''' ||Total 11 '''''' ||Main responsible '''''' ||Other participants '''''' ||SVC/ md '''''' ||KFC/ md '''''' ||ELZI/ md '''''' ||CSR/ md '''''' ||TLR/ md '''''' ||Total 0 '''''''''''' ||- '''''' || ||34. Finalize [[AssignmentGroupB2|Assignment group B.2]] - Improve and isolate bit preservation GUI ||0,5 ||ELZI ||KFC ||'''''' || || || || ||0 ||OK '''''' || ||35. Finalize assignment for [[AssignmentGroupB4|Assignment group B.4.4]] - Yet more better infrastructure ||2 ||ELZI ||SVC || || || || || ||0 ||Postponed '''''' || ||36. Write [[http://netarchive.dk/suite/AssignmentWayback|assignment]] for integration of Wayback. Includes analyse of the future requirements for index handling ||2,5 ||CSR ||SVC || || || || || ||0 ||OK. || ||36a. Task [[http://netarchive.dk/suite/AssignmentWayback|P1.1.1]]: Identify subset of CDX files for pre-prototype resourceIndex testing ||1 ||CSR ||SVC || || || || || ||x ||For Task 36a-36e missing 3 md for implementation and 1 md for review. Start of release test depends not on finalizing task 36a-36e. || ||36b. Task [[http://netarchive.dk/suite/AssignmentWayback|P1.1.2]]: Configure a prototype resourceIndex tester ||1 ||CSR ||SVC || || || || || ||x ||See task 36a. || ||36c. Task [[http://netarchive.dk/suite/AssignmentWayback|P1.1.3]]: pre-prototype resourceIndex performance testing ||3 ||CSR ||SVC || || || || || ||x ||See task 36a. || ||36d. Task [[http://netarchive.dk/suite/AssignmentWayback|P1.1.4]]: identify requirements for WBS resourceIndex server ||3 ||CSR ||SVC || || || || || ||x ||See task 36a. || ||36e. Task [[http://netarchive.dk/suite/AssignmentWayback|P1.1.5]]: configure prototype resourceIndex server ||2 ||CSR ||SVC || || || || || ||x ||See task 36a. || ||37. Write assignment for new deploy. See [[http://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/ProposalKFC21Aug2008|proposal]] from KFC. ||2,5 ||ELZI ||SVC || || || || || ||x ||See task 36a || ||38. Implement new deploy. ||?? ||JOLF ||ELZI || || || || || ||0 ||OK. || ||39. Assigment for support of WARC format. ||3,5 ||SVC ||CSR || || || || || ||0 ||Postponed || ||Other tasks calculated as implementation tasks '''''' ||Total 3,5 '''''' ||Main responsible '''''' ||Other participants '''''' ||SVC/ md '''''' ||KFC/ md '''''' ||ELZI/ md '''''' ||CSR/ md '''''' ||TLR/ md '''''' ||Total 0 '''''' ||- '''''' || ||40. [[http://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/KFCopgaverIndenOrlov2008|KFC tasks]] before December the 1st, 2008 ||1 ||KFC ||All || || || || || ||0 ||..''' ''' || ||41. Test/integrate review tool [[http://www.atlassian.com/software/crucible/|Crucible]] ||1 ||SVC ||CSR || || || || || ||0 ||OK. Crucible is operational. See [[http://netarchive.dk/suite/Process/Code_Review|Process documentation]]''' ''' || ||42. Update of manuals ||1 ||ELZI ||All || || || || || ||0 ||OK. || ||43. Role management ||0,5 ||- || || || || || || || ||On going || ||44. Update of deduplication section in developer manual ||2 ||SVC ||ELZI || || || || || ||0 ||OK || ||45. Specification of current and future requirements for development test platform ||0,5 ||TLR ||CSR || || || || || || ||In progress || ||46. Installation and test of configuration of new KB Bit platform ||2 ||TLR ||CLO || || || || || || ||On going. || ||47. Update of deploy to support new KB Bit platform ||0,5 ||ELZI ||TLR || || || || || || ||On going || ||||||||||||||||||||||Prepare release test of Iteration 36 '''''' || ||48. Prepare [[http://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/Iteration36ReleaseTest|release test]]. ||6 ||TLR ||All || || || || || ||0 ||OK || ||Total md implementation phase (Task 1-44) '''''' ||Total 48 '''''' ||- '''''' ||- '''''' ||14 '''''' ||18 '''''' ||20 '''''' ||0 '''''' ||6 ||Total 0 '''''' ||- '''''' || ||||||||||||||||||||||Release test phase (task 20-22) '''''' || ||Release test '''''' ||Total 9 '''''' ||Main responsible '''''' ||Other participants '''''' ||- '''''' ||- '''''' ||- '''''' ||- '''''' ||- '''''' ||Total 0 '''''' ||- '''''' || ||49. Execute [[http://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/Iteration36ReleaseTest|release test]]. ||6 ||TLR '''''' ||CSR,JOLF,ELZI,SVC ||1 ||1 ||1 || ||3 ||0 ||Started '''''''''''' || ||50. Component bug/feature fix/management ||2,5 ||QA ||CSR,JOLF,ELZI,SVC ||1 ||0,5 ||1 || || ||0 ||.. '''''' || ||Release notes '''''' ||Total 0,5 '''''' ||- '''''' ||- '''''' ||SVC/ md '''''' ||KFC/ md '''''' ||ELZI/ md '''''' ||NN/ md '''''' ||TLR/ md '''''' ||Total 0,5 '''''' ||- '''''' || ||51. Write [[http://netarchive.dk/suite/Release_Overview|release notes]] for iteration 36 ||0,5 ||SVC ||CSR || ||0,5 || || || ||0,5 ||- '''''''''''' || ||Total for release test phase '''''''''''' ||Total 5.5 '''''''''''' ||- '''''''''''' ||- '''''''''''' ||2 '''''''''''' ||2 '''''''''''' ||2 '''''''''''' ||0 '''''''''''' ||3 '''''''''''' ||Total 4,5 '''''''''''' ||- '''''''''''' || ||||||||||||||||||||||Assignment phase for next iteration (task 23-27) '''''''''''' || ||52. Component bug/feature fix/management ||1,5 ||QA '''''' ||All ||0,5 ||0,5 ||0,5 || || ||1,5 ||Regular process '''''' || ||53. Define goals for [[Iteration37TaskList|Iteration 37 task list.]] ||0,1 ||CHH ||CSR || ||0,1 || || || ||0,1 ||- '''''' || ||54. Presentation of goals and tasks for Iteration 37. Achieve a common understanding of the purpose of the iteration and each task on status meeting ||0,1 ||CSR '''''' ||All || ||0,1 || || || ||0,1 ||- '''''' || ||55. Assignment of tasks, bugs and feature request ||3 ||CHH ||All ||1 ||1 ||1 || || ||3 ||Regular process '''''' || ||56. Update release test procedure ||1 ||TLR '''''' ||All || || || || ||1 || ||Regular process '''''' || ||Available man-days for assigment phase''' ''' ||Total 63''' ''' ||-''' ''' ||-''' ''' ||6''' ''' ||6''' ''' ||2''' ''' ||6''' ''' ||2''' ''' ||Total 64,5''' ''' ||-''' ''' || === Detailed timetable === Iteration36DetailedTimetable (use template IterationDetailedTimetableTemplate (TBD)) === Timetable === ||Timetable iteration 36 Updated 10-2-2009''' ''' ||Start time''' ''''''''' ||End time''' ''''''''' ||Responsible''' ''' ||''Baseline 24-10-2008. Start time''' ''''' ||''Baseline 24-10-2008. End time''''' ''' || ||1. Implementation of decided tasks ||17-11-2008 ||15-04-2009 ||- ||''17-11-2008 '' ||''16-02-2009 '' || ||2. Code freeze. Create the build for release test and notify when build is ready ||16-4-2009 ||- ||SVC ||''16-2-2009 '' ||- || ||3. Release test ||20-04-2009 ||25-5-2009 ||TLR ||''17-02-2009 '' ||''19-02-2009 '' || ||4. Code unfreeze ||25-5-2009 ||- ||SVC ||''20-02-2009 '' ||- || ||5. Assignments, bug components and bug fixes ||26-5-2009 ||9-6-2009 || ||''20-02-2009 '' ||''22-02-2009 '' ||