Differences between revisions 63 and 64
Revision 63 as of 2010-07-30 05:43:46
Size: 40937
Comment:
Revision 64 as of 2010-08-16 10:24:09
Size: 41037
Editor: localhost
Comment: converted to 1.6 markup
Deletions are marked like this. Additions are marked like this.
Line 12: Line 12:
 * [http://kb-prod-udv-001.kb.dk/twiki/bin/edit/Netarkiv/SupportNetarchiveSuite Support] of released !NetarchiveSuite (http://netarchive.dk/suite).  * [[http://kb-prod-udv-001.kb.dk/twiki/bin/edit/Netarkiv/SupportNetarchiveSuite|Support]] of released !NetarchiveSuite (http://netarchive.dk/suite).
Line 14: Line 14:
 * Support of Wayback in the Netarchive.dk production site. See [:IntegrationOfWaybck:List of tasks] and [:AssignmentWaybackIntegration:Assignment] for Wayback Integration  * Support of Wayback in the Netarchive.dk production site. See [[IntegrationOfWaybck|List of tasks]] and [[AssignmentWaybackIntegration|Assignment]] for Wayback Integration
Line 18: Line 18:
 * Implementation according to [http://netarchive.dk/suite/Development implementation methodology]
 * Implementation and release test mainly in [http://www.google.com/calendar/render?gsessionid=tjZgbhGt6eNBB1mrlNwt3A intensive period]
 * Implementation according to [[http://netarchive.dk/suite/Development|implementation methodology]]
 * Implementation and release test mainly in [[http://www.google.com/calendar/render?gsessionid=tjZgbhGt6eNBB1mrlNwt3A|intensive period]]
Line 26: Line 26:
||1. [http://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/SupportNetarchiveSuite Support] of released !NetarchiveSuite ||<style="TEXT-ALIGN: center">2 ||<style="TEXT-ALIGN: center">'''All (Google calender)''' ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">2 ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Ongoing || ||1. [[http://kb-prod-udv-001.kb.dk/twiki/bin/view/Netarkiv/SupportNetarchiveSuite|Support]] of released !NetarchiveSuite ||<style="TEXT-ALIGN: center">2 ||<style="TEXT-ALIGN: center">'''All (Google calender)''' ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">2 ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Ongoing ||
Line 31: Line 31:
||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Prioritized bugs according to [https://gforge.statsbiblioteket.dk/tracker/index.php?group_id=7&atid=105 list] of priority 4 and priority 3 tasks. ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''-''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''-''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''!SubTotal 0''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">.. ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''-''' || ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Prioritized bugs according to [[https://gforge.statsbiblioteket.dk/tracker/index.php?group_id=7&atid=105|list]] of priority 4 and priority 3 tasks. ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''-''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''-''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''!SubTotal 0''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">.. ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''-''' ||
Line 34: Line 34:
||<style="VERTICAL-ALIGN: top">'''5 Module monitor?:''' ''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1895 Bug 1895]'' Running checksum gives Garbage Collector !OutOfMemoryError and schedule stops ||<style="TEXT-ALIGN: center">1 ||<style="TEXT-ALIGN: center">SVC ||<style="TEXT-ALIGN: center">CSR ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Max heap for Bitarchive monitors raised to 1936MB in prod. Awaiting results of observations after upgrade of java in production environment and installation of version 3.12.1 ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> || ||<style="VERTICAL-ALIGN: top">'''5 Module monitor?:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1895|Bug 1895]]'' Running checksum gives Garbage Collector !OutOfMemoryError and schedule stops ||<style="TEXT-ALIGN: center">1 ||<style="TEXT-ALIGN: center">SVC ||<style="TEXT-ALIGN: center">CSR ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Max heap for Bitarchive monitors raised to 1936MB in prod. Awaiting results of observations after upgrade of java in production environment and installation of version 3.12.1 ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
Line 36: Line 36:
||<style="VERTICAL-ALIGN: top">'''6. Module test''' ''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1739 Bug 1739]'' Missing information in System State GUI ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">TLR ||<style="TEXT-ALIGN: center">CSR ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">. ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">OK ||
||<style="VERTICAL-ALIGN: top">'''7. Module harvester:''' ''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=2001 Bug 2001]'' No results when startdate equals enddate on page harveststatus-alljobs.jsp ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Nicolas ||<style="TEXT-ALIGN: center">SVC ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Fixed. Awaiting review ||
||<style="VERTICAL-ALIGN: top">'''8. Module Archive:''' ''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1962 Bug 1962]'' Batchjob fails when ArcRepository is overloaded ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">JOLF ||<style="TEXT-ALIGN: center">CSR ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Awaiting results of observations after upgrade to 3.12.1 ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Need Info ||
||<style="VERTICAL-ALIGN: top">'''9. Module Common:''' ''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1849 Bug 1849]'' Invalid javadoc for DomainUtils#DOMAINNAME_CHAR_REGEX_STRING ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">CSR ||<style="TEXT-ALIGN: center">SVC ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Documentation issue. ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''10. Module harvester:''' ''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1961 Bug 1961]'' Possible to store invalid set of crawlertraps to domain causing the domain to be unreadable ||<style="TEXT-ALIGN: center">1 ||<style="TEXT-ALIGN: center">SVC ||<style="TEXT-ALIGN: center">MSS ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">. ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Fixed. Awaiting review ||
||<style="VERTICAL-ALIGN: top">'''11. Module harvester:''' ''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1972 Bug 1972]'' Resubmit selected failed job does not do anything and gives a webpage error ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Nicolas ||<style="TEXT-ALIGN: center">SVC ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''11b. Module Access:''' ''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1980 Bug 1980]'' Wayback deploy script does not rebuild wayback.war ||<style="TEXT-ALIGN: center">1 ||<style="TEXT-ALIGN: center">CSR ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Test script used as template for prod deploy ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Fixed (Review not needed) ||
||<style="VERTICAL-ALIGN: top">'''6. Module test''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1739|Bug 1739]]'' Missing information in System State GUI ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">TLR ||<style="TEXT-ALIGN: center">CSR ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">. ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">OK ||
||<style="VERTICAL-ALIGN: top">'''7. Module harvester:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=2001|Bug 2001]]'' No results when startdate equals enddate on page harveststatus-alljobs.jsp ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Nicolas ||<style="TEXT-ALIGN: center">SVC ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Fixed. Awaiting review ||
||<style="VERTICAL-ALIGN: top">'''8. Module Archive:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1962|Bug 1962]]'' Batchjob fails when ArcRepository is overloaded ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">JOLF ||<style="TEXT-ALIGN: center">CSR ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Awaiting results of observations after upgrade to 3.12.1 ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Need Info ||
||<style="VERTICAL-ALIGN: top">'''9. Module Common:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1849|Bug 1849]]'' Invalid javadoc for DomainUtils#DOMAINNAME_CHAR_REGEX_STRING ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">CSR ||<style="TEXT-ALIGN: center">SVC ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Documentation issue. ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''10. Module harvester:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1961|Bug 1961]]'' Possible to store invalid set of crawlertraps to domain causing the domain to be unreadable ||<style="TEXT-ALIGN: center">1 ||<style="TEXT-ALIGN: center">SVC ||<style="TEXT-ALIGN: center">MSS ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">. ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Fixed. Awaiting review ||
||<style="VERTICAL-ALIGN: top">'''11. Module harvester:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1972|Bug 1972]]'' Resubmit selected failed job does not do anything and gives a webpage error ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Nicolas ||<style="TEXT-ALIGN: center">SVC ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''11b. Module Access:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1980|Bug 1980]]'' Wayback deploy script does not rebuild wayback.war ||<style="TEXT-ALIGN: center">1 ||<style="TEXT-ALIGN: center">CSR ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Test script used as template for prod deploy ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Fixed (Review not needed) ||
Line 47: Line 47:
||<style="VERTICAL-ALIGN: top">'''12. Module harvester:''' ''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=688 Bug 688]'' hosts-report should be IDNA decoded when writing harvestInfo to the DB ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- ||
||<style="VERTICAL-ALIGN: top">'''13. Module Access:''' ''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=823 Bug 823]'' No index = Internal server error ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''14. Module Monitor:''' ''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1756 Bug 1756]'' JMX status page does not update when a new application is started on previously used JMX port ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''15. Module Archive:''' ''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1782 Bug 1782]'' Same datetime repeated many times, while logging batch checksum of files ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''16. Module Documentation:''' ''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1779 Bug 1779]'' Improve documentation of the additional tools ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Task 16 was fixed by JOLF in iteration 43 ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">OK ||
||<style="VERTICAL-ALIGN: top">'''17. Module Archive: '''''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1708 Bug 1708]'' bitpreservation logic offers "add to archive" for file that is not in either location ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''18. Module Archive:''' ''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1193 Bug 1193]'' Exceptions from !FileBatchJob stop batch job processing ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''19. Module Archive:''' ''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1260 Bug 1260]'' Too much and wrong feedback information on "Missing pages" ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''20. Module Monitor:''' ''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1205 Bug 1205]'' Security policy for unit tests contains hardcoded path to development environment ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''20a. Module Havester: '''''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1205 Bug 1851]'' Close connexions with Heritrix on unfinished jobs ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">?? ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''12. Module harvester:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=688|Bug 688]]'' hosts-report should be IDNA decoded when writing harvestInfo to the DB ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- ||
||<style="VERTICAL-ALIGN: top">'''13. Module Access:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=823|Bug 823]]'' No index = Internal server error ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''14. Module Monitor:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1756|Bug 1756]]'' JMX status page does not update when a new application is started on previously used JMX port ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''15. Module Archive:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1782|Bug 1782]]'' Same datetime repeated many times, while logging batch checksum of files ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''16. Module Documentation:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1779|Bug 1779]]'' Improve documentation of the additional tools ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Task 16 was fixed by JOLF in iteration 43 ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">OK ||
||<style="VERTICAL-ALIGN: top">'''17. Module Archive: '''''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1708|Bug 1708]]'' bitpreservation logic offers "add to archive" for file that is not in either location ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''18. Module Archive:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1193|Bug 1193]]'' Exceptions from !FileBatchJob stop batch job processing ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''19. Module Archive:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1260|Bug 1260]]'' Too much and wrong feedback information on "Missing pages" ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''20. Module Monitor:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1205|Bug 1205]]'' Security policy for unit tests contains hardcoded path to development environment ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''20a. Module Havester: '''''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1205|Bug 1851]]'' Close connexions with Heritrix on unfinished jobs ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">?? ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
Line 60: Line 60:
||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Prioritized Feature Requests according to [:TaskTableFromMay2009Workshop:list] of priority 4 and priority 3 tasks ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''-''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''-''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''!SubTotal 21''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''-''' || ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Prioritized Feature Requests according to [[TaskTableFromMay2009Workshop|list]] of priority 4 and priority 3 tasks ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''-''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''-''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''!SubTotal 21''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''-''' ||
Line 62: Line 62:
||<style="VERTICAL-ALIGN: top">'''23. Module Harvester: ''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1928 Feature request 1928]''''' Ability to easily resubmit a selection of failed jobs ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Nicolas/Sara ||<style="TEXT-ALIGN: center">CSR ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Implemented. See outstanding bug 1972 ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Fixed. Awaiting review ||
||<style="VERTICAL-ALIGN: top">'''24. Module Harvester:'''''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1813 Feature request 1813]'' An extra resubmit button to make it visible which jobs have already been handled ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">MSS ||<style="TEXT-ALIGN: center">JOLF ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Depends on whether feature still required after implementing FR 1928 ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''25. Module Harvester:'''''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=929 Feature request 929]'' Documentation needed for how we split jobs (incl. maybe additional splitting modularity) ||<style="TEXT-ALIGN: center">2 ||<style="TEXT-ALIGN: center">SVC ||<style="TEXT-ALIGN: center">JOLF ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Postponed ||
||<style="VERTICAL-ALIGN: top">'''26. Module Harvester:'''''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1774 Feature request 1774]'' Stop using the JMS queues for queuing snapshot harvests ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">MSS ||<style="TEXT-ALIGN: center">SVC ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''27. Module harvester:''' ''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1963 Feature request 1963]'' Make a !HarvestSchedulerApplication that runs the !HarvestScheduler ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">MSS ||<style="TEXT-ALIGN: center">JOLF ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''28. Module access:''' ''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1956 Feature request 1956]'' Shared dependency links harvester and wayback modules ||<style="TEXT-ALIGN: center">0.5 ||<style="TEXT-ALIGN: center">SVC ||<style="VERTICAL-ALIGN: top">CSR ||<style="VERTICAL-ALIGN: top"> ||<style="VERTICAL-ALIGN: top"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">OK ||
||<style="VERTICAL-ALIGN: top">'''23. Module Harvester: ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1928|Feature request 1928]]''''' Ability to easily resubmit a selection of failed jobs ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Nicolas/Sara ||<style="TEXT-ALIGN: center">CSR ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Implemented. See outstanding bug 1972 ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Fixed. Awaiting review ||
||<style="VERTICAL-ALIGN: top">'''24. Module Harvester:'''''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1813|Feature request 1813]]'' An extra resubmit button to make it visible which jobs have already been handled ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">MSS ||<style="TEXT-ALIGN: center">JOLF ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Depends on whether feature still required after implementing FR 1928 ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''25. Module Harvester:'''''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=929|Feature request 929]]'' Documentation needed for how we split jobs (incl. maybe additional splitting modularity) ||<style="TEXT-ALIGN: center">2 ||<style="TEXT-ALIGN: center">SVC ||<style="TEXT-ALIGN: center">JOLF ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Postponed ||
||<style="VERTICAL-ALIGN: top">'''26. Module Harvester:'''''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1774|Feature request 1774]]'' Stop using the JMS queues for queuing snapshot harvests ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">MSS ||<style="TEXT-ALIGN: center">SVC ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''27. Module harvester:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1963|Feature request 1963]]'' Make a !HarvestSchedulerApplication that runs the !HarvestScheduler ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">MSS ||<style="TEXT-ALIGN: center">JOLF ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''28. Module access:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1956|Feature request 1956]]'' Shared dependency links harvester and wayback modules ||<style="TEXT-ALIGN: center">0.5 ||<style="TEXT-ALIGN: center">SVC ||<style="VERTICAL-ALIGN: top">CSR ||<style="VERTICAL-ALIGN: top"> ||<style="VERTICAL-ALIGN: top"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">OK ||
Line 69: Line 69:
||<style="VERTICAL-ALIGN: top">'''28a. Module harvester: '''''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1956 Feature request 2011] ''Harvest documentation : generate an optional arcfiles-report.txt ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Nicolas ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">-''' ''' ||
||<style="VERTICAL-ALIGN: top">'''28b. Module harvester:''' ''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1956 Feature request 1969]'' Missing start and end date columns in Job status to search after ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Nicolas ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''29. Module Harvester''':''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1681 Feature request 1681]''''' '''Add seed to DB via webservice (via Browser Extension/Rich Client)''' ''' ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Andreas ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- ||
||<style="VERTICAL-ALIGN: top">'''30. Module Harvester:'''''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1682 Feature request 1682]'' Statistics (DB access, scripts, batch jobs ....) '''''' ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Andreas ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- '''''' ||
||<style="VERTICAL-ALIGN: top">'''31. Module Harvester''':''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1683 Feature request 1683]'' Util for regenerate admin.data file''' ''' ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Andreas ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- '''''' ||
||<style="VERTICAL-ALIGN: top">'''32. Module Harvester:'''''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1684 Feature request 1684]'' Activity when domain is to be crawled. One table for seed '''''' ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Andreas ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- ||
||<style="VERTICAL-ALIGN: top">'''33. Module Archive''':''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1743 Feature request 1743]'' When accessing Bitpreservation this takes really long time ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">JOLF ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- ||
||<style="VERTICAL-ALIGN: top">'''34. Module Harvester:'''''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1120 Feature request 1120]'' Crawlertrap info should be shareable between institutions '''''' ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Andreas ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">SVC will add comments to this FR. Might be an easy solution to share Crawlertraps by emailing files with crawler trap informations. ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Redundant (Copy of ??) ||
||<style="VERTICAL-ALIGN: top">'''35. Module Harvester''':''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1066 Feature request 1066]'' Show whether seed URL existed''' ''' ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Andreas ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- ||
||<style="VERTICAL-ALIGN: top">'''36. Module Archive:'''''[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1809 Feature request 1809]'' Write assignment for improving batchjob interface '''''' ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">JOLF ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">See FR 1881 ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- ||
||<style="VERTICAL-ALIGN: top">'''28a. Module harvester: '''''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1956|Feature request 2011]] ''Harvest documentation : generate an optional arcfiles-report.txt ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Nicolas ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">-''' ''' ||
||<style="VERTICAL-ALIGN: top">'''28b. Module harvester:''' ''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1956|Feature request 1969]]'' Missing start and end date columns in Job status to search after ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Nicolas ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''29. Module Harvester''':''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1681|Feature request 1681]]''''' '''Add seed to DB via webservice (via Browser Extension/Rich Client)''' ''' ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Andreas ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- ||
||<style="VERTICAL-ALIGN: top">'''30. Module Harvester:'''''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1682|Feature request 1682]]'' Statistics (DB access, scripts, batch jobs ....) '''''' ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Andreas ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- '''''' ||
||<style="VERTICAL-ALIGN: top">'''31. Module Harvester''':''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1683|Feature request 1683]]'' Util for regenerate admin.data file''' ''' ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Andreas ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- '''''' ||
||<style="VERTICAL-ALIGN: top">'''32. Module Harvester:'''''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1684|Feature request 1684]]'' Activity when domain is to be crawled. One table for seed '''''' ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Andreas ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- ||
||<style="VERTICAL-ALIGN: top">'''33. Module Archive''':''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1743|Feature request 1743]]'' When accessing Bitpreservation this takes really long time ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">JOLF ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- ||
||<style="VERTICAL-ALIGN: top">'''34. Module Harvester:'''''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1120|Feature request 1120]]'' Crawlertrap info should be shareable between institutions '''''' ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Andreas ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">SVC will add comments to this FR. Might be an easy solution to share Crawlertraps by emailing files with crawler trap informations. ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Redundant (Copy of ??) ||
||<style="VERTICAL-ALIGN: top">'''35. Module Harvester''':''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1066|Feature request 1066]]'' Show whether seed URL existed''' ''' ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">Andreas ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- ||
||<style="VERTICAL-ALIGN: top">'''36. Module Archive:'''''[[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1809|Feature request 1809]]'' Write assignment for improving batchjob interface '''''' ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">JOLF ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">See FR 1881 ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- ||
Line 83: Line 83:
||<style="VERTICAL-ALIGN: top">'''37. QA:''' [:QaTools:Assignment for enhanced QA tools] ||<style="TEXT-ALIGN: center">2 ||<style="TEXT-ALIGN: center">SVC ||<style="TEXT-ALIGN: center">CSR ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">High priority ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- ||
||<style="VERTICAL-ALIGN: top">'''38. WARC:''' Finalize [:AssignmentHarvester2:Assigment] for Harvester for support of WARC format ||<style="TEXT-ALIGN: center">? ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''39. Archive: '''Implement [:AssignmentGroupB2:Assignment B.2.3] - Use segments in bitarchives ||<style="TEXT-ALIGN: center">6 ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''40. Archive: '''Implement [:AssignmentGroupB2:Assignment B.2.4] - Write !BitPreservation scheduler ||<style="TEXT-ALIGN: center">5 ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''41. Archive: '''Implement [:AssignmentGroupB2:Assignment B.2.5] - Write !BitPreservation webinterface ||<style="TEXT-ALIGN: center">6 ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''37. QA:''' [[QaTools|Assignment for enhanced QA tools]] ||<style="TEXT-ALIGN: center">2 ||<style="TEXT-ALIGN: center">SVC ||<style="TEXT-ALIGN: center">CSR ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">High priority ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- ||
||<style="VERTICAL-ALIGN: top">'''38. WARC:''' Finalize [[AssignmentHarvester2|Assigment]] for Harvester for support of WARC format ||<style="TEXT-ALIGN: center">? ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''39. Archive: '''Implement [[AssignmentGroupB2|Assignment B.2.3]] - Use segments in bitarchives ||<style="TEXT-ALIGN: center">6 ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''40. Archive: '''Implement [[AssignmentGroupB2|Assignment B.2.4]] - Write !BitPreservation scheduler ||<style="TEXT-ALIGN: center">5 ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
||<style="VERTICAL-ALIGN: top">'''41. Archive: '''Implement [[AssignmentGroupB2|Assignment B.2.5]] - Write !BitPreservation webinterface ||<style="TEXT-ALIGN: center">6 ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||
Line 90: Line 90:
||<bgcolor="#cccccc" style="TEXT-ALIGN: center">[http://netarkivet.dk/netarkivet/index.php?title=Kendte_problemer Crawl-problems] (Netarchive.dk) . '''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Total x '''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- '''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- '''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Total x '''''''''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- '''''' || ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">[[http://netarkivet.dk/netarkivet/index.php?title=Kendte_problemer|Crawl-problems]] (Netarchive.dk) . '''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Total x '''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- '''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- '''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Total x '''''''''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- '''''' ||
Line 92: Line 92:
||<style="VERTICAL-ALIGN: top">42. Support to [http://netarkivet.dk/netarkivet/index.php?title=Kendte_problemer Crawl-problems] ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">CSR ||<style="TEXT-ALIGN: center">SVC ||<style="TEXT-ALIGN: center">1 ||<style="TEXT-ALIGN: center">High priority ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">.. '''''' ||
||<style="VERTICAL-ALIGN: top">43. [https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1881 Feature Request 1881] Quality assurance through a batchjob interface ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">JOLF ||<style="TEXT-ALIGN: center">MSS ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">High priority ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''''' ||
||<style="VERTICAL-ALIGN: top">42. Support to [[http://netarkivet.dk/netarkivet/index.php?title=Kendte_problemer|Crawl-problems]] ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">CSR ||<style="TEXT-ALIGN: center">SVC ||<style="TEXT-ALIGN: center">1 ||<style="TEXT-ALIGN: center">High priority ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">.. '''''' ||
||<style="VERTICAL-ALIGN: top">43. [[https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=1881|Feature Request 1881]] Quality assurance through a batchjob interface ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">JOLF ||<style="TEXT-ALIGN: center">MSS ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">High priority ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">'''''' ||
Line 102: Line 102:
||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Converting old Web collections to Netarchive.dk. See [http://udvikling.kb.dk/cvsshadow/digiliv/ProjektDokumenter/omkostninger%20ved%20indsamling%20af%20gammelt%20materiale-3.doc proposal]. These task will be independent of !NetarchiveSuite code-freeze. '''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Total x '''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- '''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- '''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Total x '''''''''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- '''''' || ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Converting old Web collections to Netarchive.dk. See [[http://udvikling.kb.dk/cvsshadow/digiliv/ProjektDokumenter/omkostninger%20ved%20indsamling%20af%20gammelt%20materiale-3.doc|proposal]]. These task will be independent of !NetarchiveSuite code-freeze. '''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Total x '''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- '''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- '''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">Total x '''''''''''' ||<bgcolor="#cccccc" style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- '''''' ||
Line 115: Line 115:
||<style="VERTICAL-ALIGN: top">53. Prepare [http://netarchive.dk/suite/Iteration44Releasetest release test] ||<style="TEXT-ALIGN: center">6 ||<style="TEXT-ALIGN: center">TLR ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">6 ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- || ||<style="VERTICAL-ALIGN: top">53. Prepare [[http://netarchive.dk/suite/Iteration44Releasetest|release test]] ||<style="TEXT-ALIGN: center">6 ||<style="TEXT-ALIGN: center">TLR ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">6 ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- ||
Line 119: Line 119:
||<style="VERTICAL-ALIGN: top">54. Execute [http://netarchive.dk/suite/Iteration44Releasetest release test]. ||<style="TEXT-ALIGN: center">12 ||<style="TEXT-ALIGN: center">TLR ||<style="TEXT-ALIGN: center">All ||<style="TEXT-ALIGN: center">12 ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- || ||<style="VERTICAL-ALIGN: top">54. Execute [[http://netarchive.dk/suite/Iteration44Releasetest|release test]]. ||<style="TEXT-ALIGN: center">12 ||<style="TEXT-ALIGN: center">TLR ||<style="TEXT-ALIGN: center">All ||<style="TEXT-ALIGN: center">12 ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">- ||
Line 126: Line 126:
||<style="VERTICAL-ALIGN: top">57. Define goals for [http://netarchive.dk/suite/Iteration45TaskList Iteration 45 task list] ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">CHH ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">.. '''''' || ||<style="VERTICAL-ALIGN: top">57. Define goals for [[http://netarchive.dk/suite/Iteration45TaskList|Iteration 45 task list]] ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center">CHH ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<style="TEXT-ALIGN: center"> ||<bgcolor="#cccccc" style="TEXT-ALIGN: center">.. '''''' ||

Task list and timetable for iteration 44

Status

OK/Not Ok

1. Highlights approved

OK

2. Assignment of tasks

OK

3. Task list and time table approved

OK

4. Implementation phase started

OK

5. Release test phase started

6. Assignment phase for next iteration started

7. Iteration 44 completed

Highlights for Iteration

  • Implement high priority FR from BNF, ONB and Netarkivet
  • Support of released NetarchiveSuite (http://netarchive.dk/suite).

  • Fix prioritized bugs
  • Support of Wayback in the Netarchive.dk production site. See List of tasks and Assignment for Wayback Integration

  • Migration of old Web materials to Netarchive.dk
  • Iteration 44 is planned as a development release.

Development procedure

Table of tasks

Tasks for iteration 43. Updated 30. July 2010

Estimate md

Main responsible

Reviewer

Remaining md at 30. July 2010

Comments

Status

Implementation phase (task x-n)

Open Source release + bugs and feature request

-

-

-

Support of Open Source Release

1. Support of released NetarchiveSuite

2

All (Google calender)

2

Ongoing

2. Implement translateprocess. Adjustment to Open Source partners.

1

CSR

SVC

Ongoing

3. Maintain French Translation files.

1

Nicolas/Sara

SVC

Ongoing

4. Maintain Italian and german Translation files.

1

Andreas

SVC

Ongoing

Bugs and Feature requests

Prioritized bugs according to list of priority 4 and priority 3 tasks.

-

-

SubTotal 0

..

-

Priority 5 bug

5 Module monitor?: Bug 1895 Running checksum gives Garbage Collector OutOfMemoryError and schedule stops

1

SVC

CSR

Max heap for Bitarchive monitors raised to 1936MB in prod. Awaiting results of observations after upgrade of java in production environment and installation of version 3.12.1

Priority 4 bugs

6. Module test Bug 1739 Missing information in System State GUI

TLR

CSR

.

OK

7. Module harvester: Bug 2001 No results when startdate equals enddate on page harveststatus-alljobs.jsp

Nicolas

SVC

Fixed. Awaiting review

8. Module Archive: Bug 1962 Batchjob fails when ArcRepository is overloaded

JOLF

CSR

Awaiting results of observations after upgrade to 3.12.1

Need Info

9. Module Common: Bug 1849 Invalid javadoc for DomainUtils#DOMAINNAME_CHAR_REGEX_STRING

CSR

SVC

Documentation issue.

10. Module harvester: Bug 1961 Possible to store invalid set of crawlertraps to domain causing the domain to be unreadable

1

SVC

MSS

.

Fixed. Awaiting review

11. Module harvester: Bug 1972 Resubmit selected failed job does not do anything and gives a webpage error

Nicolas

SVC

11b. Module Access: Bug 1980 Wayback deploy script does not rebuild wayback.war

1

CSR

Test script used as template for prod deploy

Fixed (Review not needed)

11c. Module Common: Bug 2017 Changes in default configuration

??

11d. Module Harvester : Bug 2016 Still missing Heritrix reports

Nicolas

11e. Module Monitor : Bug 2018 Scheduling is not always working

Nicolas

Priority 3 bugs

12. Module harvester: Bug 688 hosts-report should be IDNA decoded when writing harvestInfo to the DB

-

13. Module Access: Bug 823 No index = Internal server error

14. Module Monitor: Bug 1756 JMX status page does not update when a new application is started on previously used JMX port

15. Module Archive: Bug 1782 Same datetime repeated many times, while logging batch checksum of files

16. Module Documentation: Bug 1779 Improve documentation of the additional tools

Task 16 was fixed by JOLF in iteration 43

OK

17. Module Archive: Bug 1708 bitpreservation logic offers "add to archive" for file that is not in either location

18. Module Archive: Bug 1193 Exceptions from FileBatchJob stop batch job processing

19. Module Archive: Bug 1260 Too much and wrong feedback information on "Missing pages"

20. Module Monitor: Bug 1205 Security policy for unit tests contains hardcoded path to development environment

20a. Module Havester: Bug 1851 Close connexions with Heritrix on unfinished jobs

??

20b. Module Harvester: Bug 2019 Select of Harvest name may not work

Nicolas

..

Prioritized Feature Requests according to list of priority 4 and priority 3 tasks

-

-

SubTotal 21

-

Priority 4 Feature request

23. Module Harvester: Feature request 1928 Ability to easily resubmit a selection of failed jobs

Nicolas/Sara

CSR

Implemented. See outstanding bug 1972

Fixed. Awaiting review

24. Module Harvester:Feature request 1813 An extra resubmit button to make it visible which jobs have already been handled

MSS

JOLF

Depends on whether feature still required after implementing FR 1928

25. Module Harvester:Feature request 929 Documentation needed for how we split jobs (incl. maybe additional splitting modularity)

2

SVC

JOLF

Postponed

26. Module Harvester:Feature request 1774 Stop using the JMS queues for queuing snapshot harvests

MSS

SVC

27. Module harvester: Feature request 1963 Make a HarvestSchedulerApplication that runs the HarvestScheduler

MSS

JOLF

28. Module access: Feature request 1956 Shared dependency links harvester and wayback modules

0.5

SVC

CSR

OK

Priority 3 Feature request

28a. Module harvester: Feature request 2011 Harvest documentation : generate an optional arcfiles-report.txt

Nicolas

-

28b. Module harvester: Feature request 1969 Missing start and end date columns in Job status to search after

Nicolas

29. Module Harvester:Feature request 1681 Add seed to DB via webservice (via Browser Extension/Rich Client)

Andreas

-

30. Module Harvester:Feature request 1682 Statistics (DB access, scripts, batch jobs ....)

Andreas

-

31. Module Harvester:Feature request 1683 Util for regenerate admin.data file

Andreas

-

32. Module Harvester:Feature request 1684 Activity when domain is to be crawled. One table for seed

Andreas

-

33. Module Archive:Feature request 1743 When accessing Bitpreservation this takes really long time

JOLF

-

34. Module Harvester:Feature request 1120 Crawlertrap info should be shareable between institutions

Andreas

SVC will add comments to this FR. Might be an easy solution to share Crawlertraps by emailing files with crawler trap informations.

Redundant (Copy of ??)

35. Module Harvester:Feature request 1066 Show whether seed URL existed

Andreas

-

36. Module Archive:Feature request 1809 Write assignment for improving batchjob interface

JOLF

See FR 1881

-

Roadmap tasks

Total 52?

-

-

Total 8,5

-

Tasks from ...

37. QA: Assignment for enhanced QA tools

2

SVC

CSR

High priority

-

38. WARC: Finalize Assigment for Harvester for support of WARC format

?

39. Archive: Implement Assignment B.2.3 - Use segments in bitarchives

6

40. Archive: Implement Assignment B.2.4 - Write BitPreservation scheduler

5

41. Archive: Implement Assignment B.2.5 - Write BitPreservation webinterface

6

-

..

Crawl-problems (Netarchive.dk) .

Total x

-

-

Total x

-

Focus on following crawl-problems

42. Support to Crawl-problems

CSR

SVC

1

High priority

..

43. Feature Request 1881 Quality assurance through a batchjob interface

JOLF

MSS

High priority

Wayback/Nutchwax tasks independent of NetarchiveSuite code-freeze.

Total x

-

-

Total x

-

Tasks from ...

44. Wayback: Implement Index Aggregator

4

MSS

JOLF

High Priority

Done

45.Wayback: Documentation of Indexer/Aggregator

2

CSR

JOLF

1

High priority

-

46. Wayback deploy: Complete Wayback deploy

2

JOLF

CSR

2

High priority

-

..

Converting old Web collections to Netarchive.dk. See proposal. These task will be independent of NetarchiveSuite code-freeze.

Total x

-

-

Total x

-

Tasks from ...

47. Old Web collection: Old KB Webarchive

SVC

JOLF

High priority

-

48. Old Web collection: Old Webarchive from Niels Brugger collected by HTTrack

MSS

SVC

High priority

-

49. Old Web collection: Prepare ingest of extracted data from Internet Archive into Netarkivet.dk

SVC

MSS

High priority

-

50. Old Web collection: Ingest received data from Internet Archive into Netarkivet.dk

CLO

SVC

-

Common tasks calculated as implementation tasks

Total x

-

-

Total x

-

Others

Total x

-

-

SubTotal 2

-

51. Upgrade: New KB-PROD-UDV

5

SVC

TLR

High priority

OK

52. Batch: Create/execute a batch test script specified by 1 or 2 researchers

2

JOLF

TLR

High priority

Postponed

..

..

Prepare release test

Total x

-

-

SubTotal 12

-

53. Prepare release test

6

TLR

6

-

Available man-days for implementation phase

Total x

-

-

Total x

-

Release test phase (task ...)

Release test

Total x

-

-

Total 12

-

54. Execute release test.

12

TLR

All

12

-

'

..

Release notes

Total x

-

-

Total 0,5

-

55. Write release note

0,5

SVC

-

Available man-days for release test phase

Total x

-

-

Total 10

-

Assignment phase for next iteration (task ...)

56. Component bug/feature fix/management

QA

..

57. Define goals for Iteration 45 task list

CHH

..

58. Presentation of goals and tasks for Iteration 45. Achieve a common understanding of the purpose of the iteration and each task on status meeting

SVC

..

59. Assignment of tasks, bugs and feature request

QA

..

60. Update release test procedure

TLR

..

Available man-days for assigment phase

Total x

-

-

Total 22

-

Timetable

Timetable iteration 44. Updated 30. July 2010

Start time

End time

Responsible

Baseline 9. June 2010. Start time

Baseline 9. June 2010. End time

1. Implementation of decided tasks

14. June 2010

30. August 2010

14. June 2010

23. August 2010

2. Code freeze. Create the build for release test and notify when build is ready

31. August 2010

SVC

24. August

3. Release test

31. August 2010

3. September 2010

TLR

24. August

24. August

4. Code unfreeze

6. September 2010

SVC

27. August

5. Assignments, bug components and bug fixes

2. September 2010

3. September 2010

25. August

26. August

Iteration44TaskList (last edited 2010-09-09 09:58:45 by ChristenHedegaard)