NetarchiveSuiteWe use a patched version of the 0.3.0-20061218 beta version of the deduplicator . The patch fixes the following issues in the NetarchiveSuite:
[https://gforge.statsbiblioteket.dk/tracker/?group_id=7&atid=105&func=detail&aid=1062: Bug 1062] Indexserver skips a lot of lines due to threading problem with SimpeDateFormat
[attachment:deduplicator-0.3.0-20061218a.diff Patch against Deduplicator 0.3.0-20061218]
[attachment:Deduplicator-0.3.0-20061218-src.zip Patched sourcecode Deduplicator 0.3.0-20061218-src.zip]
Note that the deduplicator must be compiled with the same version of heritrix as the NetarchiveSuite uses, or the deduplicator will fail to work during runtime.