We use a patched version of the 0.3.0-20061218 beta version of the deduplicator. The patches fixes the following issues in the NetarchiveSuite:

Downloads

  • [get | view] (2008-05-26 11:44:45, 2621.4 KB) [[attachment:deduplicator-0.3.0-20061218-patch-heritrix-1.12.1b.patch]]
  • [get | view] (2008-05-26 11:44:23, 0.7 KB) [[attachment:deduplicator-0.3.0-20061218-patch-index-NPE.patch]]
  • [get | view] (2008-05-26 11:44:30, 2.5 KB) [[attachment:deduplicator-0.3.0-20061218-patch-local-dateformat.patch]]
  • [get | view] (2008-05-27 08:54:28, 235.4 KB) [[attachment:deduplicator-0.3.0-20061218-patch-lucene-OutOfMemory-2.patch]]
  • [get | view] (2008-05-26 11:44:14, 24.0 KB) [[attachment:deduplicator-0.3.0-20061218-patch-lucene-OutOfMemory.patch]]
  • [get | view] (2008-05-26 11:44:38, 2648.3 KB) [[attachment:deduplicator-0.3.0-20061218-patched-20080522-cumulative.patch]]
  • [get | view] (2008-05-26 11:45:11, 0.9 KB) [[attachment:deduplicator-0.3.0-20061218-patched-20080522.patch]]
  • [get | view] (2008-05-27 08:54:35, 2859.6 KB) [[attachment:deduplicator-0.3.0-20061218-patched-20080527-cumulative.patch]]
  • [get | view] (2008-05-27 08:55:25, 0.9 KB) [[attachment:deduplicator-0.3.0-20061218-patched-20080527.patch]]
  • [get | view] (2008-05-26 11:43:51, 1929.8 KB) [[attachment:deduplicator-0.3.0-20061218-src.zip]]
 All files | Selected Files: delete move to page copy to page

Note that the deduplicator must be compiled with the same version of heritrix as the NetarchiveSuite uses, or the deduplicator will fail to work during runtime.

DeduplicatorPatches (last edited 2010-08-16 10:24:26 by localhost)