Preliminary Agenda items (proposals) for the non-technical workshop

Introduction

Presentation of participants

Expectations

Review/update of agenda

Step by step experience from Netarchive.dk concerning broad crawls

Preparation

• scheduling
• responsabilities, roles of participants in a broad crawl defining crawl target (number of URL, scope, seed lists, politeness,budget...)
• dealing with junk data
• sorting and spliting seed lists into different jobs running test crawl

Selection of sites

How to manage deduplication

Actual impact on computing and storage

Experience during the crawl

• using frontier reports
• modifying settings, creating overrides

QA

• visual QA
• running a patch crawl

Metrics from the past domain crawls : how much, how many, how fast, etc.

Collection

What's a collection?

User management NetarchiveSuite

Different set of roles using the NetarchiveSuite

A simple user interface for people who are not very familiar with webarchiving.

Statistics module

Base for all kinds of calculations and general information about the webarchive

Comparing results of crawls for quality control.

Access

Comparison of legal basis regarding access

Access with Wayback