WARC file for New York Civil Liberties Union (NYCLU) - American Civil Liberties Union of New York State, 2016 July 30

Acquisition information:

crawl: 228055

Crawl Rules

Limit host twitter.com to 1000 documents

Limit host upload.wikimedia.org to 1000 documents

Limit host en.wikipedia.org to 1000 documents

Crawl Times

start_date: 2016-07-30T19:45:59Z

original_start_date: 2016-07-30T19:45:59Z

last_resumption: None

processing_end_date: 2016-08-02T23:35:42Z

end_date: 2016-08-02T20:14:01Z

elapsed_ms: 260857522

Crawl Types

type: MONTHLY

recurrence_type: MONTHLY

pdfs_only: False

test: False

Crawl Limits

time_limit: 259200

document_limit: None

byte_limit: None

crawl_stop_requested: None

Crawl Results

status: FINISHED_TIME_LIMIT

discovered_count: 1172724

novel_count: 267392

duplicate_count: 13801

resumption_count: 0

queued_count: 891531

downloaded_count: 281193

download_failures: 10

warc_revisit_count: 13749

warc_url_count: 281154

total_data_in_kbs: 21925334

duplicate_bytes: 6569359411

warc_compressed_bytes: 842149127

Crawl Technical Details

doc_rate: 1.08

kb_rate: 84.0

Physical / technical requirements:
Researchers interested in data analysis with web archives may request a WARC file. WARC files are very large and difficult to work with. Your request may take time to process, and we may be unable to deliver your request remotely. Please consult an archivist if you are interested in advanced research with web archives.

Using these materials

Access:
The archives are open to the public and anyone is welcome to visit and view the collections.
Collection restrictions:
Access to this collection is restricted because it is unprocessed. Portions of the collection may contain recent administrative records and/or personally identifiable information. Please contact an archivist for more information. Certain restrictions may apply.
Collection terms of access:
The University Archives are eager to hear from any copyright owners who are not properly identified so that appropriate information may be provided in the future.

About the Archives

Access options

Ask an Archivist

Ask a question or schedule an individualized meeting to discuss archival materials and potential research needs.

Ask a Question Book a Meet

Schedule a Visit

Archival materials can be viewed in-person in our reading room. We recommend making an appointment to ensure materials are available when you arrive.

Schedule a Visit Hours

Make a Remote Request

We may also be able to deliver digital scans remotely for a fee.

Request Fees