All items in this manuscript group were transferred to the University Libraries, M.E. Grenander Department of Special Collections and Archives.
crawl: 231044
Crawl Rules
Ignore Robots.txt for www.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for www.alumni.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for www.ualbanysports.com (last updated 2016-02-11)
Ignore Robots.txt for library.albany.edu (last updated 2017-05-19)
Ignore Robots.txt for alumni.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for asrc.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for atmos.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for bioinformatics.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for cela.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for choose.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for cs.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for csda.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for imls.ctg.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for www.ctg.albany.edu (last updated 2017-05-19)
Ignore Robots.txt for cwig.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for events.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for hr.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for ibl.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for illiad.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for liblogs.albany.edu (last updated 2017-05-19)
Ignore Robots.txt for libguides.library.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for scholarsarchive.library.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for listserv.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for m.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for math.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for omega.math.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for mumford.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for nyjm.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for pdp.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for resnet.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for rit.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for cyberphysics.rit.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for rna.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for slsc.albany.edu (last updated 2017-05-19)
Ignore Robots.txt for uaems.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for uapps.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for wiki.albany.edu (last updated 2016-02-11)
Block host dev.library.albany.edu
Crawl Times
start_date: 2016-08-17T16:58:06Z
original_start_date: 2016-08-17T16:58:06Z
last_resumption: None
processing_end_date: 2016-08-23T01:20:50Z
end_date: 2016-08-22T23:33:45Z
elapsed_ms: 455703830
Crawl Types
type: MONTHLY
recurrence_type: MONTHLY
pdfs_only: False
test: False
Crawl Limits
time_limit: 432000
document_limit: None
byte_limit: None
crawl_stop_requested: None
Crawl Results
status: FINISHED_TIME_LIMIT
discovered_count: 1189467
novel_count: 184079
duplicate_count: 736427
resumption_count: 0
queued_count: 268961
downloaded_count: 920506
download_failures: 267
warc_revisit_count: 736404
warc_url_count: 920384
total_data_in_kbs: 202254345
duplicate_bytes: 198029314926
warc_compressed_bytes: 397770039
Crawl Technical Details
doc_rate: 2.02
kb_rate: 443.0