All items in this manuscript group were transferred to the University Libraries, M.E. Grenander Department of Special Collections and Archives.
crawl: 249672
Crawl Rules
Ignore Robots.txt for www.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for www.alumni.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for www.ualbanysports.com (last updated 2016-02-11)
Ignore Robots.txt for library.albany.edu (last updated 2017-05-19)
Ignore Robots.txt for alumni.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for asrc.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for atmos.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for bioinformatics.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for cela.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for choose.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for cs.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for csda.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for imls.ctg.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for www.ctg.albany.edu (last updated 2017-05-19)
Ignore Robots.txt for cwig.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for events.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for hr.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for ibl.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for illiad.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for liblogs.albany.edu (last updated 2017-05-19)
Ignore Robots.txt for libguides.library.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for scholarsarchive.library.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for listserv.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for m.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for math.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for omega.math.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for mumford.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for nyjm.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for pdp.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for resnet.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for rit.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for cyberphysics.rit.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for rna.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for slsc.albany.edu (last updated 2017-05-19)
Ignore Robots.txt for uaems.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for uapps.albany.edu (last updated 2016-02-11)
Ignore Robots.txt for wiki.albany.edu (last updated 2016-02-11)
Block host dev.library.albany.edu
Crawl Times
start_date: 2016-11-17T16:57:23Z
original_start_date: 2016-11-17T16:57:23Z
last_resumption: None
processing_end_date: 2016-11-22T23:26:30Z
end_date: 2016-11-22T18:15:59Z
elapsed_ms: 436686458
Crawl Types
type: MONTHLY
recurrence_type: MONTHLY
pdfs_only: False
test: False
Crawl Limits
time_limit: 432000
document_limit: None
byte_limit: None
crawl_stop_requested: None
Crawl Results
status: FINISHED_TIME_LIMIT
discovered_count: 6455397
novel_count: 904107
duplicate_count: 647491
resumption_count: 0
queued_count: 4903799
downloaded_count: 1551598
download_failures: 287
warc_revisit_count: 647445
warc_url_count: 1551482
total_data_in_kbs: 234676600
duplicate_bytes: 166348006398
warc_compressed_bytes: 22645478722
Crawl Technical Details
doc_rate: 3.55
kb_rate: 537.0