Universal Access To All Knowledge
Home Donate | Forums | FAQs | Contributions | Terms, Privacy, & Copyright | Contact | Volunteer Positions | Jobs | Bios
Search: Advanced Search
Anonymous User (login or join us) Upload

Reply to this post | See parent post | Go Back
View Post [edit]

Poster: Administrator, Curator, or Staff Jeff Kaplan Date: August 08, 2011 02:44:10pm
Forum: web Subject: Re: Cannot find archives of a company because of recent acquisition

what is the url? if there is now a robots.txt in place that prevents archive.org from crawling the site then no pages from that site will be available.

Terms of Use (10 Mar 2001)