Reply to this post | See parent post | Go Back

View Post [edit]

Poster:	Jeff Kaplan	Date:	October 16, 2011 10:19:33pm
Forum:	web	Subject:	Re: We were unable to get the robots.txt document to display this page.

That domain appear to be registered through 12/12/2011 according to InterNIC whois. I am seeing that the remote host for the domain is not responding. Any captures we have will not be available until a robots.txt can be retrieved and it is not blocking crawling by the Wayback Machine.

Reply to this post

Reply [edit]

Poster:	Donovan K. Loucks	Date:	October 17, 2011 08:52:00am
Forum:	web	Subject:	Re: We were unable to get the robots.txt document to display this page.

Jeff,

The problem is that the co-hosting company no longer performs co-hosting:

http://dyn.com/everyeditdns-discontinued/

Does that mean the pages won't be able to be retrieved? The guy who owns the domain can no longer access the files (and apparently doesn't have backups of them) and would like to resurrect the site elsewhere.

Donovan

Reply to this post

Reply [edit]

Poster:	ladynred	Date:	October 22, 2011 09:21:59am
Forum:	web	Subject:	Re: We were unable to get the robots.txt document to display this page.

This error continues. I was able to successfully view archived sites earlier this week, and today when I try to check those same URLs, this robots.txt error comes up for ANY URL I put in.

I use the archive for research, is this something you can fix??

Reply to this post

Reply [edit]

Poster:	LadyCallie	Date:	October 30, 2011 06:57:14pm
Forum:	web	Subject:	Re: We were unable to get the robots.txt document to display this page.

I'm having the same problem while looking for a previously archived prodigy.net site.

Any idea if this will be fixed?