Reply to this post | See parent post | Go Back

View Post [edit]

Poster:	Jeff Kaplan	Date:	January 02, 2012 05:56:21pm
Forum:	web	Subject:	Re: How often does archive crawl a specific website?

there is no specific schedule. if it is well linked it will have a higher likelihood of being crawled more often.

Reply to this post

Reply [edit]

Poster:	bibble01	Date:	January 03, 2012 07:24:01pm
Forum:	web	Subject:	Re: How often does archive crawl a specific website?

Jeff, thanks for your reply. So even some news website constantly update the page, it doesn't mean that the crawler will catch the update, right?

Another question: why does it take 6 months to transfer the storage? It is a little long...

thanks,