6. I am having problems monitoring certain pages: After running a crawling session, the following error message is displayed: "A connection error has caused the crawl to be cancelled". And the Log shows the message: "download failed (Read Timeout) Link found at URL... ". What should I do?
"Read Timeout" means that the maximum time allowed for a download has been reached. Some pages may take a long time to download (large size, distant server too slow or unavailable). It is therefore necessary to define the maximum downloading time after which the "Read Timeout" error should occur. This setting can be found in KB Crawl's "General Options" on the first tab. The recommended value is 60 seconds although more time may be needed in some cases.
Looking for a website crawler as well as strategic watch software? Discover the functionalities and
architecture of KB Crawl , the specialist editor in competitive intelligence.
© 2008 KB Crawl - All rights reserved
Privacy Policy