Making the Webcrawler multithreaded should speed up the process of crawling through many different webpages, as the limiting factor in speed is usually the time it takes to connect to the webpage. Having multiple threads would allow the webcrawler to connect to many different sites simultaneously.
Making the Webcrawler multithreaded should speed up the process of crawling through many different webpages, as the limiting factor in speed is usually the time it takes to connect to the webpage. Having multiple threads would allow the webcrawler to connect to many different sites simultaneously.