Use Scrapy to crawl a website and download some images.
- Choose user agent randomly when crawling
- Use "referer" to avoid being blocked
- Save the images in different folds according to the category information in the urls
$scrapy crawl sexy
| Name | Name | Last commit date | ||
|---|---|---|---|---|