a distributed crawl about using celery, scrapy, mongodb and redis
# install redis
# install mongodb
# pip install -r requirements
# cp supervisord.conf /etc/supervisord.conf
# supervisord -c /etc/supervisord.conf
fdrong/scrapy_celery
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|