Django custom management command running Scrapy: How to include Scrapy's options?

This page summarizes the projects mentioned and recommended in the original post on reddit.com/r/codehunter

Our great sponsors
  • SonarQube - Static code analysis for 29 languages.
  • JetBrains - Developer Ecosystem Survey 2022
  • Scout APM - Less time debugging, more time building
  • Scrapy

    Scrapy, a fast high-level web crawling & scraping framework for Python.

    I want to be able to run the Scrapy web crawling framework from within Django. Scrapy itself only provides a command line tool scrapy to execute its commands, i.e. the tool was not intentionally written to be called from an external program.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts