The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns
Why do you think that https://github.com/internetarchive/warctools is a good alternative to grab-site
The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns
Why do you think that https://github.com/internetarchive/warctools is a good alternative to grab-site