spider is an OD crawler that crawls through opendirectories and indexes the urls
Why do you think that https://github.com/grantjenks/python-diskcache is a good alternative to spider
spider is an OD crawler that crawls through opendirectories and indexes the urls
Why do you think that https://github.com/grantjenks/python-diskcache is a good alternative to spider