C# robots-txt

Open-source C# projects categorized as robots-txt

C# robots-txt Projects

  • Infinity Crawler

    A simple but powerful web crawler library for .NET

  • Robots Exclusion Tools

    A "robots.txt" parsing and querying library for .NET

  • InfluxDB

    Access the most powerful time series database as a service. Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).


Project Stars
1 Infinity Crawler 203
2 Robots Exclusion Tools 19
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives