C# Crawler

Open-source C# projects categorized as Crawler

Top 6 C# Crawler Projects

  • abot

    Cross Platform C# web crawler framework built for speed and flexibility. Please star this project! +1.

    Project mention: Can you build a web crawler in c#? | reddit.com/r/learnprogramming | 2022-04-11

    This can be done perfectly well in c#, https://github.com/sjdirect/abot for example.

  • TumblThree

    A Tumblr, Twitter and newTumbl Blog Backup Application

    Project mention: Twitter archive not working? | reddit.com/r/Twitter | 2022-11-18

    There's a github project called TumblThree that can pull tweets and media that have been publicly posted to any account you give it. It can't do private accounts or DMs and it doesn't pull quote tweets, but it gives you a backup of everything publicly available.

  • SonarLint

    Clean code begins in your IDE with SonarLint. Up your coding game and discover issues early. SonarLint is a free plugin that helps you find & fix bugs and security issues from the moment you start writing code. Install from your favorite IDE marketplace today.

  • Infinity Crawler

    A simple but powerful web crawler library for .NET

  • custom-crawler

    🌌 High productivity semi-automatic crawler generator 🛠️🧰

    Project mention: I made a powerful crawler creation tool on c#! | reddit.com/r/csharp | 2022-11-10
  • Spidey

    A multi threaded web crawler library that is generic enough to allow different engines to be swapped in.

    Project mention: I need data from a website. It is viable to create an API that scrapes the website and returns the data on an endpoint? | reddit.com/r/dotnet | 2022-12-20

    Didn't get a chance to reply earlier but depending on what you're trying to do, you might want a web crawler. I have a crawler on Github that I built for scraping in instances where someone doesn't have an API. If you go this route, I suggest doing it as a background task and go off cached data.

  • Scraper

    A simplistic web scraper which aims to make scrapping webpages and crawling links faster by utilizing parallel threading. (by 0x78f1935)

    Project mention: CLI Scraper | reddit.com/r/csharp | 2022-11-27
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2022-12-20.

C# Crawler related posts


What are some of the best open-source Crawler projects in C#? This list will help you:

Project Stars
1 abot 2,072
2 TumblThree 477
3 Infinity Crawler 202
4 custom-crawler 54
5 Spidey 11
6 Scraper 1
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives