CycleTLS
mimic
| CycleTLS | mimic | |
|---|---|---|
| 3 | 1 | |
| 795 | 73 | |
| - | - | |
| 6.4 | 3.5 | |
| 21 days ago | 4 months ago | |
| Go | Go | |
| GNU General Public License v3.0 only | GNU General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
CycleTLS
-
Is it possible to scrape a website protected by Cloudflare?
A lot of websites nowadays add fingerprint checking. So even if you fake the headers - it wouldn't help, as cloudflare still knows you are making request from go / python / whatever. So there is a lib to spoof fingerptint - https://github.com/Danny-Dasilva/CycleTLS . It may work for you .
- Curl’s TLS Fingerprint
-
Stack under attack: what we learned about handling DDoS attacks
While it can still be spoofed using several packages , e.g. https://github.com/Danny-Dasilva/CycleTLS it can still provide a meaningful/easy-to-manipulate signal.
Moreover, most bots conducting L7 DDoS don't use real/headless browsers in order to be able to scale their attack, so it's highly likely they'll have a discriminating/inconsistent TLS fingerprint.
This can also be done directly in Fastly using e.g. https://developer.fastly.com/reference/vcl/variables/client-...
Another approach to proactively flag malicious IPs is to scrape free proxies. Indeed, most DDoS leverage lot of cheap/know bad IPs. It's frequent to see these free proxies in these attacks.
mimic
What are some alternatives?
gost - GO Simple Tunnel - a simple tunnel written in golang
nico - A HTTP3 web server for reverse proxy and single page application, automatically apply for ssl certificate, Zero-Configuration.
go-cloudflare-scraper - A golang http.Transport layer that uses Otto to solve Cloudflare challenges
sprint - SSL Fingerprint Tool written in Go for the CLI and package use.
chromedp - A faster, simpler way to drive browsers supporting the Chrome DevTools Protocol.
rosso - Data parsers and formatters
Ponzu - Headless CMS with automatic JSON API. Featuring auto-HTTPS from Let's Encrypt, HTTP/2 Server Push, and flexible server framework written in Go.
surf - Stateful programmatic web browsing in Go.
colly - Elegant Scraper and Crawler Framework for Golang
phantomgo - a headless browser phantomjs for golang