Planning Go 1.20 Cryptography Work

• go

  2,068 119,564 10.0 Go

  The Go programming language

  

Something really important thats going under the radar is TLS fingerprinting [1].

Multiple servers are using this now, including some requests to subdomains on google.com, googleapis.com, CloudFlare and others. I keep reporting this [2][3], and no one seems to care. If a server blacklists your client, whether its cURL or Go "net/http", you can no longer request to that server using that client. Period. Any HTTP client that wants to be robust, should be thinking about this.

1. https://wikipedia.org/wiki/Device_fingerprint#Sources_of_ide...

2. https://github.com/golang/go/issues/48207

3. https://github.com/curl/curl/issues/8119

• libcurl

  300 34,167 9.9 C

  A command line tool and library for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, GOPHER, GOPHERS, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, MQTT, POP3, POP3S, RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET, TFTP, WS and WSS. libcurl offers a myriad of powerful features

  

Something really important thats going under the radar is TLS fingerprinting [1].

Multiple servers are using this now, including some requests to subdomains on google.com, googleapis.com, CloudFlare and others. I keep reporting this [2][3], and no one seems to care. If a server blacklists your client, whether its cURL or Go "net/http", you can no longer request to that server using that client. Period. Any HTTP client that wants to be robust, should be thinking about this.

1. https://wikipedia.org/wiki/Device_fingerprint#Sources_of_ide...

2. https://github.com/golang/go/issues/48207

3. https://github.com/curl/curl/issues/8119

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• utls-light

  1 23 2.9 Go

  Minimal version of utls for parrotting the TLS handshake of popular web browsers

  

This kind of flexibility is a non-goal of crypto/tls. We have a TLS stack with one of the best security track records because we implement an opinionated subset of the specification, amongst other things. Moreover, fingerprint evasion is a cat-and-mouse game we can't sustain in the six months Go release cycle.

That doesn't mean I don't care! I was just talking with a friend about this the other day, and I suggested it should be possible to make a small, easily maintained patch that focuses on chasing the fingerprint of one well-known browser. He implemented https://github.com/hellais/utls-light in that spirit, which looks like a viable solution to me.

Anyway, I think matching TLS fingerprints to HTTP User-Agent strings is a valid abuse prevention technique. Rejecting any non-browser fingerprint is bad, and websites should get pushback for that, but I am skeptical that's something they can reliably do without breaking any time Chrome flips a field study. TLS is not _that_ rusted shut.

• curl-impersonate

  31 3,319 7.1 Python

  curl-impersonate: A special build of curl that can impersonate Chrome & Firefox

Suggest a related project