Plain Text Protocols

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • SonarLint - Clean code begins in your IDE with SonarLint
  • Scout APM - Truly a developer’s best friend
  • InfluxDB - Build time-series-based applications quickly and at scale.
  • talent.io - Download talent.io’s Tech Salary Report
  • gopher-plus

    Gopher+ protocol specification

  • HTTP Parser

    http request/response parser for c

    Legacy HTTP/1.1 suffers a few issues, see the current RFC errata:

    https://www.rfc-editor.org/errata_search.php?rfc=7230&rec_st...

    There are issues particularly around how whitespace and obsolete line folding should be handled

    https://github.com/nodejs/http-parser/issues?q=is%3Aissue+wh...

    https://github.com/httpwg/http-core/issues/53

    It's not as trivial as a few string splits.

  • SonarLint

    Clean code begins in your IDE with SonarLint. Up your coding game and discover issues early. SonarLint is a free plugin that helps you find & fix bugs and security issues from the moment you start writing code. Install from your favorite IDE marketplace today.

  • cantordust

    Public repository for Cantordust Ghidra plugin.

    Looks like there's even been a recently open sourced plugin for Ghidra released by Battelle:

      https://github.com/Battelle/cantordust

  • restruct

    Rich binary (de)serialization library for Golang

    Honestly, I dislike plaintext formats a lot. It is more accessible because it’s human readable. But, this only extends to humans who happen to speak the language the protocol uses for keywords. While it’s not a huge ask, I still suggest this is mostly not that interesting of a benefit.

    Parsing and emitting plaintext formats, meanwhile, is a rabbit hole. It’s human readable which makes you tempted to make it human writable. Should you accept extraneous whitespace? Tabs vs spaces? Terminating new line? Unix or DOS line endings? Etc.

    Binary data may seem less accessible, but I blame the libraries. There’s tons of easy ways to parse text. You can use string.split, atoi and scanf in your language of choice. What is there for binary?

    In Go, the encoding/binary package actually implements something really cool. A simple reflection-based mechanism that can read and write binary data into a structure in a defined and simple way.

    lunixbochs extended this to struc[1], which adds additional tags for advanced reading and writing of binary structures, including variable length structures. I went further and maybe a bit off into the deep end with Restruct[2], a similar concept but with a lot more features, designed specifically so I could handle advanced structures quickly.

    The end result is that I can define some Go structs with integers, strings, byte arrays and corresponding tags, and be able to serialize and deserialize from those structures to their corresponding binary representation. For an overdone demo of what you could do with Restruct for example, see this (incomplete) PNG demo: https://github.com/go-restruct/restruct/blob/master/formats/... (It is mainly incomplete because I had moved focus to develop a codegen for restruct, to improve runtime performance, although such work has since stalled.)

    [1]: https://pkg.go.dev/github.com/lunixbochs/struc

    [2]: https://pkg.go.dev/github.com/go-restruct/restruct

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts