restruct

Rich binary (de)serialization library for Golang (by go-restruct)

Restruct Alternatives

Similar projects and alternatives to restruct

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better restruct alternative or higher similarity.

restruct reviews and mentions

Posts with mentions or reviews of restruct. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2022-03-25.
  • Why isn't there a Swagger/OpenAPI for binary formats?
    13 projects | news.ycombinator.com | 25 Mar 2022
    My project Restruct[1] does Kaitai-like things but also supports serialization. Unfortunately, it only supports Go and only deals with Go struct tags rather than YAML manifests. Still, it totally can be used for serialization. I use it to sketch out quick projects against arbitrary binary formats. Two examples: one, parsing PNG headers to implement a quick binwalk-like program for just PNG that looks for the IEND chunk to extract accurately[2], two, a program that splits FL Studio FLP projects by playlist track[3].

    I feel like I’ve self-promoted Restruct like four times on Hacker News, and I feel kind of bad because it could use improvements and even some bug fixes and I never seem to get around to it. Oh well. It’s still useful for me, I hope it’s useful for others, too.

    That said, Kaitai has a fairly clear path towards adding serialization from a design PoV; many things that would be calculated for parsing structures in deserialization could just become checks/assertions in serialization. As an example, checking that an expression calculates out to the expected value would be a reasonable approach. Reversible expressions could be implemented for some cases, too, if you want it to do more of the heavy lifting. I think the biggest obstacle is actually implementing it, and frankly my Scala is too weak to help with such a relatively big undertaking.

    I’ve also played with the rust nom library, which implements functional programming style parser combinators. It is quite cool how it can express fairly complex grammars and binary formats pretty much equally well, albeit optimizing it effectively requires serious magic that I do not think nom has. (I assume in Haskell, the same thing can be done with mind-boggling optimization power.)

    [1]: https://github.com/go-restruct/restruct

    [2]: https://github.com/jchv/pngextract

    [3]: https://github.com/jchv/flsplit

  • Kaitai Struct: A new way to develop parsers for binary structures
    12 projects | news.ycombinator.com | 17 Mar 2022
    I’m a big fan of Kaitai Struct, to the point where I’ve even contributed a small bit of improvements to its Go support, and I use it in a handful of small projects. It’s indispensable for spelunking blobs of binary data.

    I’ve also taken some inspiration with a Go library I wrote, restruct:

    https://github.com/go-restruct/restruct

    … which is a bit like Go’s JSON encoding/decoding library, but with kaitai-like annotations for binary encoding. (Check the PNG example to see some of what can be done with it.)

  • Plain Text Protocols
    4 projects | news.ycombinator.com | 25 Feb 2021
    Honestly, I dislike plaintext formats a lot. It is more accessible because it’s human readable. But, this only extends to humans who happen to speak the language the protocol uses for keywords. While it’s not a huge ask, I still suggest this is mostly not that interesting of a benefit.

    Parsing and emitting plaintext formats, meanwhile, is a rabbit hole. It’s human readable which makes you tempted to make it human writable. Should you accept extraneous whitespace? Tabs vs spaces? Terminating new line? Unix or DOS line endings? Etc.

    Binary data may seem less accessible, but I blame the libraries. There’s tons of easy ways to parse text. You can use string.split, atoi and scanf in your language of choice. What is there for binary?

    In Go, the encoding/binary package actually implements something really cool. A simple reflection-based mechanism that can read and write binary data into a structure in a defined and simple way.

    lunixbochs extended this to struc[1], which adds additional tags for advanced reading and writing of binary structures, including variable length structures. I went further and maybe a bit off into the deep end with Restruct[2], a similar concept but with a lot more features, designed specifically so I could handle advanced structures quickly.

    The end result is that I can define some Go structs with integers, strings, byte arrays and corresponding tags, and be able to serialize and deserialize from those structures to their corresponding binary representation. For an overdone demo of what you could do with Restruct for example, see this (incomplete) PNG demo: https://github.com/go-restruct/restruct/blob/master/formats/... (It is mainly incomplete because I had moved focus to develop a codegen for restruct, to improve runtime performance, although such work has since stalled.)

    [1]: https://pkg.go.dev/github.com/lunixbochs/struc

    [2]: https://pkg.go.dev/github.com/go-restruct/restruct

  • A note from our sponsor - InfluxDB
    www.influxdata.com | 26 Nov 2022
    InfluxDB is the Time Series Data Platform where developers build real-time applications for analytics, IoT and cloud-native services in less time with less code. Learn more →

Stats

Basic restruct repo stats
3
326
3.2
8 months ago

go-restruct/restruct is an open source project licensed under ISC License which is an OSI approved license.

Clean code begins in your IDE with SonarLint
Up your coding game and discover issues early. SonarLint is a free plugin that helps you find & fix bugs and security issues from the moment you start writing code. Install from your favorite IDE marketplace today.
www.sonarlint.org