ghidra VS hof

[1] https://github.com/boricj/ghidra/tree/feature/elfrelocatebleobjectexporter

I've been working on a specific reverse-engineering technique called _unlinking_ [1] on-and-off for the past 16 months or so. I'm on my third prototype (first a set of Ghidra scripts written in Jython [2], then a fork of Ghidra [3] and now a Ghidra extension [4]) and I've started a blog in order to document it [5], which side-tracked into writing a whole series of articles on reverse-engineering to introduce the topic.

What for, you may ask? Basically I'm trying to decompile a PlayStation 1 video game and I've quickly decided that dealing alone with multiple +500 KiB executables of complete utter spaghetti code wasn't going to work. Instead, I've decided that I'd rather divide-and-conquer the problem, so I've been tooling up to split executables into relocatable object files, in order to decompile those one at a time and _Ship of Theseus_-style my way to success.

Ironically, all of that stuff is so not done that I don't even know what meaningful feedback there could be. My prototypes do work, but only for 32 bit little endian statically-linked MIPS executables. The articles on my blog are draft-quality. As for the decompilation project itself that started all of this, it hasn't seen much progress due to all of those side-quests. The overall topic is so esoteric that so far I've only managed to hear about one group of two persons that tried to do anything remotely similar and one another anecdotal account [6] that this particular skill is very uncommon among reverse engineers.

Personally, I'm starting to think that maybe I could've actually reverse-engineered and decompiled the game in the time I took to get here. I've also tried to engage with Ghidra to upstream the foundations of my modifications in my fork, but after some back-and-forth it became clear that my prototype-grade stuff wasn't industrial-grade and couldn't be merged in its current state, which is why I'm currently reworking the code in my fork as a Ghidra extension.

To those that want to provide feedback after reading all of this: beware, I've had a lot of fun going down that rabbit hole, but this is one hell of a time sink _and_ a particularly tricky mind-bender.

[1] I don't actually _know_ what's the actual name for this technique, given that there are so few resources on it out there. I do know I didn't invent it.

[2] https://github.com/boricj/ghidra-unlinker-scripts

[3] https://github.com/boricj/ghidra/tree/feature/elfrelocateble...

[4] https://github.com/boricj/ghidra-unlinker-extension

[5] https://news.ycombinator.com/item?id=36575081#36590078

[6] https://news.ycombinator.com/item?id=35729232&p=3#35740761

- The relocation synthesizer for MIPS: https://github.com/boricj/ghidra/blob/feature/elfrelocateble...

  - The Ghidra analyzer that leverages this synthesizer: https://github.com/boricj/ghidra/blob/feature/elfrelocatebleobjectexporter/Ghidra/Features/Delinker/src/main/java/ghidra/app/analyzers/RelocationTableSynthesizerAnalyzer.java

For a auth product, you probably should go with traditional code generation, not LLMs (re: all the unsolvable problems they have (hallucinations, incomplete code answers). For an auth product, you want something definite and reproducible, or you are asking for trouble.

If you are interested in a flexible code gen framework, I built one that had this very use case in mind. (https://github.com/hofstadter-io/hof)

I'd be happy to think / talk this through with you

I recently built a TUI for exploring and developing CUE, JSON, Yaml.

Large inspiration for the overall UX comes from the Bloomberg terminal, where every function is reachable with four letter shortcut from the command box. It was really the command box that I liked. I've also built a sort of "flex" panel component for the layout so you can create as many panels as you like.

https://docs.hofstadter.io/getting-started/hof-tui/

https://github.com/hofstadter-io/hof/tree/_dev/lib/tui

Many business analysts use SQL, have for a long time. They are probably not your target audience. With the problem being JAVA specific, you'd likely want to start there

This sounds similar to the goals of my hof tool (https://github.com/hofstadter-io/hof), lift type definitions out of code so they can be defined in one place, then generate the code for all the places. Is that sounding like what you are after?

Having built something similar, the biggest challenge for users is that they have to use a bespoke language, like WASP here. I suspect that it is also your biggest challenge as well.

Mine is built on CUE, which at least has the potential to become a more widely used language. CUE hasn't reached sufficient maturity for broader adoption yet, so I continue to face this same problem.

https://github.com/hofstadter-io/hof

Here's a big one I needed to get ChatGPT to do something more sophisticated with a JSON object response (predates functions and all that)

https://github.com/hofstadter-io/hof/blob/_dev/flow/chat/pro...

It no longer worked after a model update some time ago, haven't tried recently.

I found codellama to be much better for this and require fewer instructions, an anecdotal validation for smaller, focussed models

I was surprised out how you can define arbitrary grammars using arbitrary formulation and it would follow it. Of course you have to redo the prompt every time there is an update... such a pain

https://github.com/hofstadter-io/hof/blob/_dev/flow/chat/pro...

CUE is indeed a beautiful language, will get those mind juices flowing for sure!

There is more work to be done on the codec implementation, but if you just want to split yaml/json across files, CUE is a great option

You might also like my project, built on CUE: https://github.com/hofstadter-io/hof We have a TUI where you can explore and work with CUE, JSON, Yaml

You can pretty much make up any pseudo grammar like this one, which is a reduced JSON object that is close to CUE: https://github.com/hofstadter-io/hof/blob/_dev/flow/chat/pro...

No need to be formal or use a standard format

Yea, in particular for this project, they have created a bespoke templating system.

You can get the same thing with Go text/templates by adding chat function(s) as custom a helper: https://github.com/hofstadter-io/hof/blob/_dev/lib/templates...

100% one of the best things about building a TUI is not having the pain of modern web development. I do think there is a way to have a CLI & TUI come from the same code, so you can get the best of both, or pick the best for the task at hand.

experiments in progress here: https://github.com/hofstadter-io/hof/tree/_dev/lib/tui