Our great sponsors
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
-
logseq
A local-first, non-linear, outliner notebook for organizing and sharing your personal knowledge base. Use it to organize your todo list, to write your journals, or to record your unique life.
-
ScreenBaby
A WIP of a node.js app to parse PDF screenplays into a JSON object using ScreenJSON Format.
-
-
KeenWrite
Discontinued Free, open-source, cross-platform desktop Markdown text editor with live preview, string interpolation, and math.
-
-
wisewriterv3
From an input, creates a full book, with cover art and sells it on Amazon. Using OpenAI for content, Midjourney for covers, and puppeteer for product input.
-
Filestash
🦄 A modern web client for SFTP, S3, FTP, WebDAV, Git, Minio, LDAP, CalDAV, CardDAV, Mysql, Backblaze, ...
-
menubar
Discontinued AI Chat Browser: Fast, Full webapp access to ChatGPT / Claude / Bard / Bing / Llama2! I use this 20 times a day. [Moved to: https://github.com/smol-ai/GodMode] (by smol-ai)
-
MathAnimation
A simple C++/OpenGL application to create quick and dirty mathematically accurate animations
-
-
-
-
-
squashql
Official repository of SquashQL, the SQL query engine for multi-dimensional and hierarchical analysis that empowers your SQL database
-
-
-
jekyll-sqlite
A Jekyll plugin that lets you use SQLite database instead of data files as a data source.
-
ansible-easy-vpn
An Ansible playbook that sets up a Wireguard server with ad blocking, DNS-over-HTTPS, and a WebUI with 2FA
-
-
r0b0
Discontinued r0b0 is a communication system for connecting human interface device (HID) hardware and system software; an `aconnect` for anything.
-
openai-functools
openai-functools: Simplified Generation of OpenAI Functions JSON Metadata for OpenAI Function Calling
-
-
-
highfleet-ship-opt
A c/c++ module and python extensions for automatic optimization of Highfleet ship modules. Try it live at https://hfopt.jodavaho.io
-
libhum
Libhum is a Python GPU-accelerated library to extract and compare Electricity Frequency Signals (ENF)
-
OpenAdapt
AI-First Process Automation with Large [Language (LLMs) / Action (LAMs) / Multimodal (LMMs)] / Visual Language (VLMs)) Models
-
ghidra-unlinker-scripts
Scripts for unlinking a program back into relocatable object ELF files with Ghidra
-
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Working on & off on a scuba diving site which allows you to log dives, sites, photos & sealife.
Currently only South Australian & Victorian dive sites, but probably the most extensive collection of either.
Live website here: https://divedb.net/
Source here: https://github.com/cetra3/divedb
Currently exploring activitypub integration, being able to log dives and have people follow/comment/react to them from the fediverse
We've started working on a tool to use CUE to create a single config space across TF, Helm, and other tools in the devops space.
https://github.com/hofstadter-io/cuelm
Our hypothesis is that a unified config space across our industry will bring many benefits.
Our open questions are largely around whether we
(1) replace / alternative
(2) just a skin, still same reconciliation loop from the tools
(3) replace, but support existing modules & charts
ROSboard -- visualize ROS topics in the web browser
https://github.com/dheera/rosboard
Lots of things that could be done with this but a lot of robotics teams and projects got laid in the past year and it doesn't have as many users as it once did. There is also Foxglove which is a full-blown funded startup working on something similar.
I built mirrored fables, an AI powered visual novel generator. Give it a try!
https://www.mirroredfables.com/?proxyKey=hellohackernews
source: https://github.com/mirroredfables/mirroredfables
I've been building Revenut, a web + mobile app (PWA) built in React Native + Typescript that does some revenue forecasting for a SaaS that uses Stripe as their payments provider. Stripe's mobile app and others kinda do this already but there are issues as noted in the repo's readme that made me go out on my own and open-source & solve an issue my own SaaS faced with Stripe:
https://github.com/hbcondo/revenut-app
https://github.com/SeleneSoftware/seleneCMSBundle
And a skeleton application that uses the bundle.
Probably too early even for this discussion, but why not: https://github.com/cconvey/cpu-n1
It's a simple (WIP) ISA and corresponding simulator. I want to get experience adding a new backend target for LLVM, and this is the target.
https://github.com/SeleneSoftware/SeleneCMS
Issues and Pull Requests welcome.
I am building a Web UI called paisa[1] to visualize personal finance data. We currently have very good command line tooling for this (ledger/hledger/beancount). A Web UI will make it even easier to see what's going on with your finances.
I am interested in knowing what are the common problems you face with command line approach that can be solved via a Web UI
[1]: https://github.com/ananthakumaran/paisa
[2]: https://paisa-demo.ananthakumaran.in/
https://github.com/vanna-ai/vanna/blob/main/papers/ai-sql-ac...
I'm also working on cli time tracking tool that is still very basic but I'm sort of wondering how best to add more functionality. The current main branch allows you to start a task with `$ start -t` and you can add details and then list all the events you've tracked, all saved to sqlite database. I like how immediate it is to track things from the command line, but I'm still thinking about how to display the events and tasks you've tracked (I'm building with rust and started tinkering with Ratatui to build a TUI, though I prefer just having a cli most of the time). I'm also still thinking about cli UX. I kind of prefer the git hub cli style that has commands and sub-commands because it seems more orderly than just having long lists of arguments. Anyway it's very in the beginning stages but would love recommendations on reading for UX for command line interfaces. https://github.com/farmeroy/Time-Bandit
I have been working on this podcast transcription project for a couple months and it's been super rewarding.
I listen to a podcast called No Such Thing As A Fish, where some researchers talk about their favorite facts they learned that week. Then they riff on it and are generally smart and funny. I listened to the series so many times that I decided I wanted to listen to the show on shuffle, not at the episode level, but at the fact level.
Since I have been playing around with whisper.cpp in python this seemed like a perfect way to combine some technologies I've been wanting to play with.
I ran whisper over the entire podcast and transcribed all the episodes. I had to do this multiple times because I kept messing up. It eventually took like 7 straight days of my M1 processing to get through ~490 episodes.
4 million words, and an 800Mb SQLite database later, I got the transcriptions done and have put up a nice site for searching through the data.
https://transcript.fish
Now I just need to figure out the rest. Breaking it up into facts. Getting the audio working. Highlighting and linking to words, phrases, etc.
Some cool info about the process so far:
1. The SQLite database is chunked up and stored as static files, and the frontend queries the static files directly using HTTP range requests, so it only downloads a couple hundreds kbs when querying.
2. I've been proper using ChatGPT 3.5 free version to help me write python and SQL. It's been pretty game changing as I feel basically no pain from not knowing what I'm doing.
The code is here: https://github.com/noman-land/transcript.fish
Please help if you know how to get whisper speaker diarization working!! I would really appreciate the help.
Neoq (https://github.com/acaloiaro/neoq) is a background job processor for Go.
Yes, another one. It began from my desire to have a robust Postgres-backed job processor. What I quickly realized was that the interface in front of the queue was what was really important. This allowed me to add both in-memory and Redis (provided by asynq) backends behind the same interface. Which allows dependent projects to switch between different backends in different settings/durable requirements. E.g. in-memory for testing/development, postgres when you're not running Google-scale jobs, and Redis for all the obvious use cases for a Redis-backed queue.
This allows me to swap out job queue backends without changing a line of job processor code.
I'm familiar with the theory that one shouldn't implement queues on Postgres, and to a large extent, I disagree with those theories. I'm confident you can point out a scenario in which one shouldn't, and I contend that those scenarios are the exception rather than the rule.
I want to integrate automatic code documentation with knowledge graph/wiki software. Think [Logseq](https://logseq.com/) but with code docs integrated into it, both from public documentation sites but also internal/private codebases. I've done no work to begin implementing this and I have no idea if anyone would find it helpful or not. For me, frequently the thing that slows me down most is having to break my coding flow and search for docs on the thing I'm writing. There's integrations in my editor for some things (Rust and Go both have pretty great support for this with their LSPs), but often I need more in-depth docs than what is immediately provided. Further, I firmly believe that most of the knowledge about a codebase should live in or next to that codebase. I'd really like to have a knowledge graph that integrates fully into the development cycle.
As someone who has been majorly affected by the Hollywood strikes, I finally dived in and spent the time needed to make mistakes to continually learn how to program.
That said, ive been working on a node.js psd Text screenplay parser that will ultimately push out a JSON object according to an initiative to standardize all screenplay objects in screenJSON[0].
Here is the github for my project, it’s messy but I’m about 80% there and just need to spend more time on Regex to catch edge cases. Currently working on a next.js front end that maps out the script into a visual format but that’s not ready to be seen yet. Also don’t judge me on the name, it’s not going to stay as that. No AI as of yet, but could use text classifiers to be able to tell the difference between a key prop, and say, a camera movement but I’m trying to get as far as I can without LLM help of the classifications.
https://github.com/mattrichmo/ScreenBaby.
[0] https://screenjson.com.
https://github.com/aaausar97/israpdead_react
would like to create a writers/contributors dao and make the splits of any donations go to contributors automatically. Use headless cms for backend and react for web/mobile app.
I am into old myspace early 2000s rn. Design leans into that and low attention spans lol. Think minecraft game with reddit voice background tiktoks.
It's a web app implementing the todo.txt format (see http://todotxt.org/). It's an exercise to learn frontend currently, I doubt I could successfully monetize it. Would appreciate any feedback!
Hey, so I got pretty far into implementing open api specs for LLMs. To the point where you can point to an open api spec, ingest it, and then it’s available for GPT to select to satisfy user requests.
I do this by minify the specs and building an index to feed it that it selects from using logit bias.
Im at pause point on it while I figure out how to actually have GPT build the calls, but I think something like you have might solve that.
https://github.com/ShelbyJenkins/LLM-OpenAPI-minifier
KeenWrite is my free, open-source, cross-platform desktop Markdown editor that can produce beautifully typeset PDFs. I started working on it years ago to help write a novel that has a complex timeline and I couldn't find a text editor that would allow me to integrate a character sheet with the story itself.
https://github.com/DaveJarvis/keenwrite
Tutorials:
* https://www.youtube.com/playlist?list=PLB-WIt1cZYLm1MMx2FBG9...
Here's what I mean by using variables directly:
* https://www.youtube.com/watch?v=CFCqe3A5dFg
CommonMark doesn't propose a standard for bibliographic references. Would anyone find the editor more appealing if it had cross-references and citations?
I had an issue where I had a million droplets with an app on port 3000 because nginx is hard and I'm not that smart, so I made a thing that lets you add a domain mapped to a port super easy, and the request gets dynamically routed to the correct app on the correct port. It lacks https, but is otherwise super useful.
https://github.com/donuts-are-good/appserve
Slack and discord bots for QA over docs. It’s the hello world of LLMs, but my focus is on making them polished and easy to deploy into a docker container. Use them for my current company with ok success. They actually out perform some of the funded startups in the space, but it’s an arms race and I don’t think I’ll be able to keep up against the saas offerings.
https://github.com/shelby-as-a/shelby-as-a-service
This is a single page script that that generates an idea, creates an audience profile, then writes a book of short stories in a 5d array, looping over each level, then generates an image through midjourney through a hacked together api and creates a styled cover where there are a number of steps to ensure that aesthetically, the book cover is decent. It then uploads it to Amazon using puppeteer and uploads it to the woocommerce site as a new product.
https://github.com/mattrichmo/wisewriterv3
It’s my first complex app I’ve created that required multiple apis, and dealing with errors from multiple levels but I never actually uploaded a single book to Amazon so I can’t speak to the quality of the content.
https://github.com/lukesalamone/code_nitro
Convert your code to an image from the command line. No internet connection needed, although it can also download and convert code from Github too. Made it because currently the best alternative (carbon-now-cli) sends your code over the wire and there's a lot of situations where that's not desirable.
https://github.com/mickael-kerjean/filestash
This is what I wish Dropbox was, a simple layer that make interacting with your FTP server easy so nobody has to own your data. The end game is both to be feature complete with Dropbox and be able to change every aspect of the application through plugin so everyone can get out what they want from it.
been working on https://github.com/smol-ai/menubar for a couple months, but nervous to "launch" it because i feel like it hasn't had a significant "wow" moment yet. basically the idea is to 5x the amount of chat output for every unit of human input, to get variety + familiarity with the wide range of chat models out there. its moved from a "smol menubar app" towards its own "ai chat browser". i wonder if i should put int he work to make it a full "ai browser" before i launch it.
https://github.com/ambrosiogabe/MathAnimation
If you’ve ever tried to make a mathematical animation (think 3Blue1Brown), it’s a real pain. I was using manim for awhile to make animations for my YT channel, but the whole iteration process felt very slow and repetitive. So I thought I would recreate manim over the weekend, except with a GUI and real-time feedback. It’s been a year and a half and I’m hoping this weekend will be done soon so I can move on and start making videos again.
So far, it does a lot, but it still needs a lot of polish and refinement. The readme gives some gifs and a better idea of the feature set right now.
I'm working on Knowii [0], an Open Source (AGPL) Knowledge Management platform for communities. In the first version, communities will be able to curate/capture/share collections of useful/interesting resources. A browser extension will make it easy to capture information/links.
Later on, we'll add job boards, ways to share summaries/notes, news, vote, tag, explore through user profiles, etc.
[0] https://github.com/knowii-oss/knowii
I have a project. I have posted it once herenon HN. I have not received any feedback then, it hasn't received much traction.
It is a link aggregation. Can be used as a RSS client, or YouTube front end for subscriptions.
It is intended for light, personal use, therefore it is not much scalable, but supports user management.
https://github.com/rumca-js/Django-link-archive
This is very cool! I just downloaded the extension and tried a hello world program. Will tinker a bit more.
If you can make a library to run python code inside a JS project (like Next JS) then we could build a code interpreter in Next JS. Would be super cool. Here is a python implementation - https://github.com/shroominic/codeinterpreter-api - for your reference.
I'm in the process of creating a deep learning course called Zero to GPT - https://github.com/VikParuchuri/zero_to_gpt .
It teaches you everything you need to train your own LLM, including the basics of deep learning and linear algebra. You learn the theory and the application, so you have a strong grounding in what you're doing. It includes written explanations, diagrams, and videos.
I'm up to transformers now - only a few more lessons to go. It's been fun to write, but balancing time spent training models with writing the course has been hard. Hopefully I will get the time to finish it soon.
I'm working on pls (https://github.com/dhruvkb/pls/), a prettier and more powerful alternative to ls(1) that adds a lot of customisation and provides a very fluent command-line interface. It aims to be a superset of exa in terms of the features, while being more actively maintained and targeting a smaller subset of pro-users.
It works quite well and is very usable as a daily driver. I'm adding more features to it and making it available to install it via platform-native package managers.
Wow, what a great idea for a thread!
I'm trying to pare down my personal projects to just the really exciting ones, so I don't have much, butni think the most appropriate to the thread is https://github.com/EternityForest/iot_devices
It's mean to be a cross-framework library for creating device integrations, so you can, say, write a handler for RTL SDR weather stations, and use it in a simple script up to a mega framework.
I kind of dislike the way HASS and others handle automations where they have special purpose primitives for everything that needs lots of hand written code.
I just have config entries, they must be strings, and data points, they can be strings, numbers, bytes, or objects. You can put metadata on them. There's also a few other utilities like the ability to make subdevices, and the ability to request things from the host.
There are no special subclasses, a light bulb is just a device with a brightness point.
It currently runs my security system with object detection recording, QR decoding if desired, multiple regions, motion detection without decoding every frame, and subsecond latency streaming to the browser, a nice recordings browser that can view a recording while it's being made, etc.
• GitHub: https://github.com/feature-express/feature-express
AIPL is an "Array-Inspired Pipeline Language", a tiny DSL in Python to make it easier to explore and experiment with AI pipelines.
https://github.com/saulpw/aipl
When you want to run some prompts through an LLM over a dataset, with some preprocessing and/or chaining prompts together, AIPL makes it much easier than writing a Python script.
I'm still building a low code system with an easy to use component marketplace where you can edit components within the low code tool. Still a work in progress: https://github.com/yazz/yazz
- Build your own honeypot with ContainerSSH (DevConf CZ 2021) [4]
[1]: https://containerssh.io
I am building a sort of SQL query engine compatible with several databases (bigquery, snowflake, duckdb, clickhouse, spark) in order to add more power to the SQL query language bringing multi dimensional capabilities to SQL (and more). The project was released few months ago and available on GitHub https://github.com/squashql/squashql
There are also mockups with more features, so ignore weird UI at first.
[1]: https://www.drawio.com
I’ve been working on a Jekyll SQLite plugin: https://github.com/captn3m0/jekyll-sqlite.
Jekyll let’s you use CSV/YAML/JSON files as data source, but they’re unwieldy since the schema is fixed.
The plugin is meant to let you reshape your data before you use it within Jekyll. You get complete flexibility of SQLite, to let you join tables, use indexes etc.
What I can’t figure out is how to best do “dynamic” queries, say a query that needs to show the number of reviews on the restaurant page (restaurants is the table). Each of these page will need to run a dynamic query with the page.id variable.
SQLite Injection isn’t a big issue, since this is all trusted input. But I’d like something better than just Templating queries using liquid.
I’m doing some parameterized queries on the global level (see README). Need to find a nice way to do them locally.
Maybe just putting them in metadata would be a good way?
I would put this behind a VPN to keep it private. I have used this[0] Wireguard setup before, which takes the grunt work out of configuring everything.
[0] https://github.com/notthebee/ansible-easy-vpn
I have this hobby project for personal time tracking I work on from time to time. It's usable enough for my daily use so motivation has dropped since that point.
https://github.com/eliasson/quarter
Unfortunately Nitter broke earlier this week thanks to some changes by Twitter: https://github.com/zedeus/nitter/issues/983
I was using a self-hosted Nitter instance for the RSS feeds as I deactivated my Twitter account
File sharing and soon remote execution over the internet cross OS. Private and no servers.
https://github.com/prettydiff/share-file-systems
I wrote a service (Google Cloud Run as the backend, with Telegram bot as the frontend) to generate readable ePub from URLs and send directly to e-ink readers. It was originally wrote for reMarkable 2 (using reMarkable cloud), I recently added support for Kindle (by using the send-to-kindle emails). The code is at https://github.com/fishy/url2epub and I blogged about the recently added kindle support at https://b.yuxuan.org/url2epub-kindle.
I'm open to suggestions on what other e-ink platforms to add, as long as they have a reasonable cloud API. I'm also looking for a good e-ink platform to move to personally, as it becomes apparent that reMarkable really doesn't want third parties to use their proprietary cloud "API".
- designed to feel intuitive to non-technical users
I've been at the drawing board for a long time, stuck in syntax design. So I don't have a working prototype. But I do have a landing page and some documentation:
https://matry.design/
https://github.com/matry/documentation
I've been working on a redesign of a small robot platform I designed in grad school ( https://github.com/psychomugs/r0b0 ). We designed the robot as a small robot construction kit for other researchers and roboticists, and is basically a floating head under a fabric exterior. My last research project used the robot as a tabletop motion-controlled telepresence device accessible through a no-downloads mobile browser.
I never had the time to polish the platform so I did a ground-up hardware and software refactor. The hardware was redesigned as an homage to popular snap-fit robot model kits, with runner layouts and instructions ( https://github.com/psychomugs/r0b0/blob/main/docs/assets/bls... ).
I've tried to generalize the software as an `aconnect` for non-robot-specific hardware devices. The software powers this digital back I designed for my >50-year old Leica M2 ( https://www.raspberrypi.com/news/leica-mpi-a-pi-zero-retrofi... ) and enables quick prototyping of physical IO, e.g. using a MIDI keyboard to control motors or a gaming joystick to control a mouse.
It's mostly a passion project to refine the platform into something more usable than how it was left, but I'm open to suggestions or beta testers!
Easily generate function metadata (json) automatically for GPT function calling: https://github.com/Jakob-98/openai-functools
My colleague and I have been working on this tool and using it in some internal projects. It works quite well as an intent->functions machine, and makes the process of invoking functions with GPT a lot more bearable. We hope to make it a library which does one thing so great that it will be the industry standard in its niche, so looking for some feedback on the documentation and use.
I'm working on https://commitspark.com , a headless CMS where all data is stored in a GitHub repository with branching/merging workflow support for content.
Reading/writing content from/to GitHub happens through an Open Source API library I released. For the content data schema, I simply require a plaintext GraphQL type file inside the repository. The schema then automatically determines the API structure as well as the editor-friendly UI that I can generate on the Commitspark website when you log in via GitHub.
Even though Commitspark is already publicly available, it is still 100% a side-project that doesn't earn me any money (yet).
I am currently developing an open-source (MIT) database that can be directly accessed from the frontend (browser and apps) , that is distributed, and capable to deliver data in real-time. The primary goal is to provide support for any use-case that requires close proximity to users and, most importantly, it is entirely free to use and run by yourself if desired.
If you would like to view it, please visit: https://github.com/mateusfreira/nun-db
Feedback is always welcome, especially if you have a use-case in mind that you believe it may be suitable for but are unsure. I have already utilized it in many of my personal projects and for a few clients with a small number of users, but I am hopeful that it will soon be ready for larger-scale implementation.
Currently working on a site that allows you to use your data from run/bike apps to visualize your data on a map, make it look neat, and order a print.
I have an early version of it hosted on https://ridemapper.henrygrant.dev and add features as I think of them/when I have time.
https://github.com/henrygrant/ridemapper-svelte
I’m working on ModelFusion, a TypeScript library for working with AI models (llm, image, etc.)
https://github.com/lgrammel/modelfusion
It is only getting limited traction so I’m wondering if I’m missing something fundamental with the approach that I’m taking.
If you play "Highfleet" at all, can you poke around https://hfopt.jodavaho.io and see if you can get it to generate module lists that make sense?
It's an optimizer, just add what you know you want, set some cost / range / speed limits, and it will output the cheapest version of the ship that includes all required modules and has all requested stats.
I'm working on "is". https://github.com/oalders/is
It allows me to install various things in various environments without getting too deep in to bash scripting.
For example:
$ is cli version tmux gt 3.2 && echo || echo
$ is there go && is there nvim && nvim +:GoUpdateBinaries +qa
$ is known cli version bash
I'm writing a open-source library (https://github.com/RaphaelJ/libhum/) and a webapp (https://datethis.app:8080/) that detects the Utility Grid noise emitted by electrical devices (A/C, motors, appliances ...) from Video/Audio recording.
It then tries to date the recording by matching it to historical data.
The signal processing part is getting decent. I'm still trying to collect more reference signals in RU, UK, China and Japan.
Tom Scott made a video about the idea: https://www.youtube.com/watch?v=e0elNU0iOMY
I've been working on a specific reverse-engineering technique called _unlinking_ [1] on-and-off for the past 16 months or so. I'm on my third prototype (first a set of Ghidra scripts written in Jython [2], then a fork of Ghidra [3] and now a Ghidra extension [4]) and I've started a blog in order to document it [5], which side-tracked into writing a whole series of articles on reverse-engineering to introduce the topic.
What for, you may ask? Basically I'm trying to decompile a PlayStation 1 video game and I've quickly decided that dealing alone with multiple +500 KiB executables of complete utter spaghetti code wasn't going to work. Instead, I've decided that I'd rather divide-and-conquer the problem, so I've been tooling up to split executables into relocatable object files, in order to decompile those one at a time and _Ship of Theseus_-style my way to success.
Ironically, all of that stuff is so not done that I don't even know what meaningful feedback there could be. My prototypes do work, but only for 32 bit little endian statically-linked MIPS executables. The articles on my blog are draft-quality. As for the decompilation project itself that started all of this, it hasn't seen much progress due to all of those side-quests. The overall topic is so esoteric that so far I've only managed to hear about one group of two persons that tried to do anything remotely similar and one another anecdotal account [6] that this particular skill is very uncommon among reverse engineers.
Personally, I'm starting to think that maybe I could've actually reverse-engineered and decompiled the game in the time I took to get here. I've also tried to engage with Ghidra to upstream the foundations of my modifications in my fork, but after some back-and-forth it became clear that my prototype-grade stuff wasn't industrial-grade and couldn't be merged in its current state, which is why I'm currently reworking the code in my fork as a Ghidra extension.
To those that want to provide feedback after reading all of this: beware, I've had a lot of fun going down that rabbit hole, but this is one hell of a time sink _and_ a particularly tricky mind-bender.
[1] I don't actually _know_ what's the actual name for this technique, given that there are so few resources on it out there. I do know I didn't invent it.
[2] https://github.com/boricj/ghidra-unlinker-scripts
[3] https://github.com/boricj/ghidra/tree/feature/elfrelocateble...
[4] https://github.com/boricj/ghidra-unlinker-extension
[5] https://news.ycombinator.com/item?id=36575081#36590078
[6] https://news.ycombinator.com/item?id=35729232&p=3#35740761
I've been working on a specific reverse-engineering technique called _unlinking_ [1] on-and-off for the past 16 months or so. I'm on my third prototype (first a set of Ghidra scripts written in Jython [2], then a fork of Ghidra [3] and now a Ghidra extension [4]) and I've started a blog in order to document it [5], which side-tracked into writing a whole series of articles on reverse-engineering to introduce the topic.
What for, you may ask? Basically I'm trying to decompile a PlayStation 1 video game and I've quickly decided that dealing alone with multiple +500 KiB executables of complete utter spaghetti code wasn't going to work. Instead, I've decided that I'd rather divide-and-conquer the problem, so I've been tooling up to split executables into relocatable object files, in order to decompile those one at a time and _Ship of Theseus_-style my way to success.
Ironically, all of that stuff is so not done that I don't even know what meaningful feedback there could be. My prototypes do work, but only for 32 bit little endian statically-linked MIPS executables. The articles on my blog are draft-quality. As for the decompilation project itself that started all of this, it hasn't seen much progress due to all of those side-quests. The overall topic is so esoteric that so far I've only managed to hear about one group of two persons that tried to do anything remotely similar and one another anecdotal account [6] that this particular skill is very uncommon among reverse engineers.
Personally, I'm starting to think that maybe I could've actually reverse-engineered and decompiled the game in the time I took to get here. I've also tried to engage with Ghidra to upstream the foundations of my modifications in my fork, but after some back-and-forth it became clear that my prototype-grade stuff wasn't industrial-grade and couldn't be merged in its current state, which is why I'm currently reworking the code in my fork as a Ghidra extension.
To those that want to provide feedback after reading all of this: beware, I've had a lot of fun going down that rabbit hole, but this is one hell of a time sink _and_ a particularly tricky mind-bender.
[1] I don't actually _know_ what's the actual name for this technique, given that there are so few resources on it out there. I do know I didn't invent it.
[2] https://github.com/boricj/ghidra-unlinker-scripts
[3] https://github.com/boricj/ghidra/tree/feature/elfrelocateble...
[4] https://github.com/boricj/ghidra-unlinker-extension
[5] https://news.ycombinator.com/item?id=36575081#36590078
[6] https://news.ycombinator.com/item?id=35729232&p=3#35740761
I've been working on a specific reverse-engineering technique called _unlinking_ [1] on-and-off for the past 16 months or so. I'm on my third prototype (first a set of Ghidra scripts written in Jython [2], then a fork of Ghidra [3] and now a Ghidra extension [4]) and I've started a blog in order to document it [5], which side-tracked into writing a whole series of articles on reverse-engineering to introduce the topic.
What for, you may ask? Basically I'm trying to decompile a PlayStation 1 video game and I've quickly decided that dealing alone with multiple +500 KiB executables of complete utter spaghetti code wasn't going to work. Instead, I've decided that I'd rather divide-and-conquer the problem, so I've been tooling up to split executables into relocatable object files, in order to decompile those one at a time and _Ship of Theseus_-style my way to success.
Ironically, all of that stuff is so not done that I don't even know what meaningful feedback there could be. My prototypes do work, but only for 32 bit little endian statically-linked MIPS executables. The articles on my blog are draft-quality. As for the decompilation project itself that started all of this, it hasn't seen much progress due to all of those side-quests. The overall topic is so esoteric that so far I've only managed to hear about one group of two persons that tried to do anything remotely similar and one another anecdotal account [6] that this particular skill is very uncommon among reverse engineers.
Personally, I'm starting to think that maybe I could've actually reverse-engineered and decompiled the game in the time I took to get here. I've also tried to engage with Ghidra to upstream the foundations of my modifications in my fork, but after some back-and-forth it became clear that my prototype-grade stuff wasn't industrial-grade and couldn't be merged in its current state, which is why I'm currently reworking the code in my fork as a Ghidra extension.
To those that want to provide feedback after reading all of this: beware, I've had a lot of fun going down that rabbit hole, but this is one hell of a time sink _and_ a particularly tricky mind-bender.
[1] I don't actually _know_ what's the actual name for this technique, given that there are so few resources on it out there. I do know I didn't invent it.
[2] https://github.com/boricj/ghidra-unlinker-scripts
[3] https://github.com/boricj/ghidra/tree/feature/elfrelocateble...
[4] https://github.com/boricj/ghidra-unlinker-extension
[5] https://news.ycombinator.com/item?id=36575081#36590078
[6] https://news.ycombinator.com/item?id=35729232&p=3#35740761
This is very cool, looking forward to it! I've been doing the same thing with Spanish Wikipedia articles for a while, using a few lines of Bash + Regex. I was using Apertium for it. https://apertium.org/ It's definitely worse than most ML-based solutions, but it works reliably and fast; you can run it entirely offline. With Spanish translations, the main problem I was facing is lack of vocabulary, so I created https://github.com/phil294/apertium-eng-spa-wiktionary which about doubles the amount of recognized words, albeit with wonky grammar.
This is very cool, looking forward to it! I've been doing the same thing with Spanish Wikipedia articles for a while, using a few lines of Bash + Regex. I was using Apertium for it. https://apertium.org/ It's definitely worse than most ML-based solutions, but it works reliably and fast; you can run it entirely offline. With Spanish translations, the main problem I was facing is lack of vocabulary, so I created https://github.com/phil294/apertium-eng-spa-wiktionary which about doubles the amount of recognized words, albeit with wonky grammar.
https://github.com/dbnet-io/dbnet
dbNet is a web-based SQL IDE using Go as a backend, and your browser as front-end. I built it because I was unsatisfied with the database clients out there. Alot of them are top-heavy, unintuitive, slow or expensive.
Here are some of the databases it connects to:
Clickhouse
Use kubeswitch to switch kubernetes contexts or namespaces per shell.
https://github.com/spideyz0r/kubesw
It's written in go, I did some releases but still requires more tests.
It would cool to get some feedback.
Kind of an open-source clone of Firebase. It's not only a database but also can be used to serve as a REST backend for your frontend.
I've started building this a while ago but gave up around two years ago: https://gitlab.com/restbase/restbase