Ask HN: What are some tools / libraries you built yourself?

We’ve created https://lowdefy.com and open-sourced it.

Over the past 5 years we’ve been building custom B2B and back office web applications. Nothing in the market had the level of customization we required both from a technical and commercial angle. So decided to standardize how we “configure” apps into a platform.

We did this by designing a schema by which to define web apps which is easy to read, write and understand, and works well with normal developer tools like editors, git and CI. While keeping it self hosted and even server-less.

Lowdefy has enabled us to rapidly deploy custom business apps like BI, CRM to MRPs.

Check it out :) https://github.com/lowdefy/lowdefy

I made my own HTTP app. server and distributed JSON database: https://github.com/tinspin/rupy

On top of this I have made many projects and I still use all of them today even 10 years later, some favourite:

http://fuse.rupy.se (multiplayer system)

I built https://snipp.in because I was top frustrated about the loading times of notion. I am a developer and I take notes quite often, most of us spend a lot of time on an IDE and when we need format a quick json or copy a tiny snippet we often find ourselves in opening another instance of an editor with a txt file or random website to formate json. Snipp.in is a tiny jn browser editor, note taking app and a snippet manager all in one which looks like an Editor but really fast without any bs of signin/signups or cloud. It just stores everything in your browser using IndexedDB. It's also open-sourced at https://github.com/haxzie/snipp.in

Two things that I use daily:

Serilog.Sinks.InMemory for help with unit testing log messages etc: https://github.com/sandermvanvliet/SerilogSinksInMemory

TestableHttpClient to make testing API calls easier by providing a simple way to configure responses and behaviour as well as asserting actual calls: https://github.com/sandermvanvliet/TestableHttpClient

Both I think equal parts not invented here, “how hard is this really?”, not finding something at the time and just doing it for the fun of it

Two things that I use daily:

Serilog.Sinks.InMemory for help with unit testing log messages etc: https://github.com/sandermvanvliet/SerilogSinksInMemory

TestableHttpClient to make testing API calls easier by providing a simple way to configure responses and behaviour as well as asserting actual calls: https://github.com/sandermvanvliet/TestableHttpClient

Both I think equal parts not invented here, “how hard is this really?”, not finding something at the time and just doing it for the fun of it

I created dither, a library for image dithering in Go. And then I created didder, a CLI tool for the library. I had become interested in the aesthetic effect of dithering, but realized that there aren't many tools that do the math correctly, and also provide many different algorithms to explore.

https://github.com/makeworld-the-better-one/dither

https://github.com/makeworld-the-better-one/didder

I created dither, a library for image dithering in Go. And then I created didder, a CLI tool for the library. I had become interested in the aesthetic effect of dithering, but realized that there aren't many tools that do the math correctly, and also provide many different algorithms to explore.

https://github.com/makeworld-the-better-one/dither

https://github.com/makeworld-the-better-one/didder

I created intercooler.js in 2013 so I could do AJAX in HTML:

https://intercoolerjs.org

Last year I removed the jquery dependency and cleaned it up based on a lot of lessons that I learned, renaming it to hmtx:

https://htmx.org

Same idea: extends/complete HTML as a hypertext so you can build more advanced UI within the original hypermedia web model, and cleaner implementation.

Part of that cleanup involved me pulling out some functionality around events and a proto-scripting language (ic-action), and I enjoy programming languages, so I created a front end scripting language to fill that need:

https://hyperscript.org

It's based on HyperTalk and has a lot of domain specific features for lightweight front end scripting, kind of a jQuery or AlpineJS alternative.

I created intercooler.js in 2013 so I could do AJAX in HTML:

https://intercoolerjs.org

Last year I removed the jquery dependency and cleaned it up based on a lot of lessons that I learned, renaming it to hmtx:

https://htmx.org

Same idea: extends/complete HTML as a hypertext so you can build more advanced UI within the original hypermedia web model, and cleaner implementation.

Part of that cleanup involved me pulling out some functionality around events and a proto-scripting language (ic-action), and I enjoy programming languages, so I created a front end scripting language to fill that need:

https://hyperscript.org

It's based on HyperTalk and has a lot of domain specific features for lightweight front end scripting, kind of a jQuery or AlpineJS alternative.

I've built several extensions to vscode. My best one has to be a tool to help you upgrade dependencies in your package.json: https://github.com/pgsandstrom/package-json-upgrade

I just felt that other extensions lacked polish. The UX wasn't very good, or they didn't read all npm config properly.

I wrote my own game engine that support an adaptive music system (some called dynamic music system), built using OpenGL (by SDL2), OpenAL, Physics powered by Box2D.

Pretty old projects, built during my years in the university. But also one of the proudest projects I achieved.

https://github.com/spywhere/Legacy-ParticlePlay

https://github.com/jhspetersson/fselect

A tiny tool I wrote to search within file piles (mostly unsorted downloads, torrents, and such). I could never remember `find` options, and more advanced queries are a pain. Now one can use some kind of SQL flavor to get the job done.

I started building Scrawl-canvas[1] - my Javascript canvas library - back in 2013 because I was trying to land my first full-time job in the industry and a recruiter told me it would be a really good idea if I had a portfolio of sites and projects to show prospective employers - including one on GitHub to prove that I knew what GitHub was.

I'm still developing the library because ... I don't know why. I've lived with it for so long now it's become a bad habit I can't shake. I'm also quite proud of it because it can do things that other libraries can't do[2][3].

[1] - website - https://scrawl-v8.rikweb.org.uk/

[2] - Shape, and animate, an image between two paths - https://codepen.io/kaliedarik/pen/ExyZKbY

[3] - Filter over a live media stream - https://codepen.io/kaliedarik/pen/OJNVaEX

I created mate-i3-applet as it was the missing piece in a perfect MATE/i3 integration: https://github.com/city41/mate-i3-applet

I recently built this: (https://github.com/Link-/github-app-bash) as a quick way to generate access tokens to use with GitHub's APIs. Now I can use tokens with a limited scope, and lifetime when I test which is safer than using a personal access token that never expires. It's fully written in bash and does not have many dependencies. It's quick does the job.

OwnTime.jl - A Julia profiling library [1].

It reports the time spent on each line of your project, and you can filter which files are included. For example, you might do a complicated map operation which calls a function in another library, if you filter out the library then it will report that all the time was spent in your code on the line that performed the map. However, if you don't filter out the library then it will report that all the time was spent on the line inside the library.

I couldn't figure out how to get what I wanted from the existing profiling libraries so wrote this. It was quite easy once I figured it out. Julia has a built in sampling profiler you can turn on, and from there you can process the sampled frames however you'd like and display them however you'd like.

[1]: https://github.com/DevJac/OwnTime.jl

I wrote "tuc", an alternative to the "cut" Unix tool, for those situations where I need something more (mostly the ability to collapse the separator or negative indexing)

https://github.com/riquito/tuc

- I wrote my own message queue because of dissatisfaction with existing solutions; it's like a very smart gateway: https://github.com/queer/singyeong

- My own key-value store that versions values by default: https://github.com/queer/crush

- A Spring-inspired web app 'framework' for fun: https://github.com/queer/autumn

- Slowly building my own k8s-inspired container scheduler: https://github.com/queer/mahou

- I wrote my own message queue because of dissatisfaction with existing solutions; it's like a very smart gateway: https://github.com/queer/singyeong

- My own key-value store that versions values by default: https://github.com/queer/crush

- A Spring-inspired web app 'framework' for fun: https://github.com/queer/autumn

- Slowly building my own k8s-inspired container scheduler: https://github.com/queer/mahou

- I wrote my own message queue because of dissatisfaction with existing solutions; it's like a very smart gateway: https://github.com/queer/singyeong

- My own key-value store that versions values by default: https://github.com/queer/crush

- A Spring-inspired web app 'framework' for fun: https://github.com/queer/autumn

- Slowly building my own k8s-inspired container scheduler: https://github.com/queer/mahou

- I wrote my own message queue because of dissatisfaction with existing solutions; it's like a very smart gateway: https://github.com/queer/singyeong

- My own key-value store that versions values by default: https://github.com/queer/crush

- A Spring-inspired web app 'framework' for fun: https://github.com/queer/autumn

- Slowly building my own k8s-inspired container scheduler: https://github.com/queer/mahou

I've created a simple typescript library for generating QR payment strings, which can then be encoded into an actual QR payment. The actual standard for such payments is called SPAYD and it's accepted in most of our contry's bank apps.

Initially, I needed to create such QR and hadn't found any publicly available web tool, so I created one by myself.

https://github.com/Tajnymag/spayd-js

As a researcher in machine learning, I wanted to explore applications of Deepmind’s AlphaZero algorithm beyond board games (such as in automated theorem proving or chemical synthesis).

However, I noticed that existing open-source implementations of AlphaZero mostly consisted in complex C++ codebases that are highly specialized for specific games (eg. Leela Zero and LC0). Accessible Python implementations could be found but they were usually too slow to do anything useful on limited computing power.

Seeing this, I built AlphaZero.jl: https://github.com/jonathan-laurent/AlphaZero.jl

AlphaZero.jl is written in Julia and it is consistently one to two orders of magnitude faster than competing Python alternatives, while being equally simple and flexible. I just released a new version a few days ago with many new features (support for distributed computing, support for arbitrary MDPs...).

If you are a student, a researcher or a hacker curious about AlphaZero, please consider having a look!

In an attempt to move everything out of iCloud (I only use Apple products), I missed a compelling, FOSS password manager. After some time, I found passwordstore.org. Unfortunately, there was no Safari extension, so I built it myself: https://github.com/adur1990/Pass-for-macOS

I'm building a simple replacement for Apache Airflow to manage some small data pipeline projects: https://github.com/fieldryand/goflow. Airflow started to feel too heavyweight for these projects where all the computation was offloaded to independent services. I wanted a solution with minimal memory requirements to save costs and avoid the occasional high memory usage/leak issues I was facing with Airflow. I know there are tons of existing solutions in this space, but nothing I found seem to match the Airflow features I needed, so I decided to build it myself. It's been a great way to learn Go.

Lead Engineer at PAX Labs, we just open-sourced some Java utilities i wrote in the early days building service oriented architecture on dropwizard: https://github.com/PaxLabs/java-utils

A few things in there:

* `HibernateQuery` - wrapper around hibernate jpa implementation, because using the jpa criteria api is verbose as hell (seriously, even for java). Coming from having written Django, I wanted the ease of using Querysets in java

* Makemigrations - this seemed to be a missing hole in the java ecosystem. Our contractors were hand-writing sql migrations and using some framework that just ran them. Liquibase exists for managing migrations, hibernate can generate the schemas. This library is a dropwizard command to generate yml to migrate the old schema to the new one.

* generic pagination bundle

I build a Swift library to generate 2d dungeon maps, based on Perl and Javascript code from Donjon [0].

The code is kinda ugly but it works and I think it's one of the nicer dungeon generators: https://github.com/wolf81/DungeonBuilder

I also wrote a small library to deal with AD&D dates & times, also in Swift, quite clean code I think:

A Python program to make an embeddable search engine for a static website: https://github.com/joe-crawford/Static-Site-Search

It indexes the site and generates search functionality in JavaScript (the index is just a static JSON file) that can be uploaded wherever JS/HTML/etc. files can be uploaded. I thought it might be useful for people with sites on GitHub pages or similar who don't want to use one of the major search engine company widgets.

Also my back of the envelope calculations were that the simple search index for a small–medium site won't be that big.

I made a platform for hosting game servers at scale. All of the options previously on the market at the time were highly limited, didn't really put a lot of effort into security, or had existed for 10+ years and didn't feel much need to innovate.

https://github.com/pterodactyl/panel

Kind of took off after posting here about it a few years back, and now it just kind of sustains itself with some bug fix inputs every now and then.

I wrote Helium https://github.com/geuis/helium-css a while back to help find unused css on a site. The process of building sites has changed a lot since then so it’s not as universally useful as it used to be. (...Glares at React...). Joking aside, it’s had a good run. I’ve had some ideas for a few years about an updated version but it would really be an entire new project.

Also created https://jsonip.com. Get your IP address in a json object. I created it a decade ago to solve some problem I had and have been running it ever since. Last I checked it’s supporting 4-6 million requests per day from a pretty wide group of users.

I made a CLI interface for the Microsoft Academic API so you can search for papers, download them, and generate HTML from a collection: https://github.com/mila-iqia/paperoni

I'm using it to automatically collect papers published by members of the research institute I work for, it's working pretty well.

I also made a small utility to sync folders on-demand on my different machines: https://github.com/breuleux/synecure

Most of the work is done by a library called bsync that someone else had written, which itself uses rsync, but I'm quite happy about the interface I made. By default it syncs home-to-home, so you don't have to specify both the source and destination directories. You can just type "sy -r remote" in ~/whatever to sync it to ~/whatever on the remote.

I made a CLI interface for the Microsoft Academic API so you can search for papers, download them, and generate HTML from a collection: https://github.com/mila-iqia/paperoni

I'm using it to automatically collect papers published by members of the research institute I work for, it's working pretty well.

I also made a small utility to sync folders on-demand on my different machines: https://github.com/breuleux/synecure

Most of the work is done by a library called bsync that someone else had written, which itself uses rsync, but I'm quite happy about the interface I made. By default it syncs home-to-home, so you don't have to specify both the source and destination directories. You can just type "sy -r remote" in ~/whatever to sync it to ~/whatever on the remote.

I've made a library[1] to generate MOBI books for Amazon Kindle devices. It is relatively low level (you have to manage XHTML generation yourself), but it works and does exactly what I need it too.

[1]: https://github.com/leotaku/mobi

paperoni is awesome!

I once implemented the crossref api in rust in order to search for publications from the command line https://github.com/mattsse/crossref-rs

I built XState because I wanted a way for developers to integrate state machines & statecharts into web applications in a way that could be automatically visualized: https://github.com/davidkpiano/xstate

I was dissatisfied with the available fixture/test data generation tooling for Obection.js and Knex, so I ported factory_bot from Ruby to Typescript. I’ve called it Fixture Riveter [0]. It’s more verbose than the other similar tools (factory_girl for javascript, Fishery for typescript), but it’s extremely versatile and supports factory_bot’s full api.

[0]: https://github.com/Batterii/fixture-riveter

https://github.com/cnizzardini/cakephp-swagger-bake which builds out openapi from your cakephp applications meta data. One of many projects I took up to occupy myself during covid lockdowns.

tabula-java [0], a library for extracting tables from PDF files. It started as a monolithic webapp written in JRuby, and we later extracted the table detection and segmentation logic into a Java library.

[0] https://github.com/tabulapdf/tabula-java

I built an alternative to Make written in Go that is simpler to use and cross-platform: https://taskfile.dev/

Also, a library to write tests with databases for Go: https://github.com/go-testfixtures/testfixtures

I built an alternative to Make written in Go that is simpler to use and cross-platform: https://taskfile.dev/

Also, a library to write tests with databases for Go: https://github.com/go-testfixtures/testfixtures

I built emlearn, a Machine Learning inference engine for microcontrollers and embedded systems. It allows converting traditional ML models to simple and portable C99, following best practices in embedded software (no dynamic allocations etc). https://github.com/emlearn/emlearn

I made a small tool to script source-to-source transformations and apply them on C programs: https://github.com/lou1306/absentee

I used it during my PhD to automatically instrument benchmark programs for several different software verification tools (there is some form of standardization going on, but there are still a lot of corner cases where the same program will be interpreted, and thus verified, differently by two tools).

It's written in Python 3 and uses pycparser to do most of the heavy lifting. Currently it's a bit on the back burner, but I hope to come back to it soon-ish. Some desiderata:

* pycparser's AST API is not terribly efficient when you have to rewrite large subtrees, but I don't know if there are any nice alternatives.

There was a time in my team where we were taking just too much time and derailing in our daily standups, and the manager wanted to have a way to control how much time each one of us took during our standup and leave any further discussion for the 16nth minute. I created a [DSU timer][0] for the team which we used for quite some time and it served it's purpose. We later switched to another approach for our standups. It's currently [hosted][1] on github pages.

[0]: https://github.com/deathwebo/tinydiamond-timer

I built this Subject/Topic Extractor[1] in NodeJS to quickly extract topics/nouns/subjects from a String. My Use-Case was to determine what is trending, given a number of news article titles in a day

[1]: https://github.com/mudulo/subject-extractor

I built a configuration tool for Python apps: https://github.com/Euromance/confboy

Supports dynamic values, TOML configs, runtime mutations, merging and stuff.

I created a dotfiles management tool, yadm https://yadm.io

I'm a huge fan of Git, and the Git mental model just makes sense to me. I wanted to manage my dotfiles in the same way. At the time I had tried several other solutions, mostly similar tools like homeschick and vcsh.

I wanted a solution that was very portable, which both of those existing tools were. However, homeschick became a bit clunky to use because each time I wanted to do most operations I had to first "cd" into my "castle". In addition, that tool used symlinks, which doesn't work out well to manage a file that may get overwritten by a tool.

vcsh was the other main tool I tried. This tool adopts the bare repo technique, allowing files to be used instead of symlinks. The major drawback it had was that it was designed to work with multiple repos, and forces you to specify the repo with every command. Some simple aliasing helped mitigate that though. But by that time I also started writing some custom code to encrypt private data and was struggling to maintain different branches for different systems which had slight tweaks to configurations. It just became all fragmented and difficult to manage.

That is what lead me to create my own tool. The important features I wanted: portability, seamless interaction with Git, alternate files for different hosts, encryption for private data. I shared it as open source back in 2015 and it has grown in popularity some. It's been improved a lot over time including contributions from others. I'm glad so many others have found it useful.

I built milkman[1], an alternative to postman because I wanted one tool to integrate with the whole web development stack. One tool that contains my SQL, rest and grpc calls to easily debug issues. Also nearly none of those kind of tools that existed supported SSE which I also use in my day to day work.

[1] https://github.com/warmuuh/milkman

I wanted to easily deploy chrome's puppeteer scripts on AWS lambda. The goal was to alert me when a delivery slot was available on Bigbasket during peak covid.

Now I use the same to check if my websites are running properly :)

I call this project little lambda puppets and it deploys puppeteer script on aws lamba in one click and comes with inbuilt telegram api integration to send out alerts. It has served me well for almost over an year now.

(1) https://github.com/san-kumar/lambdapuppets

I created React Native styling library that support media queries and it even lets you create your own style rule.

https://github.com/anubhavgupta/react-native-styleman

Fromable[1] allows you take any Javascript iterable, transform it using .map and .filter, and then output it into a collection, without creating intermediate values.

[1] https://github.com/baron816/fromable

I created Lua with C/C++/Java/Javascript syntax https://github.com/mingodad/ljs

Also forked a scripting language https://github.com/mingodad/squilu and extended it to accept a pseudo C++/Java/CSharp/Typescript syntax to make easy to reuse code and have fast iteration when creating new functions (with it and minor changes I could get https://ssw.jku.at/Research/Projects/Coco/ parser to work for easy grammar development).

Create a tool for create, document and prototype database applications at "user level" in near realtime (live) https://github.com/mingodad/db-api-server.

Extended GLPK to accept a bigger subset of AMPL and made some performance improvements in the GMPL interpreter https://github.com/mingodad/GLPK

I created Lua with C/C++/Java/Javascript syntax https://github.com/mingodad/ljs

Also forked a scripting language https://github.com/mingodad/squilu and extended it to accept a pseudo C++/Java/CSharp/Typescript syntax to make easy to reuse code and have fast iteration when creating new functions (with it and minor changes I could get https://ssw.jku.at/Research/Projects/Coco/ parser to work for easy grammar development).

Create a tool for create, document and prototype database applications at "user level" in near realtime (live) https://github.com/mingodad/db-api-server.

Extended GLPK to accept a bigger subset of AMPL and made some performance improvements in the GMPL interpreter https://github.com/mingodad/GLPK

Currently:

AudioWrangler: https://apps.apple.com/us/app/audiowrangler/id1565701763

Got sick of my Mac using the wrong audio device when waking up, so I wrote a little utility that lets me prioritize which devices to use based on what's connected. It's my first app on the App Store. I'm enjoying learning Swift and using it to write desktop apps!

In the past:

* Lucid: https://github.com/mattgreen/lucid.fish

A pure-style prompt for fish shell that pioneered truly async fish prompts. Still use it daily.

* Fogeybot: https://github.com/mattgreen/fogeybot

A Discord bot for Heroes of the Storm that creates teams for pickup games using player ratings.

I built a C actor-like event library: https://github.com/FedeDP/libmodule

It actually has lots of improvements in the 6.0.0 pull request that i am delaying because i am too lazy (and haven't got much time) to fully rewrite doc.

I created Lua with C/C++/Java/Javascript syntax https://github.com/mingodad/ljs

Also forked a scripting language https://github.com/mingodad/squilu and extended it to accept a pseudo C++/Java/CSharp/Typescript syntax to make easy to reuse code and have fast iteration when creating new functions (with it and minor changes I could get https://ssw.jku.at/Research/Projects/Coco/ parser to work for easy grammar development).

Create a tool for create, document and prototype database applications at "user level" in near realtime (live) https://github.com/mingodad/db-api-server.

Extended GLPK to accept a bigger subset of AMPL and made some performance improvements in the GMPL interpreter https://github.com/mingodad/GLPK

Currently:

AudioWrangler: https://apps.apple.com/us/app/audiowrangler/id1565701763

Got sick of my Mac using the wrong audio device when waking up, so I wrote a little utility that lets me prioritize which devices to use based on what's connected. It's my first app on the App Store. I'm enjoying learning Swift and using it to write desktop apps!

In the past:

* Lucid: https://github.com/mattgreen/lucid.fish

A pure-style prompt for fish shell that pioneered truly async fish prompts. Still use it daily.

* Fogeybot: https://github.com/mattgreen/fogeybot

A Discord bot for Heroes of the Storm that creates teams for pickup games using player ratings.

I created Lua with C/C++/Java/Javascript syntax https://github.com/mingodad/ljs

Also forked a scripting language https://github.com/mingodad/squilu and extended it to accept a pseudo C++/Java/CSharp/Typescript syntax to make easy to reuse code and have fast iteration when creating new functions (with it and minor changes I could get https://ssw.jku.at/Research/Projects/Coco/ parser to work for easy grammar development).

Create a tool for create, document and prototype database applications at "user level" in near realtime (live) https://github.com/mingodad/db-api-server.

Extended GLPK to accept a bigger subset of AMPL and made some performance improvements in the GMPL interpreter https://github.com/mingodad/GLPK

I made a video input form: https://github.com/wgryc/video-input-js

The idea is that you have a form field-like object that you can record a response with. You can then submit the video via a regular POST request.

I couldn't find any simple tools like this for rapid prototyping different survey responses and experiences. It was fun to build and nice that some people have starred this on GitHub!

Elder.js, an SEO first, Svelte framework and Static Site Generator. [0]

Lots of SSGs existed but none supported Svelte and I wanted partial hydration when people said it was impossible.

[0]: https://elderguide.com/tech/elderjs/

At this point I've made a habit out of building homebrew tools and languages. Very few of these are purely because I was dissatisfied with off-the-shelf solutions; many of these just exist because I thought it would be fun/educational/challenging to build an X for myself from scratch.

I've made

- A dynamic programming language, Ink (https://dotink.co), which runs in "production" (for whatever that means for side projects) for around a dozen projects written in it.

- A compiler to compile that to JavaScript (https://github.com/thesephist/september)

- A bunch of language tooling around that language, like syntax highlighters, editor plugins, code formatters (for example, the code formatter https://github.com/thesephist/inkfmt)

- A small UI library (https://github.com/thesephist/torus)

- A suite of productivity tools (https://thesephist.com/posts/tools/) like notes, todos, shared whiteboard, contacts/CRM

- Twitter client (https://github.com/thesephist/lucerne/)

- Theres a few dozen more at (https://thesephist.com/projects/) :)

Many of these end up building on top of each other, so across the few dozen projects built on top of these tools they form a nice dependency graph -> https://twitter.com/thesephist/status/1367675987354251265

At this point I've made a habit out of building homebrew tools and languages. Very few of these are purely because I was dissatisfied with off-the-shelf solutions; many of these just exist because I thought it would be fun/educational/challenging to build an X for myself from scratch.

I've made

- A dynamic programming language, Ink (https://dotink.co), which runs in "production" (for whatever that means for side projects) for around a dozen projects written in it.

- A compiler to compile that to JavaScript (https://github.com/thesephist/september)

- A bunch of language tooling around that language, like syntax highlighters, editor plugins, code formatters (for example, the code formatter https://github.com/thesephist/inkfmt)

- A small UI library (https://github.com/thesephist/torus)

- A suite of productivity tools (https://thesephist.com/posts/tools/) like notes, todos, shared whiteboard, contacts/CRM

- Twitter client (https://github.com/thesephist/lucerne/)

- Theres a few dozen more at (https://thesephist.com/projects/) :)

Many of these end up building on top of each other, so across the few dozen projects built on top of these tools they form a nice dependency graph -> https://twitter.com/thesephist/status/1367675987354251265

At this point I've made a habit out of building homebrew tools and languages. Very few of these are purely because I was dissatisfied with off-the-shelf solutions; many of these just exist because I thought it would be fun/educational/challenging to build an X for myself from scratch.

I've made

- A dynamic programming language, Ink (https://dotink.co), which runs in "production" (for whatever that means for side projects) for around a dozen projects written in it.

- A compiler to compile that to JavaScript (https://github.com/thesephist/september)

- A bunch of language tooling around that language, like syntax highlighters, editor plugins, code formatters (for example, the code formatter https://github.com/thesephist/inkfmt)

- A small UI library (https://github.com/thesephist/torus)

- A suite of productivity tools (https://thesephist.com/posts/tools/) like notes, todos, shared whiteboard, contacts/CRM

- Twitter client (https://github.com/thesephist/lucerne/)

- Theres a few dozen more at (https://thesephist.com/projects/) :)

Many of these end up building on top of each other, so across the few dozen projects built on top of these tools they form a nice dependency graph -> https://twitter.com/thesephist/status/1367675987354251265

At this point I've made a habit out of building homebrew tools and languages. Very few of these are purely because I was dissatisfied with off-the-shelf solutions; many of these just exist because I thought it would be fun/educational/challenging to build an X for myself from scratch.

I've made

- A dynamic programming language, Ink (https://dotink.co), which runs in "production" (for whatever that means for side projects) for around a dozen projects written in it.

- A compiler to compile that to JavaScript (https://github.com/thesephist/september)

- A bunch of language tooling around that language, like syntax highlighters, editor plugins, code formatters (for example, the code formatter https://github.com/thesephist/inkfmt)

- A small UI library (https://github.com/thesephist/torus)

- A suite of productivity tools (https://thesephist.com/posts/tools/) like notes, todos, shared whiteboard, contacts/CRM

- Twitter client (https://github.com/thesephist/lucerne/)

- Theres a few dozen more at (https://thesephist.com/projects/) :)

Many of these end up building on top of each other, so across the few dozen projects built on top of these tools they form a nice dependency graph -> https://twitter.com/thesephist/status/1367675987354251265

I wrote a small library https://github.com/paulbares/spring-testcontainer that provides integration between the Spring (Data JPA) testing framework and Testcontainer library to easily test my application with different containerized database instances.

I built a website for making it easier to read Wayland protocols documentation (which are originally published as XML): https://wayland.app/protocols/

I wrote pandas2numpy [0] it helps you turn a pandas dataframe into one (or two if you have categorical variables) numpy tensors ready to be pushed into a deep learning framework.

It can deal with continuous data, categorical data, normalisation of the data, log scale and NA.

I found it really useful to work on tabular data and, since it uses numpy, it is agnostic to the deep-learning framework used.

[0]: https://github.com/nestordemeure/pandas2numpy

I created Ideogram.js, a JavaScript library for chromosome visualization [1]. Ideogram supports drawing and animating genome-wide datasets, enabling a variety of genomic views [3].

[1] https://github.com/eweitz/ideogram

[2] https://eweitz.github.io/ideogram

I built Shynet [0], a modern, privacy-friendly, and detailed web analytics tool that works without cookies or JS.

I also built a17t [1], an atomic web design toolkit that makes using Tailwind CSS a bit easier from the start.

[0] https://github.com/milesmcc/shynet

[1] https://github.com/milesmcc/a17t

I built Shynet [0], a modern, privacy-friendly, and detailed web analytics tool that works without cookies or JS.

I also built a17t [1], an atomic web design toolkit that makes using Tailwind CSS a bit easier from the start.

[0] https://github.com/milesmcc/shynet

[1] https://github.com/milesmcc/a17t

I've made a few things:

- Zola (https://github.com/getzola/zola) a SSG that works like I wanted and had a template engine close to Jinja2. I was using Hugo at the time but can't stand Golang template engine (imagine my sadness recently when I realised what Helm Charts are using...)

- Tera (https://github.com/keats/tera): a template engine pretty close to Jinja2/Django templates - at the time it was made only Handlebars existed in Rust and it was not enough for what I needed in Zola

- validator (https://github.com/Keats/validator): my take on Python marshmallow validations, nothing existed at the time.

So yeah, most of my open-source is re-creating libraries I use with Python in Rust.

A command line client for the firestore DB, because there was none:

https://github.com/sgarciac/fuego

I spent a much of 2020 developing a project that auto provisions REST, GraphQL & gRPC services that support CRUD operations (with full validation) to tables, views and materialized views of several popular databases (postgres, postgis, mysql, sqlite).

Features include: Full CRUD, geoquery support (bbox, radius, custom wkt polygon) complex_resources (aggregate & sub queries), middleware (access the query before db execution), permissions (table/view level CRUD configs), field redaction (enable query support -- without publication), schema migrations, auto generated openapi3/swagger docs, auto generated proto file, full validation.

GitHub https://github.com/sudowing/service-engine

YouTube Overview https://www.youtube.com/playlist?list=PLxiODQNSQfKOVmNZ1ZPXb...

I built a really simple cloud watchdog using only free-tier services:

http://www.watchdog.email/

https://github.com/jareklupinski/watchdog-email

planning on adding a timeout parameter soon!

I'm not the only developer but I made (and still make) GoJS, an interactive diagramming library for the web.

https://gojs.net

Most of its popularity is with large companies building their own internal tools where they need diagramming visualization capabilities (layouts, save/load, undo/redo, data binding, etc). So you won't see it too often in the wild, but under the surface it's used in every industry.

I love it, and I love HTML Canvas, which I wish was more popular.

I used to maintain a sprawling spreadsheet to manage my retirement investments. I would use it to compute the trades I'd need to do to rebalance (so I can stay on-target with my desired asset allocation).

I ended up making https://github.com/cjjeakle/rebalance-calc to do the same thing, but with a slicker user experience. I use it quarterly, so it definitely fits the bill for my use case, and I occasionally show the tool to friends to help explain what rebalancing is and how to do it.

I've written some pretty low-level OpenGL bindings in Rust, because the existing solutions are either too high level or over the top complexity wise.

At the time I only had experience with the OpenGL C api and didn't want to learn a totally different Rust abstraction or mix C/C++ libraries into a pure Rust project.

https://github.com/DomWilliams0/name-needed/tree/develop/ren...

I needed a good Node wrapper for ExifTool and wrote https://github.com/photostructure/exiftool-vendored.js

When I saw how slow it is to fork child processes in Windows, I then realized I wanted to run ExifTool in "stay-open" mode, which meant I needed to manage 1 or more long-lived child processes that communicate via stdin/stdout, so I wrote https://github.com/photostructure/batch-cluster.js

I also really missed scala's `lazy` operator, so I built that (and several other small, helpful functions/classes) that I documented here: https://photostructure.com/coding/uncertain-lazy-forgetful-a...

I needed a good Node wrapper for ExifTool and wrote https://github.com/photostructure/exiftool-vendored.js

When I saw how slow it is to fork child processes in Windows, I then realized I wanted to run ExifTool in "stay-open" mode, which meant I needed to manage 1 or more long-lived child processes that communicate via stdin/stdout, so I wrote https://github.com/photostructure/batch-cluster.js

I also really missed scala's `lazy` operator, so I built that (and several other small, helpful functions/classes) that I documented here: https://photostructure.com/coding/uncertain-lazy-forgetful-a...

I often have to perform statistical tests on genomic data. Which in practice means running the same test on every gene (row of a matrix). Running it separately on each row is often slow, specially in R. So to speed it up I started an R package "matrixTests": https://github.com/karoliskoncevicius/matrixTests which is a lot faster and deals nicely with edge case scenarios (missing values, infinities, empty matrices, etc).

Another notable mention is maybe vim plugin "sendtowindow": https://github.com/karoliskoncevicius/vim-sendtowindow which implements an operator for sending text to another window. Handy for sending code in a REPL within vim ":terminal" buffer.

I often have to perform statistical tests on genomic data. Which in practice means running the same test on every gene (row of a matrix). Running it separately on each row is often slow, specially in R. So to speed it up I started an R package "matrixTests": https://github.com/karoliskoncevicius/matrixTests which is a lot faster and deals nicely with edge case scenarios (missing values, infinities, empty matrices, etc).

Another notable mention is maybe vim plugin "sendtowindow": https://github.com/karoliskoncevicius/vim-sendtowindow which implements an operator for sending text to another window. Handy for sending code in a REPL within vim ":terminal" buffer.

https://github.com/alschwalm/transient - A tool for quickly creating short-lived virtual machines for doing kernel or hypervisor testing

Wrote an implementation of the ZIP specification in Swift: https://github.com/weichsel/ZIPFoundation

I made my own HTTP app. server and distributed JSON database: https://github.com/tinspin/rupy

On top of this I have made many projects and I still use all of them today even 10 years later, some favourite:

http://fuse.rupy.se (multiplayer system)

https://github.com/GuB-42/dupfiletree

A tool to find identical files.

Compared to other tools that do similar things, it can also find identical directories, handles archives and is designed to work with millions of files efficiently.

It works is two steps: first make a list of files and checksums with xmd5 and analyze it with dupfiletree.

Made for personal use, not documented, designed for performance first, stable enough for me but use at your own risk.

I started (but many many others contributed) to the Go implementation of WebRTC Pion WebRTC https://github.com/pion/webrtc

Along the way I worked on a Go implemenation of DTLS that gets used on its own a bit https://github.com/pion/dtls

In another life I wrote Therac https://github.com/sean-der/therac. A PHP debugger that was viewable from a browser. I was a remote at the time (2015) and it was a great tool to use with others.

I also wrote fail2web https://github.com/sean-der/fail2web a web frontend to fail2ban

I started (but many many others contributed) to the Go implementation of WebRTC Pion WebRTC https://github.com/pion/webrtc

Along the way I worked on a Go implemenation of DTLS that gets used on its own a bit https://github.com/pion/dtls

In another life I wrote Therac https://github.com/sean-der/therac. A PHP debugger that was viewable from a browser. I was a remote at the time (2015) and it was a great tool to use with others.

I also wrote fail2web https://github.com/sean-der/fail2web a web frontend to fail2ban

I started (but many many others contributed) to the Go implementation of WebRTC Pion WebRTC https://github.com/pion/webrtc

Along the way I worked on a Go implemenation of DTLS that gets used on its own a bit https://github.com/pion/dtls

In another life I wrote Therac https://github.com/sean-der/therac. A PHP debugger that was viewable from a browser. I was a remote at the time (2015) and it was a great tool to use with others.

I also wrote fail2web https://github.com/sean-der/fail2web a web frontend to fail2ban

I started (but many many others contributed) to the Go implementation of WebRTC Pion WebRTC https://github.com/pion/webrtc

Along the way I worked on a Go implemenation of DTLS that gets used on its own a bit https://github.com/pion/dtls

In another life I wrote Therac https://github.com/sean-der/therac. A PHP debugger that was viewable from a browser. I was a remote at the time (2015) and it was a great tool to use with others.

I also wrote fail2web https://github.com/sean-der/fail2web a web frontend to fail2ban

Recently started working on a command line tool to easily record things I learnt.

https://github.com/kamyar/til

Repoint, a thingy for checking out and versioning your source code dependencies - for situations where you don't have a good language-specific source package manager to hand.

https://thebreakfastpost.com/2018/05/18/repoint-a-manager-fo...

https://github.com/cannam/repoint

I wrote this, as you do, because I was slightly dissatisfied with all the options I could find that would work with the language and hosting combination I happened to be using. I use it for quite a number of projects now.

It's written in Standard ML, which might tickle a few HN readers.

I have 112 mostly-maintained released project listed on https://github.com/simonw/simonw/blob/main/releases.md now - most of which are tools for loading data into SQLite database files (a surprisingly under-served use-case given how ubiquitous SQLite is) and plugins for my https://datasette.io/ project for reading data back out of SQLite again.

I realized a few years ago that SQLite was the perfect tool for doing data analysis at the small-data scale, where small data is less that 10GB which is pretty much everything you might want to analyze - especially for personal projects.

So I've been building tools to fill that niche!

I have 112 mostly-maintained released project listed on https://github.com/simonw/simonw/blob/main/releases.md now - most of which are tools for loading data into SQLite database files (a surprisingly under-served use-case given how ubiquitous SQLite is) and plugins for my https://datasette.io/ project for reading data back out of SQLite again.

I realized a few years ago that SQLite was the perfect tool for doing data analysis at the small-data scale, where small data is less that 10GB which is pretty much everything you might want to analyze - especially for personal projects.

So I've been building tools to fill that niche!

I build a logging library for Go, because I couldn't find one that logs to stdout AND stderr. If you used a logging lib on GCP for example, all log output went into the same pile of junk and it was hard to find "real" errors: https://github.com/emvi/logbuch

Then there is "null", also because I couldn't find one that got both, marshalling to JSON and be able to store null values in db: https://github.com/emvi/null

And finally, our "flagship" open-source project Pirsch, an embedded library for web analytics: https://github.com/pirsch-analytics/pirsch

I build a logging library for Go, because I couldn't find one that logs to stdout AND stderr. If you used a logging lib on GCP for example, all log output went into the same pile of junk and it was hard to find "real" errors: https://github.com/emvi/logbuch

Then there is "null", also because I couldn't find one that got both, marshalling to JSON and be able to store null values in db: https://github.com/emvi/null

And finally, our "flagship" open-source project Pirsch, an embedded library for web analytics: https://github.com/pirsch-analytics/pirsch

I build a logging library for Go, because I couldn't find one that logs to stdout AND stderr. If you used a logging lib on GCP for example, all log output went into the same pile of junk and it was hard to find "real" errors: https://github.com/emvi/logbuch

Then there is "null", also because I couldn't find one that got both, marshalling to JSON and be able to store null values in db: https://github.com/emvi/null

And finally, our "flagship" open-source project Pirsch, an embedded library for web analytics: https://github.com/pirsch-analytics/pirsch

Dbmate: https://github.com/amacneil/dbmate

I found most database migration tools tried to be too smart (I'm not a fan of automatic migrations), were too tightly integrated with specific languages or frameworks, or didn't support basic developer workflow pleasantries such as dropping and recreating the database. Dbmate works with any language or framework, and generally tries to be useful while staying out of your way.

I created a simple single binary migration tool that runs migrations (sql files), and keeps track of applied migrations in a table (inspired by Active Record migrations).

Comparison against other tools here: https://github.com/amacneil/dbmate#alternatives

https://github.com/SonyMobile/py-hprof is a Python library that exposes a Java heap dump's objects as Python objects, allowing convenient interactive and/or scripted analysis.

IIRC, I started out with Android Studio's dump viewer, but I wanted to check a "nested" attribute (.x.y.z) for all instances of a class -- and there were several thousand instances. I quickly tired of expanding and scrolling the nested attribute lists.

I think I also tried MAT's OQL, but still missed the ability to add whatever logic I needed on-the-fly.

My "Show HN" post got no traction, and I don't think very many people have used it, but I'm still proud of it.

https://github.com/JaDogg/expressPython - goal was to learn some C++ and Python, I wanted to work on text input and wanted a UI.

https://github.com/JaDogg/sbx - flashcards console app (not sure if there are similar applications)

https://github.com/JaDogg/pydoro - pomodoro timer console app (I couldn't find such application)

https://github.com/JaDogg/expressPython - goal was to learn some C++ and Python, I wanted to work on text input and wanted a UI.

https://github.com/JaDogg/sbx - flashcards console app (not sure if there are similar applications)

https://github.com/JaDogg/pydoro - pomodoro timer console app (I couldn't find such application)

https://github.com/JaDogg/expressPython - goal was to learn some C++ and Python, I wanted to work on text input and wanted a UI.

https://github.com/JaDogg/sbx - flashcards console app (not sure if there are similar applications)

https://github.com/JaDogg/pydoro - pomodoro timer console app (I couldn't find such application)

Would a browser-based voxel game engine count?

https://github.com/andyhall/noa

The reason for not using an existing library is just that there weren't any usable alternatives at the time!

Curious about the lack of comparison to https://flywaydb.org/ , which I've seen a ton of support for in the past and people seem to really enjoy

I created a CMake package that autogenerates Python bindings for annotated C++ functions.

If you have a C++ static lib in CMake, it takes two more lines of CMake to generate a Python extension module, and a single line of annotation per C++ function that you want to export.

https://github.com/bebump/bbmp_interop

I use it mostly for testing numerical C++ code using Numpy and Scipy.

A silly userscript [1] that uses jQuery selectors to interact with YouTube's DOM to help me move videos around my playlists. Sadly it breaks every few months whenever YouTube updates their DOM. AFIK this is the only way to "programmatically" move videos out of my Watch Later list since they removed the functionality from the API

[1] https://github.com/Trinovantes/userscript-youtube-playlist-o...

A silly userscript [1] that uses jQuery selectors to interact with YouTube's DOM to help me move videos around my playlists. Sadly it breaks every few months whenever YouTube updates their DOM. AFIK this is the only way to "programmatically" move videos out of my Watch Later list since they removed the functionality from the API

[1] https://github.com/Trinovantes/userscript-youtube-playlist-o...

A CLI tool for downloading the Java Specification Request (JSR) of a given Java package:

https://github.com/silb/jsrlib

I found downloading the JSRs manually too cumbersome.

It is acompanied by an Eclipse plugin that i never got around to publish on GitHub. It lets me right click a Java type in Eclipse and open its JSR PDF file.

sqldb-logger: https://github.com/simukti/sqldb-logger

It's a thin layer that wrap Go (Golang) SQL database driver, it does not change sql.DB usage in any existing Go app.

It is because I prefer to use vanilla Go sql.DB in my web service and I could not find any SQL logger for Go SQL database standard library that can use structured JSON logging.

I built an open source library for Android called Showkase [1]. Showkase autogenerates a UI component browser for your Jetpack Compose components on Android and allows you to search, visualize and organize your UI elements. Think Storybook but for Android development.

[1] https://github.com/airbnb/Showkase

Night Patrol - https://github.com/jahed/night-patrol

I got tired of passing flags to Nightwatch to filter tests and looking through results to re-run failures. So I made an interactive CLI for all of that. I stopped using this once I moved to Cypress but even with Cypress' web interface, I feel something like Night Patrol would be a lot more productive.

Firebase Rules - https://github.com/jahed/firebase-rules

I didn't like how Firebase's RTDB rules were expressions in JSON strings so I built a library to build them using a lisp-like syntax to compose and re-use rules. Firebase also has its own rules language (Bolt) but I'd rather not learn and maintain yet another tool-specific language.

Promises - https://github.com/jahed/promises

This isn't a complete solution yet, I didn't get time to refine the API. But I really prefer using Promises as result types (a.k.a. either, left/right) rather than using async/await/try/catch. It's so much more powerful to write code in a way that doesn't care if your functions are async or sync. Again, kind of like a lisp, there's also power in not being tied to a language's keywords and being able to provide a more tailored vocabulary.

Night Patrol - https://github.com/jahed/night-patrol

I got tired of passing flags to Nightwatch to filter tests and looking through results to re-run failures. So I made an interactive CLI for all of that. I stopped using this once I moved to Cypress but even with Cypress' web interface, I feel something like Night Patrol would be a lot more productive.

Firebase Rules - https://github.com/jahed/firebase-rules

I didn't like how Firebase's RTDB rules were expressions in JSON strings so I built a library to build them using a lisp-like syntax to compose and re-use rules. Firebase also has its own rules language (Bolt) but I'd rather not learn and maintain yet another tool-specific language.

Promises - https://github.com/jahed/promises

This isn't a complete solution yet, I didn't get time to refine the API. But I really prefer using Promises as result types (a.k.a. either, left/right) rather than using async/await/try/catch. It's so much more powerful to write code in a way that doesn't care if your functions are async or sync. Again, kind of like a lisp, there's also power in not being tied to a language's keywords and being able to provide a more tailored vocabulary.

Night Patrol - https://github.com/jahed/night-patrol

I got tired of passing flags to Nightwatch to filter tests and looking through results to re-run failures. So I made an interactive CLI for all of that. I stopped using this once I moved to Cypress but even with Cypress' web interface, I feel something like Night Patrol would be a lot more productive.

Firebase Rules - https://github.com/jahed/firebase-rules

I didn't like how Firebase's RTDB rules were expressions in JSON strings so I built a library to build them using a lisp-like syntax to compose and re-use rules. Firebase also has its own rules language (Bolt) but I'd rather not learn and maintain yet another tool-specific language.

Promises - https://github.com/jahed/promises

This isn't a complete solution yet, I didn't get time to refine the API. But I really prefer using Promises as result types (a.k.a. either, left/right) rather than using async/await/try/catch. It's so much more powerful to write code in a way that doesn't care if your functions are async or sync. Again, kind of like a lisp, there's also power in not being tied to a language's keywords and being able to provide a more tailored vocabulary.

I'm building a turn-based strategy game for iOS (in Swift).

Just extracted the Monte Carlo Search Tree logic into a package called Carlo [1]!

[1] https://github.com/maxhumber/Carlo

I've been working on gazpacho [1] for last two years.

It's a general purpose web scraping library for Python that replaces BeautifulSoup + requests for most projects.

Just surpassed ~2K downloads every week!

[1] https://github.com/maxhumber/gazpacho

I struggled to find a job board that was not filled with shabby recruitment agencies, that had clear high quality job postings and clear salary ranges in each job opening. I therefore created the job board and it now includes a directory for Go developers, Companies using Go and a salary trends section. It's all been written in pure Go, basic HTML/JS/CSS and PostgreSQL and it's open source :)

- Golang Cafe

- https://golang.cafe

- https://github.com/golang-cafe/golang.cafe

Vaku - A CLI for Vault that lets you operate on folders instead of just paths. Search, copy, move, read vault folders easily.

https://github.com/lingrino/vaku

I created TopShell - a purely functional, reactive scripting language for fetch/process/visualize style tasks: https://github.com/topshell-language/topshell

PDFVUER - https://github.com/arkokoley/pdfvuer

I also built a time tracker library that tracks how much time was spent on the current open tab. It also detects when the user moves away from the page or comes back.

Actively - https://github.com/arkokoley/actively

For a GIS project, I needed to extract county level data from NASA's MODIS dataset. Wrote a bash utility to do that:

PDFVUER - https://github.com/arkokoley/pdfvuer

I also built a time tracker library that tracks how much time was spent on the current open tab. It also detects when the user moves away from the page or comes back.

Actively - https://github.com/arkokoley/actively

For a GIS project, I needed to extract county level data from NASA's MODIS dataset. Wrote a bash utility to do that:

njk - https://github.com/mohitsinghs/njk

Annoyed by amount of tooling required to render markdown, nunjucks and front-matter, I wrote this cli and used for all of my static sites. Since then, I’ve mostly migrated my sites to nextjs.

I have a python library that performs dice rolls: https://github.com/the-nick-of-time/dndice

Basically just made for fun.

I've created the Automunge python library to automate tabular data cleaning for ML, including preprocessing transformations, missing data infill, etc. It's very useful!

https://github.com/Automunge/AutoMunge

https://github.com/agentultra/postgresql-replicant — stream delta changes from a PostgreSQL database. Not a novel concept or implementation but this library doesn’t exist in Haskell and that’s a shame because Haskell has lots of great libraries for working with streams.

https://github.com/agent ultra/DataVersion — type-directed data migration library. I needed to migrate some data in a document database that had no schema with years of data in it to a format that had a schema. I leveraged some other stuff to achieve it but this library was a key piece to being confident that the transformations were correct and complete. Migrated the database and backfill it with a Lambda function using this library.

Those are just the most recent ones.

Working on building userTrack[0] I always encountered the need of various auxiliary tools. I had to implement a custom deploy system to build different variants of the product, to create a licensing server (to create and verify license codes and to allow downloads for valid license owners), a blogging platform[1], some JS snippets [2], etc.

Most of the times, the libraries/tools that you build yourself are either to connect and interact with a specific external service OR to have a simpler (only the features you need) or cheaper version of an existing product/platform.

[0]: https://www.usertrack.net/

[1]: https://github.com/Cristy94/markdown-blog

[2]: https://github.com/Cristy94/dynamic-listener

I find most Continuous Delivery tools/services overkill most of the time, so I wrote (and heavily use, even in enterprise setups) my own tool „SimpleCD“, which is just a single Bash script, but is extremely powerful and flexible: https://github.com/manuelkiessling/simplecd

I needed a simple command line tool that could tell me how far apart two post codes in North America are, so I built one. You could in theory load it as a library too.

https://github.com/jedberg/Postcodes

A curious question. Aside from my day job, this seems like a thing I do all day. :-)

I'm not sure what the motivation for your question is. Do you feel like everything's been invented and built already, and it's just a matter of (at most) plugging the things together?

I find myself constantly thinking "this should exist". I don't have time to make them all exist.

https://github.com/ThomasHabets/arping

Nothing like it existed at the time, and I wanted to send ARP requests as easily as sending ICMP ping.

https://github.com/ThomasHabets/simple-tpm-pk11

I wanted to use a TPM chip for SSH client keys, and couldn't find anything like it.

https://github.com/ThomasHabets/tlssh

I wanted to explore what it would be like to have SSH, but with identities not based on providing username, but an x509 cert. (and TPM chip protecting the key)

A curious question. Aside from my day job, this seems like a thing I do all day. :-)

I'm not sure what the motivation for your question is. Do you feel like everything's been invented and built already, and it's just a matter of (at most) plugging the things together?

I find myself constantly thinking "this should exist". I don't have time to make them all exist.

https://github.com/ThomasHabets/arping

Nothing like it existed at the time, and I wanted to send ARP requests as easily as sending ICMP ping.

https://github.com/ThomasHabets/simple-tpm-pk11

I wanted to use a TPM chip for SSH client keys, and couldn't find anything like it.

https://github.com/ThomasHabets/tlssh

I wanted to explore what it would be like to have SSH, but with identities not based on providing username, but an x509 cert. (and TPM chip protecting the key)

A curious question. Aside from my day job, this seems like a thing I do all day. :-)

I'm not sure what the motivation for your question is. Do you feel like everything's been invented and built already, and it's just a matter of (at most) plugging the things together?

I find myself constantly thinking "this should exist". I don't have time to make them all exist.

https://github.com/ThomasHabets/arping

Nothing like it existed at the time, and I wanted to send ARP requests as easily as sending ICMP ping.

https://github.com/ThomasHabets/simple-tpm-pk11

I wanted to use a TPM chip for SSH client keys, and couldn't find anything like it.

https://github.com/ThomasHabets/tlssh

I wanted to explore what it would be like to have SSH, but with identities not based on providing username, but an x509 cert. (and TPM chip protecting the key)

I built daria (https://github.com/MrPowers/spark-daria) to make it easier to write Spark and spark-fast-tests (https://github.com/MrPowers/spark-fast-tests) to provide a good testing workflow.

quinn (https://github.com/MrPowers/quinn) and chispa (https://github.com/MrPowers/chispa) are the PySpark equivalents.

Built bebe (https://github.com/MrPowers/bebe) to expose the Spark Catalyst expressions that aren't exposed to the Scala / Python APIs.

Also build spark-sbt.g8 to create a Spark project with a single command: https://github.com/MrPowers/spark-sbt.g8

I built daria (https://github.com/MrPowers/spark-daria) to make it easier to write Spark and spark-fast-tests (https://github.com/MrPowers/spark-fast-tests) to provide a good testing workflow.

quinn (https://github.com/MrPowers/quinn) and chispa (https://github.com/MrPowers/chispa) are the PySpark equivalents.

Built bebe (https://github.com/MrPowers/bebe) to expose the Spark Catalyst expressions that aren't exposed to the Scala / Python APIs.

Also build spark-sbt.g8 to create a Spark project with a single command: https://github.com/MrPowers/spark-sbt.g8

I built daria (https://github.com/MrPowers/spark-daria) to make it easier to write Spark and spark-fast-tests (https://github.com/MrPowers/spark-fast-tests) to provide a good testing workflow.

quinn (https://github.com/MrPowers/quinn) and chispa (https://github.com/MrPowers/chispa) are the PySpark equivalents.

Built bebe (https://github.com/MrPowers/bebe) to expose the Spark Catalyst expressions that aren't exposed to the Scala / Python APIs.

Also build spark-sbt.g8 to create a Spark project with a single command: https://github.com/MrPowers/spark-sbt.g8

I built daria (https://github.com/MrPowers/spark-daria) to make it easier to write Spark and spark-fast-tests (https://github.com/MrPowers/spark-fast-tests) to provide a good testing workflow.

quinn (https://github.com/MrPowers/quinn) and chispa (https://github.com/MrPowers/chispa) are the PySpark equivalents.

Built bebe (https://github.com/MrPowers/bebe) to expose the Spark Catalyst expressions that aren't exposed to the Scala / Python APIs.

Also build spark-sbt.g8 to create a Spark project with a single command: https://github.com/MrPowers/spark-sbt.g8

I built daria (https://github.com/MrPowers/spark-daria) to make it easier to write Spark and spark-fast-tests (https://github.com/MrPowers/spark-fast-tests) to provide a good testing workflow.

quinn (https://github.com/MrPowers/quinn) and chispa (https://github.com/MrPowers/chispa) are the PySpark equivalents.

Built bebe (https://github.com/MrPowers/bebe) to expose the Spark Catalyst expressions that aren't exposed to the Scala / Python APIs.

Also build spark-sbt.g8 to create a Spark project with a single command: https://github.com/MrPowers/spark-sbt.g8

I built daria (https://github.com/MrPowers/spark-daria) to make it easier to write Spark and spark-fast-tests (https://github.com/MrPowers/spark-fast-tests) to provide a good testing workflow.

quinn (https://github.com/MrPowers/quinn) and chispa (https://github.com/MrPowers/chispa) are the PySpark equivalents.

Built bebe (https://github.com/MrPowers/bebe) to expose the Spark Catalyst expressions that aren't exposed to the Scala / Python APIs.

Also build spark-sbt.g8 to create a Spark project with a single command: https://github.com/MrPowers/spark-sbt.g8

https://github.com/kmurph73/set-state-is-great

Dumping mysql to bigquery (gcp). The current tool doesn't handle large files. It was accepted after a year.

https://github.com/james-ransom/mysql-to-google-bigquery

I've built with a collegue Continuum: https://github.com/Continvvm/continuum

It's a library giving utilities to handle data in Continual Learning, a subfield of Machine/Deep-learning where data comes incrementally.

There are others actors in the field that came atfer us and do more ads about their slightly similar library, but we manage to produce something useful to many PhD students, so that's a win for me.

Years back I made a circular progress bar library for Android. Nothing special but it was something that was missing at that time. Surprisingly, 1000+ downloads monthly (according to JitPack) and I still get occasional issues reported on GitHub.

The lib - https://github.com/krtkush/LinearTimer

My company does most of our development in Delphi, but unfortunately that means that crash dumps are very difficult or impossible to utilize with the embedded Borland/Embarcadero TDS debugging information. I wrote a utility that extracts the TDS info from a Delphi executable and uses an LLVM debug info library to write out a PDB and also modify the executable to point to that PDB.

The source can be found here: https://github.com/powerworld/TDSToPDB

1. React Leaflet canvas overlay. Leaflet is a popular mapping library and React Leaflet brings the declarative nature of React to the Leaflet api. I needed to display some 3D rendered shadows on a map and needed to overlay a canvas at specific lat/lng bounds so I made a React component to plug into React Leaflet.

2. I needed a solution for recording Twilio live video and hosting it myself in S3. I made a docker container that launches Chrome headless, points it at any website (including a Twilio video conference) and uses and Chrome extension to record the screen and stream it to S3. In theory it could be used to record any website on the internet.

[1] https://github.com/ted-piotrowski/react-leaflet-canvas-overl...

1. React Leaflet canvas overlay. Leaflet is a popular mapping library and React Leaflet brings the declarative nature of React to the Leaflet api. I needed to display some 3D rendered shadows on a map and needed to overlay a canvas at specific lat/lng bounds so I made a React component to plug into React Leaflet.

2. I needed a solution for recording Twilio live video and hosting it myself in S3. I made a docker container that launches Chrome headless, points it at any website (including a Twilio video conference) and uses and Chrome extension to record the screen and stream it to S3. In theory it could be used to record any website on the internet.

[1] https://github.com/ted-piotrowski/react-leaflet-canvas-overl...

I built Dud[1] because I wanted a simpler, faster DVC[2]. If DVC is Django, I set out to build Flask. In using Dud myself, I think I've succeeded thus far. But the only way to know for sure is to publicize the project and get it in other people's hands. Here's to that.

[1]: https://github.com/kevin-hanselman/dud

[2]: https://dvc.org/

I built Dud[1] because I wanted a simpler, faster DVC[2]. If DVC is Django, I set out to build Flask. In using Dud myself, I think I've succeeded thus far. But the only way to know for sure is to publicize the project and get it in other people's hands. Here's to that.

[1]: https://github.com/kevin-hanselman/dud

[2]: https://dvc.org/

- https://github.com/soheilpro/zsh-vi-search - Adds support for searching the current line (in normal vi - https://github.com/soheilpro/catj - A better way to display JSON files

- https://github.com/soheilpro/zsh-vi-search - Adds support for searching the current line (in normal vi - https://github.com/soheilpro/catj - A better way to display JSON files

- https://github.com/soheilpro/sqlmon - Collect events from SQL Server and save them to Elasticsearch

In 2016 I was unsatisfied with the QR Code generator libraries available in the wild, especially for C, C++, and JavaScript. Hence I wrote my own: https://www.nayuki.io/page/qr-code-generator-library ; https://github.com/nayuki/QR-Code-generator

I also made some visualizations into the internals of QR Codes: https://www.nayuki.io/page/creating-a-qr-code-step-by-step ; https://www.nayuki.io/page/optimal-text-segmentation-for-qr-...

Last year I made Kondo, to clean up all the dependencies files from software projects you're not actively working on (eg node_modules). Awesome if you about to Zip up a bunch of old projects for archiving, or just want to reclaim disk space without deleting anything of value.

It was an experiment in Rust, and I even made a little GUI too!

https://github.com/tbillington/kondo