Programming Breakthroughs We Need

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • Appwrite - The Open Source Firebase alternative introduces iOS support
  • Scout APM - Truly a developer’s best friend
  • SonarQube - Static code analysis for 29 languages.
  • talent.io - Download talent.io’s Tech Salary Report
  • infer

    A static analyzer for Java, C, C++, and Objective-C

    > Maybe you could write tests as queries that would test a whole set of possible programs, not only the current version of your program at the moment.

    I think that the future of programming is more sophisticated static analysis. Programmers will write statements like, "every code path that writes to the Payments database must have called validate_user()." Then, the tooling will confirm that rule with every commit.

    We kind of have this already (for example, Facebook's Infer tool [0]), but I think it will become much more important in the coming decade.

    0: https://github.com/facebook/infer

  • foam3

    FOAM: Feature-Oriented Active Modeller, Version 3 (unstable)

    https://github.com/kgrgreer/foam3#videos

    FOAM is a modelling framework that generates cross-language boilerplate for you, but it takes a much broader view of what constitutes boilerplate than most systems. Typically, it can generate between 95-98% of a working cross-language cross-tier system.

    FOAM helps you create features for modelled data. Features include things like a Java/Javascript/Swift classes to hold your modelled data, code to marshall to/from JSON/XML/CSV/etc., various GUI Views, and support for storing your data in various databases or file formats. However, FOAM models are themselves modelled, meaning they're afforded all of the above benefits as well. This lets you apply the MVC technique of having multiple views work against the same underlying data-model concurrently (say a grid and a pie-chart in a spreadsheet), so that you can choose the best view or views for your current need. When treated this way, your code is no longer text (but it can be, if that's one of your views), and you can easily view and store it in many different ways and more easily programmatically manipulate it.

  • Appwrite

    Appwrite - The Open Source Firebase alternative introduces iOS support. Appwrite is an open source backend server that helps you build native iOS applications much faster with realtime APIs for authentication, databases, files storage, cloud functions and much more!

  • TreeHouse

    A structure built in trees of abstract syntax - for habitation (by mikewarot)

  • spotless

    Keep your code spotless

    Some code formatters such as Spotless (https://github.com/diffplug/spotless/tree/main/plugin-gradle...) allow you to format code only in files that have changes against some designated branch such as `master`. So, you check out your feature branch, make changes, do some commits, and run spotless. Only the files which have some changes between your workspace and the master branch will be formatted. This allows you to gradually format the project as and when files would be changed anyways.

  • MonkeyType

    A Python library that generates static type annotations by collecting runtime types (by Instagram)

    https://github.com/instagram/MonkeyType can perform the call logging, and can export a static typing file which is used by mypy, but also e.g. PyCharm. It doesn't expose such fine grained types, but you could build that based on the logged data.

  • wasmtalk

    Personal WebAssembly learning project (build a SmallTalk like environment but with WASM)

    I've had very similar thoughts and have wrote about them here. I would be interested in discussing more.

    https://github.com/brennancheung/wasmtalk

    Some key takeaways from the above link:

    - The programmer's tool should be a tool for manipulating an annotated AST (not text)

    - There should be many different types of UX's for different scenarios, each maps to and from an AST in a UX that is optimal for the developer for that scenario

    - We must be conscious of human brain limitations and cognitive psychology and work within those constraints

    - "Reading" and "Writing" code should have different UX's because they are radically different use cases

    - Use RPN. It models the real world. Humans are designed to manipulate their environment in an incremental manner seeing the result each step of the way. When we have to plan out and write code for an extended period of time, trying to play compiler in our head, we overload our brain unnecessarily and highly likely to make simple mistakes.

    - Testing should be a first class citizen in the developer experience and indeed baked into how we develop at a fundamental level that it seems strange that they are even decoupled to begin with.

  • pharo

    The Sources for Pharo

  • Scout APM

    Truly a developer’s best friend. Scout APM is great for developers who want to find and fix performance issues in their applications. With Scout, we'll take care of the bugs so you can focus on building great things 🚀.

  • dylint

    A tool for running Rust lints from dynamic libraries

    RE: Program is a model

    There are some more advanced refactoring tools now available. These tools enable you to write code to detect bad code patterns and even automatically fix them. You can use them to write one-off transformations of code too. Rust has Dylint [1] and C# has Roslyn Analyzers [2]. Facebook has tooling [3] that helps writing CodeMods, enabling authors to generate changes for thousands of files at a time.

    The thing I really would like to see is a smarter CI system. Caching of build outputs, so you don't have to rebuild the world from scratch every time. Distributed execution of tests and compilation, so you are not bottle-necked by one machine. Something that keeps track of which tests are flaky and which are broken on master, so you don't have to diagnose spurious build failures. Something that only runs the test that transitively depend on the code you change. Automatic bisecting of errors to the offending commit.

    [1] https://github.com/trailofbits/dylint

    [2] https://docs.microsoft.com/visualstudio/code-quality/roslyn-...

    [3] one example: https://github.com/facebook/jscodeshift

  • jscodeshift

    A JavaScript codemod toolkit.

    RE: Program is a model

    There are some more advanced refactoring tools now available. These tools enable you to write code to detect bad code patterns and even automatically fix them. You can use them to write one-off transformations of code too. Rust has Dylint [1] and C# has Roslyn Analyzers [2]. Facebook has tooling [3] that helps writing CodeMods, enabling authors to generate changes for thousands of files at a time.

    The thing I really would like to see is a smarter CI system. Caching of build outputs, so you don't have to rebuild the world from scratch every time. Distributed execution of tests and compilation, so you are not bottle-necked by one machine. Something that keeps track of which tests are flaky and which are broken on master, so you don't have to diagnose spurious build failures. Something that only runs the test that transitively depend on the code you change. Automatic bisecting of errors to the offending commit.

    [1] https://github.com/trailofbits/dylint

    [2] https://docs.microsoft.com/visualstudio/code-quality/roslyn-...

    [3] one example: https://github.com/facebook/jscodeshift

  • remote-apis

    An API for caching and execution of actions on a remote system.

    > The thing I really would like to see is a smarter CI system. Caching of build outputs, so you don't have to rebuild the world from scratch every time. Distributed execution of tests and compilation, so you are not bottle-necked by one machine.

    This is already achievable nowadays using Bazel (https://bazel.build) as a build system. It uses a gRPC based protocol for offloading/caching the actual build on a build cluster (https://github.com/bazelbuild/remote-apis). I am the author of one of the Open Source build cluster implementations (Buildbarn).

  • Bazel

    a fast, scalable, multi-language and extensible build system

    > The thing I really would like to see is a smarter CI system. Caching of build outputs, so you don't have to rebuild the world from scratch every time. Distributed execution of tests and compilation, so you are not bottle-necked by one machine.

    This is already achievable nowadays using Bazel (https://bazel.build) as a build system. It uses a gRPC based protocol for offloading/caching the actual build on a build cluster (https://github.com/bazelbuild/remote-apis). I am the author of one of the Open Source build cluster implementations (Buildbarn).

  • c4-notation

    Technical resources for using the C4 model for visualizing software architecture.

    There's a hierarchical modeling paradigm/tools called C4 that (while being boxes and lines) helps with the zoom-in/zoom-out nature of understanding systems: https://c4model.com/

  • MyDef

    Programming in the next paradigm -- your way

    For those who are interested, checkout MyDef - https://github.com/hzhou/MyDef

  • pbrt-v3

    Source code for pbrt, the renderer described in the third edition of "Physically Based Rendering: From Theory To Implementation", by Matt Pharr, Wenzel Jakob, and Greg Humphreys.

    Very nice article. Literate programming might have some interesting features in this direction. The not so obvious thing is how it can scale.

    A very good example of literate programming can be found in the fantastic book Physically Based Rendering [1]. It also has a preface discussing it [2]

    The authors write a full physically-based rendering system in a declarative fashions. This approach has also led to a nicely adapted web-based version of the book [3].

    Although we have successful examples like TeX and HTML, it would be very interesting to see this approach explored further by a more modern language or programming methodology.

    [1] https://pbrt.org/

  • Sourcetrail

    Sourcetrail - free and open-source interactive source explorer

    Sourcetrail actually tried to do that for a select few languages https://github.com/CoatiSoftware/Sourcetrail

    Sadly, they retired the entire project a while back.

  • gui-thunks

    how to create GUIs that queue

    I spend everyday thinking of what my computer could be doing.

    Most of the time the CPU is waiting for IO - memory, network, SSD, disk and not doing any work.

    You might like my idea called GUI Thunking.

    https://github.com/samsquire/gui-thunks

  • language-server-protocol

    Defines a common protocol for language servers.

    >IntelliJ helps you with things like "extract parameter" or "inline function", but I wanted to do stuff like...

    In the same vein as the developments around LSP, TreeSitter, etc. I had the idea that one could create a scripting language dedicated to refactors. The language would make it simple to talk about the AST of your programming language and provide simple primitives for safely transforming it.

    Doing this in a language-agnostic way is pretty difficult, though. There seems to be some discussion among the LSP folks here (https://github.com/microsoft/language-server-protocol/issues...) but no resolution. It's worth thinking about!

  • SonarQube

    Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts