State of the Common Lisp ecosystem, 2020

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SaaSHub - Software Alternatives and Reviews
  • generic-cl

    Generic function interface to standard Common Lisp functions

  • > Common Lisp's worst feature is that it feels like just about everything imaginable has a catch.

    You've put into words something I've struggled with saying for a while, and the exact reason I could never really enjoy working with Common Lisp.

    The example I always return to is the lack of pervasive use of CLOS throughout the standard. Something like generic-cl[0] feels like it should be a baseline part of the language. As it is, it's a great library. But people are hesitant to even use it, because the result of making everything generic is a loss in performance!

    Maybe you could use something like [1] to regain it, but to my knowledge there's no easy distribution to simply import and have it immediately usable. All this results in that, as you geniusly put it, "bad mouthfeel".

    [0] https://github.com/alex-gutev/generic-cl

    [1] https://github.com/marcoheisig/fast-generic-functions

  • magicl

    Matrix Algebra proGrams In Common Lisp.

  • I think this article (of sorts) is definitely helpful for onlookers to Common Lisp, but doesn't provide the full "story" or "feel" of Common Lisp, and I want to offer to HN my own perspective.

    Disclaimer #1: I've been working professionally as a Common Lisp programmer---not as a contractor!---for the past decade. I have a vested interest in the language and hiring for it.

    Disclaimer #2: I am going to ignore commercial implementations of Lisp here, which provide very useful and advanced features, like GUI development, a user-friendly IDE, paid support, etc. [1,2]

    So let's get started. Common Lisp's best feature is that it allows you to be insanely productive at the "raw programmer" level. You can write, edit, and debug code very quickly and incrementally, and end up with safe & performant code.

    There's a price to pay: currently the best-in-class experience is still Emacs and SLIME (which come nicely packaged here [3]). As an Emacs fan, that's the best news, but to my fellow PyCharm/VSCode/vim users, it's terrible and alienating news. My colleagues who aren't Emacs users managed to learn just enough Emacs to be productive in a week, but they still frequently fired up their editor of choice in times of need.

    It really is worth underscoring that the Emacs+SLIME experience truly fits Common Lisp development like a glove, and the experience, in my opinion, is better than almost every mainstream editor environment out there.

    Common Lisp's worst feature is that it feels like just about everything imaginable has a catch. I don't mean "there's no free lunch", I mean that things just plainly don't feel 100% most of the time. To name a few examples:

    1. GUIs: If you want to make a traditional, native GUI using open source solutions, you're stuck with really goofy libraries that are non-obvious to get working. As the article points out, you have options. Lisp actually has a renowned framework called CLIM, but I consider the open-source variant McCLIM [ currently only principally useful to hobbyists and hackers.

    2. Deploying applications: Almost every implementation of Lisp has some way to create an executable. But very important aspects that real people care about in production are missing, inconsistent, or poorly documented. For example, almost no open source implementations of Lisp have first-class support for signing binaries on MacOS. Almost no open source implementations have a "tree shaker" to remove unnecessary cruft from the executable. Almost no open source implementations make building a shared library practical.

    3. Libraries: Many libraries don't do even usual things people might want to do. The linear algebra library MAGICL [8], for example, doesn't at the time of writing have a way to solve the matrix equation Ax=B. This isn't due to laziness of the authors or lack of foresight, but rather that it's a library that's just not used by enough people to see regular, high-quality contributions as an open-source project. I'm sure MAGICL solves problems for the authors, but the authors haven't taken it upon themselves to make a general, useful, and quasi-complete library for matrix programming in Lisp.

    These examples are just examples, maybe not even the top examples.

    One thing that I constantly hope for in the Lisp world is for more "hyper productive" programmers to join it, or programmers whose livelihood depends on it. Of course, since Lisp is used by hobbyists, you see tons of hobbyist code. To be sure, a lot of this hobbyist code is perfectly fine. Usually it works, but it's just a tad incomplete. However, in my opinion, the worst thing about hobbyist code is that it usually doesn't do something useful.

    What does "useful" even mean? I won't claim to be able to define this term in a one-size-fits-all fashion, but "useful" to me is about getting practical computing work done. The further away from being concrete the library is, typically the less useful it is. For example, a typical Lisp programmer will have a penchant for writing a domain-specific language for parsing binary files (cool!), will open-source that code (cool!), but then nobody---including the author of said library---will actually use it to, say, write a parser for GIFs [5]. In Lisp, it seems popular to solve meta-problems instead of problems, which is partly due to the fact that Lisp lets you think about problems at very high levels of abstraction using its advanced object system, the meta-object protocol, and macros.

    (One of my biggest "pet peeve" projects in Lisp, second only to "utility libraries", are documentation generator libraries. As soon as somebody figures out that documentation strings can actually be programmatically queried in Lisp, they invariably write a baroque "generator" that spits out HTML. I've never, not a single time, ever, used a documentation generator for doing real, paid work. I think one Lisp programmer I know uses it nicely is Nicolas Hafner, aka Shinmera, who uses a documentation generator simply to augment his long-form documentation writing. Staple [9] is one example library of his, where you can see some generated documentation at the bottom.)

    "Useful" also has to do with how a library is consumed. In the Common Lisp, a library like this [6] is typical. It's a bare page (be it on GitHub or otherwise) that provides no examples, no indication of dependencies, etc. Not all libraries are like this, but you run into it frequently enough.

    The Common Lisp ecosystem lacks a certain "go-getter" philosophy, needed to forge through "boring" work, that some other language ecosystems seem to have. To cherry pick one example, though I don't use it, Zig [7] comes out with interesting stuff all the time that's genuinely useful. Andrew Kelly, it's main developer, is putting tons of hours into getting details around deployment right (e.g., cross-compilation). Little about Common Lisp prevents a motivated person from making equally productive-enhancing strides with the language, but I find that either (a) the interest isn't there or (b) the interest is there but the interest is for developing weird, esoteric stuff in Lisp.

    (My favorite example of a "productive stride" that happened in Lisp is the following. For context, people talk about all the time how difficult it would be to port a Lisp compiler to a new architecture. I myself have clamored for documentation on how to do it with SBCL. But, out of nowhere, some grad student named Charles Zhang came out with a port of SBCL to RISC-V. Not only did he port it, he's maintained it with 100s of new commits, making it more performant and less buggy [10].)

    Common Lisp is an amazing language purely from a practical point-of-view. As I said, to me, it's bar-none the best and most productive language to use if you want to "sit down and write code". The implementations of Lisp, like SBCL, are marvels. Lisp code, once you write it, will work forever (seriously, decades). The #lisp channel on Freenode is nice and helpful, and there are so many amazing people in the community. In Lisp, it's seamless to inspect assembly code and work with the world's most high-level, meta-object systems all at the same time. But the ecosystem mouthfeel is still off, and Common Lisp would greatly benefit from programmers obsessed with making the language more useful to themselves and others today.

    [1]: LispWorks: http://www.lispworks.com/

    [2]: Allegro CL: https://franz.com/products/allegro-common-lisp/

    [3]: Portacle: https://portacle.github.io/

    [4]: McCLIM: https://common-lisp.net/project/mcclim/

    [5]: There is a GIF parser though called SKIPPY! https://www.xach.com/lisp/skippy/

    [6]: MIDI: http://www.doc.gold.ac.uk/isms/lisp/midi/

    [7]: Zig: https://ziglang.org/

    [8]: MAGICL: https://github.com/rigetti/magicl

    [9]: Staple: https://shinmera.github.io/staple/

    [10]: Charles Zhang's SBCL commits https://github.com/sbcl/sbcl/commits?author=karlosz

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo
  • reagent

    A minimalistic ClojureScript interface to React.js

  • I'm going to say Clojure is probably what you want. Via clojurescript, it has very good interoperability with the tools you're already using and it's possible that you might even choose to use it on the front end in the browser. That, coupled with the size of the community suggest to me that it is the winner here.

    For the (browser) front-end, there are lots of neat idiomatic-clojure libraries like https://reagent-project.github.io/

    For command-line scripting, your program will ultimately mostly access the filesystem or other OS functionality via interop with the host platform (either Java or Node).

  • inlined-generic-function

    Discontinued Bringing the speed of Static Dispatch to CLOS. Succeeded by https://github.com/marcoheisig/fast-generic-functions

  • Do you mean something like this? https://github.com/guicho271828/inlined-generic-function

    Even if you know all the types, you still have polymorphism that can’t be efficiently monomorphized without an exponential space penalty.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts