nand2tetris VS equinox

Yes, please do! Warning: F# will ruin other languages for you. I find it rather painful to work in basically anything else after using F#, with gradients of pain for different languages. Haha.

And that's a good question. I have basically every book written on F#, but I can't say I have ever used them for anything more than reference.

The official docs/guide/reference are actually really good, and I refer to them a lot when using some feature I'm not familiar with: https://docs.microsoft.com/en-us/dotnet/fsharp/what-is-fshar...

F# For Fun and Profit is well-known, but I can't say I use it a lot: https://fsharpforfunandprofit.com/

The same author's (Scott Wlaschin) book is very good: https://pragprog.com/titles/swdddf/domain-modeling-made-func...

As for books, I have always liked:

* Functional Programming Using F# by Hansen and Rischel (might be too simple if you are already comfortable with functional programming and is out of date every now and then with changes to F# that's happened)

* Expert F# 4.0 by Don Syme and others (contains a lot of nice things by the designer of F#

One of the latest books is Stylish F# 6: Crafting Elegant Functional Code for .NET 6 by Kit Eason. I have the first edition but haven't read it.

My personal recommendation is to take the approach of type/domain driven design. That is, I start off every F# module the same:

1. Define my types with discriminated unions, records, type aliases (such as for tuples) or single case discriminated unions. Use classes when necessary but try to prefer the more functional types.

2. Start writing functions against these.

And that's basically it. One thing to recognize with F# is that it mixes OOP rather nicely. Even discriminated unions and records, which are immutable, can have members defined on them, including operator overloading (something F# is pretty good about). They can even implement interfaces and be defined with generic types, which is also nice and powerful.

I have some projects that might of interest, since they're simple enough and illustrate the above process.

https://github.com/bmitc/the-ray-tracer-challenge-fsharp

https://github.com/bmitc/nand2tetris

Lastly, I'd suggest just starting up some projects. You could also take the Programming Languages course on Coursera by Dan Grossman. Part A uses SML, and you could port the examples and homework solutions to F# (I did so when I took the course). I also take books written for other languages and port the code to F#, usually taking a more idiomatic functional style. .NET Interactive notebooks (https://github.com/dotnet/interactive) are a great way to get started. You just need to install the .NET 6 SDK (which gets you F#) and then install the .NET Interactive Notebook extension in VS Code. That's it. There is also the book The Little MLer which gets people comfortable with discriminated unions (sum types), and I used the book and ported the examples to F#. I need to go back and finish that annotation project (https://github.com/bmitc/the-little-fsharper). I'll probably convert the script files to .NET Interactive notebooks if I do.

See references here: https://github.com/bmitc/nand2tetris

One project I have been doing is implementing the software stack from the Nand2Tetris course in F#. You can find my repository here: https://github.com/bmitc/nand2tetris

I love this book (the first edition), project, and course.

It is incomplete, but I thought I would share my implementation of the software stack in F#. Currently, only the assembler is implemented, but in my personal opinion, I think it showcases the beauty of F# for domain modeling. When I return to the project, I hope to restart the VM implementation and continue adding to the FPGA implementation as well. My eventual goal is to have the entire software stack built using F# that can than be run on an FPGA implementation of the CPU.

https://github.com/bmitc/nand2tetris

I even have a comment mentioning this: https://github.com/bmitc/nand2tetris/blob/3e145df44a97e2df094c585a1e97b7b1713dc582/dotnet/Nand2Tetris/Assembler/Types.fs#L90

If you scroll down to #2 in this repository, I show an example of active patterns in use. The active pattern is defined here. This particular active pattern is described in the docs even, but I find it extremely useful for parsing. It basically allows you to build a sane parser that takes in a string and maps it, via regular expressions, to a discriminated union. That means that your parsing step will generate known types that can be exhaustively pattern matched, making it basically impossible for your downstream functions to have an error due to an unhandled parsing step.

I wrote a JAX-based neural network library (Equinox [1]) and numerical differential equation solving library (Diffrax [2]).

At the time I was just exploring some new research ideas in numerics -- and frankly, procrastinating from writing up my PhD thesis!

But then one of the teams at Google starting using them, so they offered me a job to keep developing them for their needs. Plus I'd get to work in biotech, which was a big interest of mine. This was a clear dream job offer, so I accepted.

Since then both have grown steadily in popularity (~2.6k GitHub stars) and now see pretty widespread use! I've since started writing several other JAX libraries and we now have a bit of an ecosystem going.

[1] https://github.com/patrick-kidger/equinox

The elevator pitch is Optimistix is really fast, especially to compile. It plays nicely with Optax for first-order gradient-based methods, and takes a lot of design inspiration from Equinox, representing the state of all the solvers as standard JAX PyTrees.

If you like PyTorch then you might like Equinox, by the way. (https://github.com/patrick-kidger/equinox ; 1.4k GitHub stars now!)

You're thinking of `jax.closure_convert`. :)

(Although technically that works by tracing and extracting all constants from the jaxpr, rather than introspecting the function's closure cells -- it sounds like your trick is the latter.)

When you discuss dynamic allocation, I'm guessing you're mainly referring to not being able to backprop through `jax.lax.while_loop`. If so, you might find `equinox.internal.while_loop` interesting, which is an unbounded while loop that you can backprop through! The secret sauce is to use a treeverse-style checkpointing scheme.

https://github.com/patrick-kidger/equinox/blob/f95a8ba13fb35...

I'm a big fan of using ABCs to declare interfaces -- so much so that I have an improved abc.ABCMeta that also handles abstract instance variables and abstract class variables: https://github.com/patrick-kidger/equinox/blob/main/equinox/_better_abstract.py

For the daily research, I use Equinox (https://github.com/patrick-kidger/equinox) as a DL librarry in JAX.

It sounds like you're concerned about how downstream libraries tend to wrap JAX transformations to handle their own thing? (E.g. `haiku.grad`.)

If so, then allow me to make my usual advert here for Equinox:

https://github.com/patrick-kidger/equinox

This actually works with JAX's native transformations. (There's no `equinox.vmap` for example.)

On higher-order functions more generally, Equinox offers a way to control these quite carefully, by making ubiquitous use of callables that are also pytrees. E.g. a neural network is both a callable in that it has a forward pass, and a pytree in that it records its parameters in its tree structure.