empirical-lang VS april

Statically typed dataframes are exactly why I created the Empirical programming language:

https://www.empirical-soft.com

It can infer the column names and types from a CSV file at compile time.

Here's an example that misspells the "ask" column as if it were plural:

  let quotes = load("quotes.csv")

I'm interested in languages experimenting with integrating tables/dataframes and the relational model as native parts of the language

With how much of the code I've written that's been around representing entities and doing CRUD on collections of them it seems a bit wild not to have native "Table" types.

There's an HN user who has a neat language called "Empirical" based on this, and there's an active research language called "Flix" that has this as well. APL/K and other array langs feature a similar concept as I understand it.

https://www.empirical-soft.com/

https://flix.dev/

The most used in finance is a few functions in pandas, like merge_asof(). It's pretty industry standard now.

https://pandas.pydata.org/docs/reference/api/pandas.merge_as...

The biggest open-source thing I've ever written is Empirical, a language for time-series analysis.

https://www.empirical-soft.com

I'm also top 1% on Stack Overflow. A lot of companies have found me through there.

https://stackoverflow.com/users/478288/chrisaycock

I started Empirical with the goal of "q like Haskell". The end result went in a radically different direction, but the guiding light has always been to have a statically typed language where tables and queries are a first-class operation.

The source code is publicly available under AGPL with the Commons Clause:

https://github.com/empirical-soft/empirical-lang

Some of the OP's observations are why I wrote Empirical [1].

- I wanted static types that felt dynamic, which meant being able to infer types from a file as long as the file path can be determined at compile time.

- I wanted Dataframes to be a first-class container since the high-level of abstraction leads to great performance when doing analytics.

Therefore, I created a language with statically typed Dataframes. Empirical can infer a CSV's type at compile time, namely because performs compile-time function evaluation automatically. The result is a language that runs easily in a REPL but can be used for long-running scripts without worrying about common programming errors.

[1] https://www.empirical-soft.com

I wanted to hit the runtime repeatedly with as little overhead as possible. So VVM has no type look-up, multiple operands per instruction, and a cache-efficient IR. The sum() operand for the above example is invoked directly in a loop almost as fast as hand-written C++.

VVM has its own assembly language [2]. I have blog post that explains some of the design choices [3].

[1] https://github.com/empirical-soft/empirical-lang/tree/master...

[2] https://github.com/empirical-soft/empirical-lang/tree/master...

[3] https://www.empirical-soft.com/2020/09/03/a-tour-of-the-vect...

Other work on integrating APL and lisp is here: https://github.com/phantomics/april

Try J or APL, K, BQN, or April, and be prepared to rethink how you implement solutions to problems you've tackled in other PLs. I am an array language user and fan. I have been playing with April and I use J regularly at home and sometimes for work when I can.

From the April github site: "April compiles a subset of the APL programming language into Common Lisp. Leveraging Lisp's powerful macros and numeric processing faculties, it brings APL's expressive potential to bear for Lisp developers. Replace hundreds of lines of number-crunching code with a single line of APL."

  https://github.com/phantomics/april

There are attempts to combine those...

April (Array Programming Re-Imagined in Lisp)

https://github.com/phantomics/april

> operations that apply to the whole array

like MAP and REDUCE, higher order functions are not really new to Lisp. In Common Lisp they are extended to vectors.

> list languages and array languages are quite different.

There are some common things like interactive use, functional flavor, etc.

There's also April APL: https://github.com/phantomics/april

Also the array language family seems to be stronger than ever with foss: ngn/k, BQN, uiua, and of course J but as you mentioned they're all different languages.

I love J and APL, but April takes the cake for me[1]. APL in Lisp.

I also prefer SPARK2014 instead of Rust if I am not going to use C. I've started learning Rust a few times. SPARK2014 is easier to get going for me, and it has been used to produce high-integrity software and real-world applications for over a decade, and more if you include Ada from which it sprang[2].

[1] https://github.com/phantomics/april

[2] https://www.adacore.com/about-spark

The one big use case was RabbitMQ in a messaging app, not HFT. I doubt Elixir even with Nx can compete with low-level HFT code. Python DL/ML code libraries are just wrappers around C too. Maybe if BeamAsm and Nx are used Elixir could be used for more numerical or not just distributed applications.

I've programmed in Python and Julia, and when I worked at an engineering (mechanical, entertainment engineering) company, Julia was great for its similarity to Matlab. I am a self-taught engineer, so I did not get pulled into Matlab in college.

Personally, I took to Erlang, so I could write plugins for Wings3D back in the early 2000s, but I never stuck with Erlang, or Wings3D (Blender3D was my choice and I even contributed to have it go opensource way back when). I like Erlang's syntax better for some reason, although Elixir's is beautiful too. I was not a Ruby programmer, and I had delved into Haskell and Prolog, so I think Erlang made more sense to me. I think Elixir has a lot more momentum behind it than Erlang, but at the root it's Erlang, so I think I'll stick with Erlang for BEAM apps. My favorite language is April[1] (APL in Lisp), and given my love of J, would be a better fit for any finance apps I might write. I am trying to convert some of the Lisp code in this book, "Professional Automated Trading: Theory and Practice" to April.

Maybe I'll write some equivalent Elixir code to compare.

[1] https://github.com/phantomics/april

I'm also very curious for hear from expert lispers. I've tried to find the sweat spot where lisp would fit better than what I already know: shell for glue and file ops, R for data munging and vis, python to not reinvent things, perl/core-utils for one liners. But before I can find the niche, I get turned off by the amount of ceremony -- or maybe just how different the state and edit/evaluate loop is.

I'm holding onto some things that make common lisp look exciting and useful (static typing[0], APL DSL[1], speed [2,3,4]) and really want to get familiar with structural editing [5]

[0] https://github.com/phantomics/april - APL dsl

The 2 0 at the start of the APL line above controls the mirroring behavior. The second number can be set to 0 or 1 to choose which side of the image to mirror, while the 2 sets the axis along which to mirror. This will be 1 or 2 for a raster image but this function can mirror any rank of array on any axis.

April was used to teach image filtering in a programming class for middle-schoolers, you can see a summary in this video: https://vimeo.com/504928819

For more APL-driven graphics, April's repo includes an ncurses demo featuring a convolution kernel powered by ⌺, the stencil operator: https://github.com/phantomics/april/tree/master/demos/ncurse...