ccl VS sketch

There's plenty of history of s-expression formats for documentation. One example is: https://github.com/Clozure/ccl/tree/master/doc/manual

But, also, there's plenty of uses of XML that are not "artcles and books". For example, Maven's pom.xml and log4j2.xml.

The descendant of CCL runs on modern Intel Macs. (It also runs on Linux and Windows but without the IDE.) The modern IDE is quite a bit different from the original. In particular, it no longer has the interface builder. But it's still pretty good. It is now called Clozure Common Lisp (so the acronym is still CCL) and you can find it here:

https://ccl.clozure.com/

If you want to run the original that is a bit of a challenge, but still possible. The original was never ported directly to OS X so you have to run it either on old hardware or an emulator running some version of the original MacOS, or on an older Mac running Rosetta 1. In the latter case you will want to look for something called RMCL. Also be aware that Coral Common Lisp was renamed Macintosh Common Lisp (i.e. MCL) before it became Clozure Common Lisp (CCL again).

This looks like it might be a promising place to start:

https://github.com/binghe/mcl

If you need more help try this mailing list:

https://lists.clozure.com/mailman/listinfo/openmcl-devel

Download: https://github.com/Clozure/ccl/releases/tag/v1.12.2

I was not aware that UIOP provided that function. plain-common-lisp used to be implemented with Clozure CL but eventually moved to SBCL due to the lack of maintenance of CCL. But now there is a hard dependency on SBCL.

I have decided it is time to have some fun and use Common Lisp to create algorithm representation that deals with parallel execution. For this I decided to use Clozure common lisp, put basic Qucklisp there and load some libraries to do this.

Lisp must be read outside->in to understand what it is saying. Given (foo (a) (b c)), if you don't know what foo is and just start reading (b c), which is inside, hoping that later you can work out what is foo, you could be going down a blind alley. foo could be a macro or special operator which entirely controls what (b c) means.

To understand what is calculated in Lisp, given that you understand what the syntax means, the evaluation is inside->out.

That's no different from math. In any languages that have math-like nested expressions with bracketing, you have inside-out evaluation.

The alternative are catenative languages and such, which have never been mainstream.

There are assembly languages which go line by line.

Imperative languages with statements and expressions tend to have small expressions where evaluation is followed inside-out; the rest of the control flow is just top down, with some forward and backward skips.

Lisp has all of the above in it. Lisp can be assembly language. For instance, in thsi source file from Clozure Common Lisp:

https://github.com/Clozure/ccl/blob/master/level-0/ARM/arm-h...

  (defarmlapfunction fast-mod-3 ((number arg_x) (divisor arg_y) (recip arg_z))

We've made a lot of data tooling things based on LLMs, and are in the process of rebranding and launching our main product.

1. sketch (in notebook, ai for pandas) https://github.com/approximatelabs/sketch

2. datadm (open source, "chat with data", with support for the open source LLMs (https://github.com/approximatelabs/datadm)

3. Our main product: julyp. https://julyp.com/ (currently under very active rebrand and cleanup) -- but a "chat with data" style app, with a lot of specialized features. I'm also streaming me using it (and sometimes building it) every weekday on twitch to solve misc data problems (https://www.twitch.tv/bluecoconut)

For your next question, about the stack and deploy:

This morning I added a "Related Projects" [3] Section to the Buckaroo docs. If Buckaroo doesn't solve your problem, look at one of the other linked projects (like Mito).

[1] https://github.com/approximatelabs/sketch

[2] https://github.com/paddymul/buckaroo

[3] https://buckaroo-data.readthedocs.io/en/latest/FAQ.html

For GPT/Copilot style help for pandas, in notebooks REPL flow (without needing to install plugins), I built sketch. I genuinely use it every-time I'm working on pandas dataframes for a quick one-off analysis. Just makes the iteration loop so much faster. (Specifically the `.sketch.howto`, anecdotally I actually don't use `.sketch.ask` anymore)

https://github.com/approximatelabs/sketch

https://github.com/approximatelabs/lambdaprompt It has served all of my personal use-cases since making it, including powering `sketch` (copilot for pandas) https://github.com/approximatelabs/sketch

Core things it does: Uses jinja templates, does sync and async, and most importantly treats LLM completion endpoints as "function calls", which you can compose and build structures around just with simple python. I also combined it with fastapi so you can just serve up any templates you want directly as rest endpoints. It also offers callback hooks so you can log & trace execution graphs.

All together its only ~600 lines of python.

I haven't had a chance to really push all the different examples out there, but most "complex behaviors", so there aren't many patterns to copy. But if you're comfortable in python, then I think it offers a pretty good interface.

I hope to get back to it sometime in the next week to introduce local-mode (eg. all the open source smaller models are now available, I want to make those first-class)

Please look at sketch and langchain pandas/SQL plugins. I have seen excellent results with both of these approaches. Both of these approaches will require you to send metadata to openAI.

👉 Understand your data through questions 👉 Create code from plain text Quick Read: https://www.marktechpost.com/2023/02/01/meet-sketch-an-ai-code-writing-assistant-for-pandas/ Github: https://github.com/approximatelabs/sketch

(3) Asking for re-writes of failed queries (happens occasionally) also helps

The main challenge I think with a lot of these "look it works" tools for data applications, is how do you get an interface that actually will be easy to adopt. The chat-bot style shown here (discord and slack integration) I can see being really valuable, as I believe there has been some traction with these style integrations with data catalog systems recently. People like to ask data questions to other people in slack, adding a bot that tries to answer might short-circuit a lot of this!

We built a prototype where we applied similar techniques to the pandas-code-writing part of the stack, trying to help keep data scientists / data analysts "in flow", integrating the code answers in notebooks (similar to how co-pilot puts suggestions in-line) -- and released https://github.com/approximatelabs/sketch a little while ago.

Python AI Helper https://github.com/approximatelabs/sketch