playground VS simpleaichat

Not the parent, but NNs typically work better when you can't linearize your data. For classification, that means a space in which hyperplanes separate classes, and for regression a space in which a linear approximation is good.

For example, take the circle dataset here: https://playground.tensorflow.org

That doesn't look immediately linearly separable, but since it is 2D we have the insight that parameterizing by radius would do the trick. Now try doing that in 1000 dimensions. Sometimes you can, sometimes you can't or do want to bother.

For visualisation and some fun: http://playground.tensorflow.org/

https://seeing-theory.brown.edu/

https://www.3blue1brown.com/

https://playground.tensorflow.org/

There’s an interactive neural network you can train here, which can give some intuition on wider vs larger networks:

https://mlu-explain.github.io/neural-networks/

See also here:

http://playground.tensorflow.org/

This site is worth playing around with to get a feel for neural networks, and somewhat about ML in general. There are lots of strategies for statistical learning, and neural nets are only one of them, but they essentially always boil down into figuring out how to build a “classifier”, to try to classify data points into whatever category they best belong in.

I don’t know much experimenting you’ve done, but many repeated small scale experiments might give you a better intuition at least. I highly recommend this online tool for playing with different environmental variables, even if you’re comfortable coding up your own experiments: http://playground.tensorflow.org

Even if you can’t code, play around with this tool: https://playground.tensorflow.org — you can adjust the shape of the NN and watch how well it classifies the data. Model size obviously matters.

I don't think so. You can easily play around in the browser, using Javascript, or on https://processing.org/, https://playground.tensorflow.org/, https://scratch.mit.edu/, etc.

If anything the problem is that today's kids have too many options. And sure, some are commercial.

Well there is no point of using a multilayer linear neural network, because a cascade of linear transformations can be reduced to a single linear transformation. So you can only approximate linear functions. However if you have prior knowledge about the non linearity of your data lets say you know that it is a linear combination of polynomials up to certain degree, you can expand your input space by explicitly making non linear transformation. For instance a 1D linear regression can be modeled by 2 input neurons and 1 output neuron where the activation of the output is the identity. The input neuron x0 will take a constant input namely 1 and the second input neuron x1 will takes your data x. The output neuron will be y=w_0 * 1+w_1 *x which is equal to y=w_0 +w_1 * x. Let us say that your data follows a polynomial form, the idea is to add input neurons and expand your input to for instance X=[1 x x2] in this case you have 3 input neurons where the third is an explict non linear form of the input so y=w_0 + w_1 x +w_2 x2. The general idea is to find a space where the problem becomes linear. In real life example these spaces are non trivial the power of neural network is that they can find by optimization such space without explicitly encoding these non linearities. Try playing around with https://playground.tensorflow.org/ you can get an intuition about your question.

I checked both the issues mentioned, people have been respectful and showing empathy to author's situation

https://github.com/minimaxir/simpleaichat/issues/91

https://github.com/minimaxir/simpleaichat/issues/92

ChatGPT's June updated added support for "function calling", which in practice is structured data I/O marketed very poorly: https://openai.com/blog/function-calling-and-other-api-updat...

Here's an example of using structured data for better output control (lightly leveraging my Python package to reduce LoC: https://github.com/minimaxir/simpleaichat/blob/main/examples... )

So what are the alternatives to LangChain that the HN crowd uses?

I see two contenders:

https://github.com/minimaxir/simpleaichat/tree/main/simpleai...

https://github.com/griptape-ai/griptape

There is also the llm command line utility that has a very thin underlying library, but which might grow eventually:

A fun note is that even with system prompt engineering it may not give the most efficient solution: ChatGPT still outputs the avergage case.

I tested around it and doing two passes (generate code and "make it more efficient") works best, with system prompt engineering to result in less code output: https://github.com/minimaxir/simpleaichat/blob/main/examples...

I played around with simpleaichat for a few minutes just now, and I really like it. Unlike LangChain, I can understand what it does in minutes, and it looks like its primitives are fairly powerful. It looks like it's going to replace the `openai` library for me, it seems like a nice wrapper.

I'm especially looking forward to playing with the structured data models bit: https://github.com/minimaxir/simpleaichat/blob/main/examples...

Well done, Max!

https://github.com/minimaxir/simpleaichat bills itself as a simpler alternative to langchain. I have not tried it, but it looks interesting.

I think you are asking specifically about practical LLM engineering and not the underlying science.

Honestly this is all moving so fast you can do well by reading the news, following a few reddits/substacks, and skimming the prompt engineering papers as they come out every week (!).

https://www.latent.space/p/ai-engineer provides an early manifesto for this nascent layer of the stack.

Zvi writes a good roundup (though he is concerned mostly with alignment so skip if you don’t like that angle): https://thezvi.substack.com/p/ai-18-the-great-debate-debates

Simon W has some good writeups too: https://simonwillison.net/

I strongly recommend playing with the OpenAI APIs and working with langchain in a Colab notebook to get a feel for how these all fit together. Also, the tools here are incredibly simple and easy to understand (very new) so looking at, say, https://github.com/minimaxir/simpleaichat/tree/main/simpleai... or https://github.com/smol-ai/developer and digging in to the prompts, what goes in system vs assistant roles, how you gourde the LLM, etc.

This notebook from the repo I linked to is a concise example, and the reason you would want to optimize prompts.