playground VS wordsandbuttons

Not the parent, but NNs typically work better when you can't linearize your data. For classification, that means a space in which hyperplanes separate classes, and for regression a space in which a linear approximation is good.

For example, take the circle dataset here: https://playground.tensorflow.org

That doesn't look immediately linearly separable, but since it is 2D we have the insight that parameterizing by radius would do the trick. Now try doing that in 1000 dimensions. Sometimes you can, sometimes you can't or do want to bother.

For visualisation and some fun: http://playground.tensorflow.org/

https://seeing-theory.brown.edu/

https://www.3blue1brown.com/

https://playground.tensorflow.org/

There’s an interactive neural network you can train here, which can give some intuition on wider vs larger networks:

https://mlu-explain.github.io/neural-networks/

See also here:

http://playground.tensorflow.org/

This site is worth playing around with to get a feel for neural networks, and somewhat about ML in general. There are lots of strategies for statistical learning, and neural nets are only one of them, but they essentially always boil down into figuring out how to build a “classifier”, to try to classify data points into whatever category they best belong in.

I don’t know much experimenting you’ve done, but many repeated small scale experiments might give you a better intuition at least. I highly recommend this online tool for playing with different environmental variables, even if you’re comfortable coding up your own experiments: http://playground.tensorflow.org

Even if you can’t code, play around with this tool: https://playground.tensorflow.org — you can adjust the shape of the NN and watch how well it classifies the data. Model size obviously matters.

I don't think so. You can easily play around in the browser, using Javascript, or on https://processing.org/, https://playground.tensorflow.org/, https://scratch.mit.edu/, etc.

If anything the problem is that today's kids have too many options. And sure, some are commercial.

Well there is no point of using a multilayer linear neural network, because a cascade of linear transformations can be reduced to a single linear transformation. So you can only approximate linear functions. However if you have prior knowledge about the non linearity of your data lets say you know that it is a linear combination of polynomials up to certain degree, you can expand your input space by explicitly making non linear transformation. For instance a 1D linear regression can be modeled by 2 input neurons and 1 output neuron where the activation of the output is the identity. The input neuron x0 will take a constant input namely 1 and the second input neuron x1 will takes your data x. The output neuron will be y=w_0 * 1+w_1 *x which is equal to y=w_0 +w_1 * x. Let us say that your data follows a polynomial form, the idea is to add input neurons and expand your input to for instance X=[1 x x2] in this case you have 3 input neurons where the third is an explict non linear form of the input so y=w_0 + w_1 x +w_2 x2. The general idea is to find a space where the problem becomes linear. In real life example these spaces are non trivial the power of neural network is that they can find by optimization such space without explicitly encoding these non linearities. Try playing around with https://playground.tensorflow.org/ you can get an intuition about your question.

Meanwhile, all the pages on https://wordsandbuttons.online/ with all the animation and interactivity are still below 64 KB.

This one, for example, https://wordsandbuttons.online/trippy_polynomials_in_arctang... is 51 KB.

And the code is not at all economical. It's 80% copy-paste with little deviations. There is no attempt to save by being clever either, it's all just good old vanilla JS. And no zipping, no space reduction. The code is perfectly readable when opened with the "View page source" button.

The trick is - zero dependency policy. No third party, no internal. All the code you need, you get along with the HTML file. Paradoxically, in the long run, copy-paste is a bloat preventor, not a bloat cause.

I do all the https://wordsandbuttons.online/ by hand, and this is my top 10:

    2527 p

Cloning things makes maintenance harder linearly. Generalizing things makes maintenance harder exponentially. Counterintuitively, generalization only makes sense on small numbers or reiterations, not the vice-versa.

I started https://wordsandbuttons.online as an experiment in zero-dependencies architecture. No third-party, no self-reference. Every page is completely independent. I was told, that as it grew, it would inevitably become impossible to maintain.

Five years passed, including a two-year pause for writing a book. I did more than half a hundred interactive tutorials and quizzes, and continue to add them when I have time. The thing simply refuses to go "too complex to maintain". All the maintenance problems I ever faced with this design were handled within minutes. If there is too much typing, I write a Python script. If not, I do the change manually, replicate it in a few pages, and go on.

As a free bonus, since all my pages are essentially hand-written, no dependencies = no uncontrolled growth, they are all fewer than 64 KB each. Ultra-fast to load, and I never get a "Reddit effect" since even 1000 requests a second is only 64 MB of data. It's green then tea too. I usually have a few hundred thousand visitors a year, and they barely consume enough electricity to boil a kettle of water.

I think this rule of three, like many others is just an attempt to escape well-known complexity issues by putting them under a carpet of less known complexity issues. I'm very happy to have ignored it when considering design for my site.

Done! All the expandables have background colors now. https://github.com/akalenuk/wordsandbuttons/commit/22ef6295c... Thanks for the idea!

Very indirectly. My https://wordsandbuttons.online/ helped me cement a publishing deal with Manning, and the book I wrote for them (https://www.manning.com/books/geometry-for-programmers) works as a hourly rate magnifier in contract work negotiations.

I do https://wordsandbuttons.online/ as a personal-ish website. I don't append my face to every page but a visitor is usually a few clicks away from my other works so the site is de-facto more or less personal.

First of all, it's a nice hobby. No bullshit programming, no frameworks, no dependencies, no annoying editors. I just write my code and text and enjoy doing so.

Second, it gives powerful motivation to study. I'm now writing a new page on rational interpolation and just yesterday I accidentally found a very simple way to avoid the Runge effect. I was just playing with interactives and it came out of the blue. There is no way I would have learned it otherwise.

Third, it helped me cement a publishing deal with Manning. They came to me and proposed to propose them a book on geometry. And so I did. The book is called Geometry for Programmers and it's coming this summer.

Fourth, I do public lectures (or at least I used to before the war), and the audience loves interactive illustrations. So I usually turn my site pages into presentation-like pages and do lectures with them.

So for me, having a website pays off in multiple ways.