are-we-fast-yet VS Smalltalk

> Sure there's a small overhead to smart pointers

Not so small, and it has the potential to significantly speed down an application when not used wisely. Here are e.g. some measurements where the programmer used C++11 and did everything with smart pointers: https://github.com/smarr/are-we-fast-yet/issues/80#issuecomm.... There was a speed down between factor 2 and 10 compared with the C++98 implementation. Also remember that smart pointers create memory leaks when used with circular references, and there is an additional memory allocation involved with each smart pointer.

> Garbage collection has an overhead too of course

The Boehm GC is surprisingly efficient. See e.g. these measurements: https://github.com/rochus-keller/Oberon/blob/master/testcase.... The same benchmark suite as above is compared with different versions of Mono (using the generational GC) and the C code (using Boehm GC) generated with my Oberon compiler. The latter only is 20% slower than the native C++98 version, and still twice as fast as Mono 5.

See https://github.com/smarr/are-we-fast-yet/blob/master/docs/guidelines.md.

That's a very interesting article, thanks. Interesting to note that Cython is only about twice as fast as Python 3.10 and only about 40% faster than Python 3.11.

The official Python site advertises a speedup of 25% from 3.10 to 3.11; in the article a speedup of 60% was measured. It therefore usually makes sense to measure different algorithms. Unfortunately there is no Python or C++ implementation yet for https://github.com/smarr/are-we-fast-yet.

> Software benchmarks are super subjective.

No, they are not, but they are just a measurement tool, not a source of absolute thruth. When I studied engineering at ETH we learned "Who measures measures rubbish!" ("Wer misst misst Mist!" in German). Every measurement has errors and being aware of these errors and coping with it is part of the engineering profession. The problem with programming language benchmarks is often that the goal is to win by all means; to compare as fairly and objectively as possible instead, there must be a set of suitable rules adhered to by all benchmark implementations. Such a set of rules is e.g. given for the Are-we-fast-yet suite (https://github.com/smarr/are-we-fast-yet).

I am aware of the various published test results where CoreCLR shows fantastic speed-ups compared to Mono, e.g. when calculating MD5 or SHA hash sums.

But my measurements based on the Are-we-fast-yet benchmark suite (see https://github.com/smarr/are-we-fast-yet and https://github.com/rochus-keller/Oberon/tree/master/testcases/Are-we-fast-yet) show a completely different picture. Here the difference between Mono and CoreCLR (both versions 3 and 5) is within +/- 10%, so nothing earth shattering.

Here are my measurement results:

https://github.com/rochus-keller/Oberon/blob/master/testcases/Are-we-fast-yet/Are-we-fast-yet_results_linux.pdf comparing the same benchmark on the same machine run under LuaJIT, Mono, Node.js and Crystal.

https://github.com/rochus-keller/Oberon/blob/master/testcases/Are-we-fast-yet/Are-we-fast-yet_results_windows.pdf comparing Mono, .Net 4 and CoreCLR 3 and 5 on the same machine.

Here are the assemblies of the Are-we-fast-yet benchmark suite used for the measurements, in case you want to reproduce my results: http://software.rochus-keller.ch/Are-we-fast-yet_CLI_2021-08-28.zip.

I was very surprised by the results. Perhaps it has to do with the fact that I measured on x86, or that the benchmark suite used includes somewhat larger (i.e. more representative) applications than just micro benchmarks.

What are your opinions? Do others have similar results?

There is a good reason for this; have a look at e.g. https://github.com/smarr/are-we-fast-yet/blob/master/docs/guidelines.md.

Then apparently the SOM nbody program is taken as the basis of a new Java nbody program.

> * I'm assuming the "by the Bluebook" implementation they're referring to is this: *

Or this: https://github.com/rochus-keller/Smalltalk/

> message passing and late binding combined. "Duck typing" is seriously diminishing it

Actually even ST-72 made synchronous calls, but at least with a token stream interpreted by the receiving object (thus at least a bit of "message passing"). In ST-76 and later versions "message passing" is just nomenclature used by the ST folks for something that is just ordinary method dispatch and call (if you have doubts, you can analyze the innards of the ST-80 VM yourself e.g. with these tools: https://github.com/rochus-keller/Smalltalk ). The major difference is the dispatch based on signature hash (similar to e.g. Java interface method calls) instead of static positions, which enables late binding (at the expense of performance); and since everything including ordinary integers derive from Object, all values and objects are subject to dynamic method dispatch; it's no coincidence that Smalltalk was the first language to allow real duck typing. The unification of scalar values and references, dynamic typing, and likewise the minimal syntax where control structures are implemented by means of runtime constructs were already known from Lisp; also closures (i.e. ST blocks) were already known before they were added to ST.

Here is one even in Lua: https://github.com/rochus-keller/Smalltalk/

> 10kSLOC for the entire universe

It is the nature of idealists that they see the world idealized. Smalltalk-80 itself has nearly 30 kSLOC; it's just more difficult to count, but I wrote tools which can do it (https://github.com/rochus-keller/Smalltalk/).

Why should "perform" be a message? It's just a method of the Object class, which is the superclass of Integer. You can use my St80ClassBrowser and St80ImageViewer (see https://github.com/rochus-keller/Smalltalk/) to check the ST-80 source code and image if you want; there is a list of all selectors and the classes which implement them. Going up the class hierarchy when doing virtual method dispatch is a fundamental concept of all object-oriented implementations; in contrast to e.g. C++ this can be done dynamically at runtime in Smalltalk or Java (which is also called late binding). In contrast to Smalltalk in Java the class loader verifies that a method for the referenced signature actually exists; in Smalltalk you can try to dispatch any signature which can result in a call to the doesNotUnderstand method of the Object class.

> a large lua game code base, over 4000 files, 1.5 million lines of code

Interesting; how do you manage to keep consistency? Do you have special tools to e.g. detect inadvertent global variables? I once wrote a Smalltalk VM in Lua (https://github.com/rochus-keller/Smalltalk/blob/master/Inter...) which is a much smaller code base but even with this size I quickly would have lost track of e.g. scopes and names without tools I had to write myself (https://github.com/rochus-keller/LJTools).

It is true, that there is uncollected garbage in the original Xerox ST80 image. I've built some tools to analyze the image and also a VM which can be interrupted at any time to analyze the current state of the image (see https://github.com/rochus-keller/Smalltalk).

There are two zombie processes (OID 6662 and 19ba). There are also a couple of BlockContext and MethodContext which have a nil sender and a reference to an unknown method, but which are still referenced from somewhere (i.e. the collection is prevented even if it is not implemented by reference counting. E.g. OID 79a2 of class BinaryChoice. I have a full list if anybody is interested.

Integers are actually directly stored, i.e. without boxing/indirection by a pointer. The Smalltalk object memory doesn't have pointers in the C sense, but rather indices into the object table. If you're interested I've implemented a couple of tools to study the original Smalltalk-80 VM, see https://github.com/rochus-keller/Smalltalk/.

- https://github.com/rochus-keller/Smalltalk/ Parser, code model, interpreter and navigable browser for the original Xerox Smalltalk-80 v2 sources and virtual image file

- https://github.com/rochus-keller/Som/ Parser, code model, navigable browser and VM for the SOM Smalltalk dialect

- https://github.com/rochus-keller/Simula A Simula 67 parser written in C++ and Qt

> do you regret those endeavours?

No, not in any way; the projects were very entertaining and gave me interesting insights.