runtimelab VS .NET Runtime

For everyone reading this blog post I caution that the conclusions there are at best creative interpretations of the notes written down here: https://github.com/dotnet/runtimelab/blob/feature/async2-exp...

It is quite literally impossible to draw conclusions on e.g. memory consumption until the work on this, which is underway, makes it into mainline runtime. It's important to understand that the experiment was first and foremost a research to look into modernizing async implementation, and was a massive success. Now once that is proven, the tuned and polished implementation will be made.

Once it is done and makes into a release (it could even be as early as .NET 10), then further review will be possible.

This FAQ is a bit outdated in places, and is not something most users should worry about in practice.

JVM Green Threads here serve predominantly back-end scenarios, where most of the items on the list are not of concern. This list also exists to address bad habits that carried over from before the tasks were introduced, many years ago.

In general, the perceived want of green threads is in part caused by misunderstanding of that one bad article about function coloring. And that one bad article about function coloring also does not talk about the way you do async in C#.

Async/await in C# is just a better model with explicit understanding where a method returns an operation that promises to complete in the future or not, and composting tasks for easy (massive) concurrency is significantly more idiomatic than doing so with green threads or completable futures that existed in Java before these.

Also one change to look for is "Runtime Handled Tasks" project in .NET that will replace Roslyn-generated state machine code with runtime-provided suspension mechanism which will only ever suspend at true suspension points where task's execution actually yields asynchronously. So far numbers show at least 5x decrease in overhead, which is massive and will bring performance of computation heavy async paths in line with sync ones: https://github.com/dotnet/runtimelab/blob/feature/async2-exp...

Async/await is not a tight corner as showcased by a multitude of languages adopting the pattern: Rust, Python, JavaScript and Swift.

In fact, it is a clean abstraction where future progress is possible while retaining the convenience of its concurrency syntax and task composition.

Green threads experiment proved net negative in terms of benefit but its the follow-up work on modernizing the implementation detail was very successful: https://github.com/dotnet/runtime/issues/94620 / https://github.com/dotnet/runtimelab/blob/feature/async2-exp...

It also seems that common practices in Java indicate that properties are not a mistake as showcased by popularity of Lombok and dozens of other libraries to generate builders and property-like methods (or, worse, Java developers having to write them by hand).

There was a "green thread" experiment for dotnet a while ago, here is the conclusion: https://github.com/dotnet/runtimelab/issues/2398

Experiment result write-up: https://github.com/dotnet/runtimelab/blob/e69dda51c7d796b812...

TLDR: The green threads experiment was a failure as it found (expected and obvious) issues that the Java applications are now getting to enjoy, joining their Go colleagues, while also requiring breaking changes. It, however, gave inspiration to subsequent re-examination of current async/await implementation and whether it can be improved by moving state machine generation and execution away from IL completely to runtime. It was a massive success as evidenced by preliminary overhead estimations in the results.

Yeah, it kind of is. There are quite a few of experiments that are conducted to see if they show promise in the prototype form and then are taken further for proper integration if they do.

Unfortunately, object stack allocation was not one of them even though DOTNET_JitObjectStackAllocation configuration knob exists today, enabling it makes zero impact as it almost never kicks in. By the end of the experiment[0], it was concluded that before investing effort in this kind of feature becomes profitable given how a lot of C# code is written, there are many other lower hanging fruits.

To contrast this, in continuation to green threads experiment, a runtime handled tasks experiment[1] which moves async state machine handling from IL emitted by Roslyn to special-cased methods and then handling purely in runtime code has been a massive success and is now being worked on to be integrated in one of the future version of .NET (hopefully 10?)

[0] https://github.com/dotnet/runtime/issues/11192

[1] https://github.com/dotnet/runtimelab/blob/feature/async2-exp...

Unlike these folks from dotnet, which tested directly on ASP for real workload

  https://github.com/dotnet/runtimelab/issues/2398?darkschemeovr=1

[1] https://github.com/dotnet/runtimelab/issues/2398

The only way to avoid it is to not build on top of Java or not adding any features on top of Java.

> To give another example with C#, there has been a lot of recent discussion about finding potential alternatives to their async-await concurrency model. They cite the level of effort it takes to maintain the async await style code and the costs that come from this.

I had a very different take-away. They did PoC with virtual threads and decided it's not worth the switch now and async-await that they have is good enough.

https://github.com/dotnet/runtimelab/issues/2398

> Some of the languages it gets compared too aren't even that old yet.

C# is old enough to drink and Scala just had its 20th birthday this week :)

It appears to use Socket and SslStream. Can't say much about HttpListener itself, but the first two are used by Kestrel (and everything else). There is Http.sys integration for Windows though.

https://github.com/dotnet/runtime/blob/ac663e44b907618c631ed...

> Does that mean that this mono project and its associated repo and what is within the dotnet repo are not the same and could (if they have not already) diverge?

Yes, they have diverged. Just as Microsoft forked the CLR to create CoreCLR, so too has mono been forked. Features like multiple AppDomains have been removed from this fork. Here is an example pull request:

https://github.com/dotnet/runtime/pull/47955

> And for the transition phase, there has to be interop for async ↔ async2

For those like me who weren't clear whether `async2` was expected to be a real keyword in the final language, it's not[0].

0 - https://github.com/dotnet/runtime/issues/94620#issuecomment-...

Embedding Mono today might be a mistake. You really do want to embed CoreCLR instead if you can, even though it's a bit more complex.

The reason for this is the up-to-date Mono (that keeps up on runtime features and library support) lives here: https://github.com/dotnet/runtime/tree/main/src/mono After .NET became what it is today, many Mono components simply became part of the codebase there. Most notably Mono linker which became ILLink, a critical component of assembly trimming and AOT compilation to native binaries.

However, Mono is significantly slower than CoreCLR, frequently does not have the optimizations that performance-oriented code paths expect, only supports 128b SIMD and now serves the purpose of supporting exotic targets like WASM, ARMv6 or just new platforms in the process of bring-up.

In any case, if you still plan to use Mono, it is best to use the one from dotnet/runtime.

> Not sure what you mean by "peephole heuristic optimizations"

Post-emit or within-emit stage optimization where a sequence of instruction is replaced with a more efficient shorter variant.

Think replacing pairs of ldr and str with ldp and stp, changing ldr and increment with ldr with post-index addressing mode, replacing address calculation before atomic load with atomic load with addressing mode (I think it was in ARMv8.3-a?).

The "heuristic" here might be possibly related to additional analysis when doing such optimizations.

For example, previously mentioned ldr, ldr -> ldp (or stp) optimization is not always a win. During work on .NET 9, there was a change[0] that improved load and store reordering to make it more likely that load simple consecutive loads and stores are merged on ARM64. However, this change caused regressions in various hot paths because, for example, previously matched ldr w0, [addr], ldr w1, [addr+4] -> modify -> str w0, [addr] pair got replaced with ldp w0, w1, [add] -> modify w0, str w0 [addr].

Turns out this kind of merging defeated store forwarding on Firestorm (and newer) as well as other ARM cores. The regression was subsequently fixed[1], but I think the parent comment author may have meant scenarios like these in mind.

[0]: https://github.com/dotnet/runtime/pull/92768

[1]: https://github.com/dotnet/runtime/pull/105695

I don't think there was any work on escape analysis in current .NET related to Midori, at the very least not in public.

However, there has been separate research and support for object escape analysis unrelated to this. Here's the rough timeline, given your interest:

Initial issue https://github.com/dotnet/runtime/issues/11192 was submitted in 2018 and concerns general research direction on feasibility and profitability of EA.

At the time, the results were very unpromising given compiler throughput impact: there were very few objects that were not escaping due to inlining limitations and (relatively) rudimentary approach to EA. In addition, there obviously was no impact on performance sensitive code that just used structs instead that are not reliant on fragile optimizations like this one.

Here, I would like to add a personal note that .NET teams are exposed to a much more well-behaved code and there existed (and still exists to an extent) bias that overlooks the nastiest and worst codebases that, no matter their issues and unnecessary allocation on every step, are an important optimization target. I think today there is a good degree of awareness and understanding of this bias, which drives further compiler improvements.

Back to EA. After initial research was done, the relevant code was merged into JIT - the feature was added around .NET 5 but remained disabled by default. Later on, it was occasionally in a broken state and generally not well validated against, given nothing used it besides compiler tests.

This, however, has changed in .NET 9. In order to understand why we first have to consider what were the changes in .NET's compiler in the previous versions.

Before .NET 7 and 8, JIT had nice but limited capability to perform something Java calls "scalar replacement" except for structs, the .NET name for this is "struct promotion". This feature "promotes" constituent struct fields to individual local values which are then stored in CPU registers or individual stack locations instead of "together" and being copied every time. This also included other common cases like treating single-field structs as an underlying field (making such wrappers effectively free). At the time, this optimization was great but nevertheless limited - there were restrictions on the amount of fields which could have been "promoted" and "enregistered" (up to 4 I believe?) as well as the depth - the quality of compiled code could easily regress if the target of promotion was nested within another struct, etc etc.

In order to address this, a new handling of struct promotion was introduced - "physical promotion": https://github.com/dotnet/runtime/issues/76928. This did away with the limitations of the past and enabled significantly better handling of structs, allowing them to be optimized away, promoted without depth and count restrictions, propagate constants and assertions through struct fields, CSE more expressions on struct and more. This was a significant overhaul which pushed .NET's compiler output quality concerning structs way closer to the kind of behavior you would usually expect from GCC and LLVM.

At this point, I think you have an idea where this is leading to. Come in https://github.com/dotnet/runtime/pull/102808, an unrelated change which enabled the compiler to reason about propagation of addresses (object references, byref and unmanaged pointers) through struct fields. However, the importance of this change is that Steve (hez2010) noticed[0] that it may have closed the critical gap that previously prevented generalized optimizations enabling optimal struct handling from being applied to objects allocated on the stack. Once it was merged, it made the simpler scenarios EA was working for to produce the same optimal codegen you would see from structs as of .NET 8.

This and related discussions prompted further work on analyzing the most common unescaped allocation patterns and resulted in https://github.com/dotnet/runtime/pull/103361 by Andy Ayers. Once it was merged, it extended supported cases by previously limited EA capability, tuned inlining heuristics to make it light up more often and proved that the way it was handled as of the change was noticeably more profitable. It was subsequently for NativeAOT and R2R as well: https://github.com/dotnet/runtime/pull/104411

As a result, .NET 9 will come with improved escape analysis capability, now enabled by default. It is still fairly limited but nonetheless an another change that incrementally adds to the nice "free" performance improvements everyone has grown to expect from bumping up a TFM version with each release for the last 7 years or so.

Further EA work is planned https://github.com/dotnet/runtime/issues/104936 and there are already initial experiments for .NET 10 like this one https://github.com/dotnet/runtime/pull/104906

I don't claim this timeline/sequence of events is perfectly accurate but I hope it sheds light on how and why this optimization has been evolving in .NET so far.

[0]: https://github.com/dotnet/runtime/pull/102808#issuecomment-2...

Extend System.Guid with a new creation API for v7. Here you can read the discussion about the UUIDv7 support in .NET 9. This will bring several enhancements: improved time sorting, compatibility with other systems and standards, high uniqueness due to random and incremental data, and enhanced performance when generating and using UUIDs.