sdk VS .NET Runtime

I hate to defend telemetry of all things but in this particular case the criticism is unfounded and lacks context:

https://dotnet.microsoft.com/en-us/platform/telemetry

https://learn.microsoft.com/en-us/dotnet/core/tools/telemetr...

https://github.com/dotnet/sdk/tree/main/src/Cli/dotnet/Telem...

In any case, Debian would use https://github.com/dotnet/source-build and dotnet/dotnet, and could easily include the argument or a patch for this. It’s unlikely to be an issue. My bet it was not in Debian because there was no one to take initiative or there was but that person has faced a backlash by people in Debian who are similar to vocal minority here that posts FUD because of their little personal crusade.

FWIW telemetry can be easily opted-out, which the SDK explicitly tells you about, and there is a page that has full list of the kind of data that is collected (usage metrics and tooling crash stack traces). You can also review the metrics yourself.

About: https://learn.microsoft.com/en-us/dotnet/core/tools/telemetr...

Collected statistics: https://dotnet.microsoft.com/en-us/platform/telemetry

Source code: https://github.com/dotnet/sdk/tree/main/src/Cli/dotnet/Telem...

In either case, many CIs simply have 'DOTNET_CLI_TELEMETRY_OPTOUT=' and call it a day.

They used to gather all command line arguments until they later decided that (oops!) it's "not acceptable per our privacy policies"[0] and they really shouldn't have been doing that. They have also had issues with anonymization not being implemented properly, the opt-out mechanism not working in some edge cases, forgetting to even tell users about the need to opt out, and who knows what else.

Also, monetary value is not the only reason you might want to keep information private.

[0] https://github.com/dotnet/sdk/issues/6145#issuecomment-22010...

[DllImport("libc")] private static extern uint geteuid(); public bool IsCurrentProcessElevated() { if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows)) { // https://github.com/dotnet/sdk/blob/v6.0.100/src/Cli/dotnet/Installer/Windows/WindowsUtils.cs#L38 using var identity = WindowsIdentity.GetCurrent(); var principal = new WindowsPrincipal(identity); return principal.IsInRole(WindowsBuiltInRole.Administrator); } // https://github.com/dotnet/maintenance-packages/blob/62823150914410d43a3fd9de246d882f2a21d5ef/src/Common/tests/TestUtilities/System/PlatformDetection.Unix.cs#L58 // 0 is the ID of the root user return geteuid() == 0; }

> There's definitely things they tried to improve on that... weren't really improvements. The way "assemblies" are matched in .NET is much more sophisticated- the goal there was to try to kill DLL hell. It evolved into the Global Assembly Cache, which is sort of the Windows Registry of DLLs. Not a huge fan of those bits.

The Global Assembly Cache did not make the jump to the modern .NET (Core). There was the thing called `dotnet store`, but it’s broken since .NET 6: https://github.com/dotnet/sdk/issues/24752

The assembly redirection hell has also been greatly reduced there.

I do the same.

I have a small write-up here: https://chrlschn.dev/blog/2023/10/end-to-end-type-safety-wit...

You get end-to-end type safety (even better once you connect it to EF Core since you get it all ways to your DB).

With this setup with hot-reload (currently broken in .NET 8 [0]), productivity is really, really good. Like tRPC but with one of the most powerful ORMs out there right now.

[0] https://github.com/dotnet/sdk/issues/36918

Noticed an open issue just now.

You're thinking of https://github.com/dotnet/sdk/issues/22247

That's a twisted and wrong narrative

Unity like refers to a Editor driven approach

Unity became popular with its moonscript language (javascript like), they then ditched it to focus on C#, but what propelled unity to what it is today is the Editor driven approach, not c#, not DOTS

They are forced to transpile C# to C++ via IL2CPP as a result to target consoles/mobiles

C# is a disease when it comes to console/mobile support

It's a substantial dependency, quite heavy

And you are not free of unity like fuck ups, it's a microsoft language after all:

https://github.com/dotnet/sdk/issues/22247

And let's not forget when they changed the license of their debugger overnight to prevent people from using it in their products (jetbrains for example)

And them deprecating open source tooling to a proprietary/closed one for vscode (c# devkit)

Let's be careful when we recommend evil as an alternative to evil ;)

The script can be found at the end of the thread here https://github.com/dotnet/sdk/issues/8742

Personally, I'm disappointed that there is too much superstitions and assumptions about GC designs going around, which lead to the discussion like this one that treats specialized designs with explicit tradeoffs, which both Go and BEAM are, as universally superior options.

When it comes to Java - it has multiple GC implementations with different tradeoffs and degree of sophistication. I'm not very well versed in their details besides the fact that pretty much all of them are quite liberal with the use of host memory. So the way I approach it is by assuming that at least some of them resemble the GC implementation in .NET, given extensive evidence that under allocation-heavy scenarios they have similar (throughput) performance characteristics.

As for .NET itself, in server scenarios, it uses SRV GC which has per-core heaps (the count is sizing is now dynamically scalable per workload profile, leading to much smaller RAM footprint) and multi-threaded collection, which lends itself to very high throughput and linear scaling with cores even on very large hosts thanks to minimal contention (think 128C 1TiB RAM, stometimes you need to massage it with flags for this, but it's nowhere near the amount of ceremony required by Java).

Both SRV and WKS GC implementations use background collection for Gen2, large and pinned object heaps. Collection of Gen0 and Gen1 is pausing by design as it lends itself for much better throughput and pause times are short enough anyway.

They are short enough that modern .NET versions end up having better p99 latency than Go on multi-core throughput saturated nodes. Given decent enough codebase, you only ever need to worry about GC pause impact once you go into the territory of systems with hard realtime requirements. One of the better practical examples of this that exists in open source is Osu! which must run its game loop 1000hz - only 1ms of budget! This does pose challenges and requires much more hands-on interaction with GC like dynamically switching GC behaviopr depending on scenario: https://github.com/dotnet/runtime/issues/96213#issuecomment-... This, however, would be true with any language with automatic memory management, if it's possible to implement such a system in it in the first place.

It appears to use Socket and SslStream. Can't say much about HttpListener itself, but the first two are used by Kestrel (and everything else). There is Http.sys integration for Windows though.

https://github.com/dotnet/runtime/blob/ac663e44b907618c631ed...

> Does that mean that this mono project and its associated repo and what is within the dotnet repo are not the same and could (if they have not already) diverge?

Yes, they have diverged. Just as Microsoft forked the CLR to create CoreCLR, so too has mono been forked. Features like multiple AppDomains have been removed from this fork. Here is an example pull request:

https://github.com/dotnet/runtime/pull/47955

> And for the transition phase, there has to be interop for async ↔ async2

For those like me who weren't clear whether `async2` was expected to be a real keyword in the final language, it's not[0].

0 - https://github.com/dotnet/runtime/issues/94620#issuecomment-...

Embedding Mono today might be a mistake. You really do want to embed CoreCLR instead if you can, even though it's a bit more complex.

The reason for this is the up-to-date Mono (that keeps up on runtime features and library support) lives here: https://github.com/dotnet/runtime/tree/main/src/mono After .NET became what it is today, many Mono components simply became part of the codebase there. Most notably Mono linker which became ILLink, a critical component of assembly trimming and AOT compilation to native binaries.

However, Mono is significantly slower than CoreCLR, frequently does not have the optimizations that performance-oriented code paths expect, only supports 128b SIMD and now serves the purpose of supporting exotic targets like WASM, ARMv6 or just new platforms in the process of bring-up.

In any case, if you still plan to use Mono, it is best to use the one from dotnet/runtime.

> Not sure what you mean by "peephole heuristic optimizations"

Post-emit or within-emit stage optimization where a sequence of instruction is replaced with a more efficient shorter variant.

Think replacing pairs of ldr and str with ldp and stp, changing ldr and increment with ldr with post-index addressing mode, replacing address calculation before atomic load with atomic load with addressing mode (I think it was in ARMv8.3-a?).

The "heuristic" here might be possibly related to additional analysis when doing such optimizations.

For example, previously mentioned ldr, ldr -> ldp (or stp) optimization is not always a win. During work on .NET 9, there was a change[0] that improved load and store reordering to make it more likely that load simple consecutive loads and stores are merged on ARM64. However, this change caused regressions in various hot paths because, for example, previously matched ldr w0, [addr], ldr w1, [addr+4] -> modify -> str w0, [addr] pair got replaced with ldp w0, w1, [add] -> modify w0, str w0 [addr].

Turns out this kind of merging defeated store forwarding on Firestorm (and newer) as well as other ARM cores. The regression was subsequently fixed[1], but I think the parent comment author may have meant scenarios like these in mind.

[0]: https://github.com/dotnet/runtime/pull/92768

[1]: https://github.com/dotnet/runtime/pull/105695

I don't think there was any work on escape analysis in current .NET related to Midori, at the very least not in public.

However, there has been separate research and support for object escape analysis unrelated to this. Here's the rough timeline, given your interest:

Initial issue https://github.com/dotnet/runtime/issues/11192 was submitted in 2018 and concerns general research direction on feasibility and profitability of EA.

At the time, the results were very unpromising given compiler throughput impact: there were very few objects that were not escaping due to inlining limitations and (relatively) rudimentary approach to EA. In addition, there obviously was no impact on performance sensitive code that just used structs instead that are not reliant on fragile optimizations like this one.

Here, I would like to add a personal note that .NET teams are exposed to a much more well-behaved code and there existed (and still exists to an extent) bias that overlooks the nastiest and worst codebases that, no matter their issues and unnecessary allocation on every step, are an important optimization target. I think today there is a good degree of awareness and understanding of this bias, which drives further compiler improvements.

Back to EA. After initial research was done, the relevant code was merged into JIT - the feature was added around .NET 5 but remained disabled by default. Later on, it was occasionally in a broken state and generally not well validated against, given nothing used it besides compiler tests.

This, however, has changed in .NET 9. In order to understand why we first have to consider what were the changes in .NET's compiler in the previous versions.

Before .NET 7 and 8, JIT had nice but limited capability to perform something Java calls "scalar replacement" except for structs, the .NET name for this is "struct promotion". This feature "promotes" constituent struct fields to individual local values which are then stored in CPU registers or individual stack locations instead of "together" and being copied every time. This also included other common cases like treating single-field structs as an underlying field (making such wrappers effectively free). At the time, this optimization was great but nevertheless limited - there were restrictions on the amount of fields which could have been "promoted" and "enregistered" (up to 4 I believe?) as well as the depth - the quality of compiled code could easily regress if the target of promotion was nested within another struct, etc etc.

In order to address this, a new handling of struct promotion was introduced - "physical promotion": https://github.com/dotnet/runtime/issues/76928. This did away with the limitations of the past and enabled significantly better handling of structs, allowing them to be optimized away, promoted without depth and count restrictions, propagate constants and assertions through struct fields, CSE more expressions on struct and more. This was a significant overhaul which pushed .NET's compiler output quality concerning structs way closer to the kind of behavior you would usually expect from GCC and LLVM.

At this point, I think you have an idea where this is leading to. Come in https://github.com/dotnet/runtime/pull/102808, an unrelated change which enabled the compiler to reason about propagation of addresses (object references, byref and unmanaged pointers) through struct fields. However, the importance of this change is that Steve (hez2010) noticed[0] that it may have closed the critical gap that previously prevented generalized optimizations enabling optimal struct handling from being applied to objects allocated on the stack. Once it was merged, it made the simpler scenarios EA was working for to produce the same optimal codegen you would see from structs as of .NET 8.

This and related discussions prompted further work on analyzing the most common unescaped allocation patterns and resulted in https://github.com/dotnet/runtime/pull/103361 by Andy Ayers. Once it was merged, it extended supported cases by previously limited EA capability, tuned inlining heuristics to make it light up more often and proved that the way it was handled as of the change was noticeably more profitable. It was subsequently for NativeAOT and R2R as well: https://github.com/dotnet/runtime/pull/104411

As a result, .NET 9 will come with improved escape analysis capability, now enabled by default. It is still fairly limited but nonetheless an another change that incrementally adds to the nice "free" performance improvements everyone has grown to expect from bumping up a TFM version with each release for the last 7 years or so.

Further EA work is planned https://github.com/dotnet/runtime/issues/104936 and there are already initial experiments for .NET 10 like this one https://github.com/dotnet/runtime/pull/104906

I don't claim this timeline/sequence of events is perfectly accurate but I hope it sheds light on how and why this optimization has been evolving in .NET so far.

[0]: https://github.com/dotnet/runtime/pull/102808#issuecomment-2...