Our great sponsors
-
amazon-sqs-java-extended-client-lib
An extension to the Amazon SQS client that enables sending and receiving messages up to 2GB via Amazon S3.
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
amazon-sqs-net-extended-client-lib
Extension to Amazon SQS that adds support for sending and receiving messages greater than 256K
-
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
- Small features being first party like https://github.com/awslabs/amazon-sqs-java-extended-client-l... vs https://github.com/raol/amazon-sqs-net-extended-client-lib
- Small features being first party like https://github.com/awslabs/amazon-sqs-java-extended-client-l... vs https://github.com/raol/amazon-sqs-net-extended-client-lib
- Small features being first party like https://github.com/awslabs/amazon-sqs-java-extended-client-l... vs https://github.com/raol/amazon-sqs-net-extended-client-lib
- Missing features or those that lag behind: https://github.com/confluentinc/confluent-kafka-dotnet/issue...
- Usually manually wired and configured vs the spring boot "starter" pattern of having libraries that automatically do some of the manual setup work for you: https://github.com/spring-projects/spring-boot/blob/main/spr...
I wish more client library sets had the feature-matrix that the pulsar one does, because in practice most end up being the same: Java supports everything because it's either built in the same codebase or is the most used client and gets the most support, while the dotnet client codebase has many feature-requests or performance improvement issues, often leading to a "third-party client" being created.
Performance.
Generally speaking, F# was actually very fast, and had nice concurrency support, but there were times that wasn't the case.
For example, in 2016 I was part of the initiative to rewrite the ad feed. We had to read in several Kafka topics, do some joining on our end, and emit to a separate Kafka topic. This isn't terribly hard to write, but we were dealing on the order of about ~100gb of data being pushed into memory. This is hardly "big data" stuff, but it's enough to highlight some issues.
Specifically, the built F# persistent map structure was simply too slow to get the performance we wanted. I really like that structure, it's really handy and nice, but I ended up having to make heavy use of the ConcurrentDictionary that was built into .NET. This wasn't that hard or anything, but it made me a little sad that I had to move to a mutable store to get the performance I needed.
There was also the fact that the `async` monad, while generally very good and useful, had bizarre bottlenecks that were hard to measure. It was difficult to know when the async task was actually started, and when you tried to measure performance bottlenecks you were really only measuring the scheduler, not the actual performance. This isn't really F#'s fault, this is an issue with any kind of cooperative scheduling system, but occasionally to get the performance we needed we'd have to move to lower level threads instead of the pretty monadic stuff. Microsoft eventually released the Task monad which generally performed a bit better.
There were other things here and there; the Kafka client libraries for .NET simply aren't as good as the Java ones. Jet actually open-sourced their own (https://github.com/jet/kafunk) which did make it a bit more functional and nice, but it had performance issues as well, so a lot of us ended up using Confluent.
There were little annoyances specific to F# as well; there's no real concept of a monad transformer, so if you wanted to do something like, for example, combine an Option and an Async into generalized syntax, you'd have to write your own wrapper monad thing, which wasn't that hard but was sort of ad hoc.
The general rule of thumb was that the first draft of software, we would try and keep as functional and pretty. If that was too slow, we allowed mutation but only within a function. If that was too slow, we'd allow global mutation but only with thread-safe stuff.
Using bindings to C++ libraries is likely to yield better experience. C# has really good interop API (P/Invoke).
I don't have experience with machine vision but here's an example of a library that integrates whisper.cpp in an idiomatic way: https://github.com/sandrohanea/whisper.net
There are many good community projects with code quality way higher than your average enterprise SDK with layers upon layers of abstractions and allocations. Finding them is the same as with most other languages like Rust or TS.
Isn't not having perfect support with Apache is, in a way, a criticism of any non-JVM language? Perhaps the problem is on the other end?
We need a couple more Apache Struts and Log4J incidents for the public to realize there may be better technologies and ecosystems. Spring is very slow anyways.
As for Pulsar, there is a nice and optimized client (written in F#!) https://github.com/fsprojects/pulsar-client-dotnet