Our great sponsors
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
This has been posted and reposted continuously for a year and I still don’t understand the comparisons in the article. Either use SQLite across the board or use MySQL/Postgres across the board. Or do both. You can even model a self-managed rdbms install on the clouds that don’t have that turnkey offering. But mixing and matching makes no sense.
I’m a huge fan of SQLite and have open sourced some .NET stuff around it (eg https://github.com/neosmart/AspSqliteCache ) but learned a very expensive mistake in using it for an ASP.NET Core Project with the default pattern (i.e. with EF Core).
SQLite locks (tables or the entire db depending on configuration) upon write. If you use shared cache mode and WAL you can get very far with one write thread and many competing reads - depending on shared cache mode, WAL, and other options. I benchmarked the different configurations with one or more writing threads here to show how it scales: https://github.com/mqudsi/sqlite-readers-writers
But this approach is hard to model with EF Core. If you use the default request-scoped DI injected connection, you risk any writes upgrading the read lock to a write lock for the duration of the request. The better approach is to use the default request-scoped connection for RO operations and then request a scoped/transient DI connection for any write ops, but copying internal EF entity tracking state from one EF instance to another is tedious and fraught with issues. You’re at least able to work around this if you try to always keep in mind write transaction lifetimes, though.
The problem comes as soon as you need a “background service” in the sense of “an operation running independently of requests and parallel to them.” If that service needs a write lock for any amount of time, you’re suddenly going to be seeing write timeouts (since default behavior is to poll repeatedly until a write lock is obtained) and that is pretty much impossible to fix.
As one of the biggest advantages of using a resident executor like .NET or Java vs a per-request stateless option like PHP is that you can do stuff independent of requests, SQLite is tricky to use correctly in prod in this model.
The good news is that if you use the SQLite EF provider and run into this, it’s usually not too hard to switch to a real DB provider as a lot of the work is abstracted.
This has been posted and reposted continuously for a year and I still don’t understand the comparisons in the article. Either use SQLite across the board or use MySQL/Postgres across the board. Or do both. You can even model a self-managed rdbms install on the clouds that don’t have that turnkey offering. But mixing and matching makes no sense.
I’m a huge fan of SQLite and have open sourced some .NET stuff around it (eg https://github.com/neosmart/AspSqliteCache ) but learned a very expensive mistake in using it for an ASP.NET Core Project with the default pattern (i.e. with EF Core).
SQLite locks (tables or the entire db depending on configuration) upon write. If you use shared cache mode and WAL you can get very far with one write thread and many competing reads - depending on shared cache mode, WAL, and other options. I benchmarked the different configurations with one or more writing threads here to show how it scales: https://github.com/mqudsi/sqlite-readers-writers
But this approach is hard to model with EF Core. If you use the default request-scoped DI injected connection, you risk any writes upgrading the read lock to a write lock for the duration of the request. The better approach is to use the default request-scoped connection for RO operations and then request a scoped/transient DI connection for any write ops, but copying internal EF entity tracking state from one EF instance to another is tedious and fraught with issues. You’re at least able to work around this if you try to always keep in mind write transaction lifetimes, though.
The problem comes as soon as you need a “background service” in the sense of “an operation running independently of requests and parallel to them.” If that service needs a write lock for any amount of time, you’re suddenly going to be seeing write timeouts (since default behavior is to poll repeatedly until a write lock is obtained) and that is pretty much impossible to fix.
As one of the biggest advantages of using a resident executor like .NET or Java vs a per-request stateless option like PHP is that you can do stuff independent of requests, SQLite is tricky to use correctly in prod in this model.
The good news is that if you use the SQLite EF provider and run into this, it’s usually not too hard to switch to a real DB provider as a lot of the work is abstracted.