boxball VS sqlx

I haven't worked with BI tools, but I'll make the same recommendation for Retrosheet data that I always do: check out the Boxball distributions, specifically the CSV files (or the parquet files if you prefer). You can load those right in, rather than futzing with Retrosheet's event files and processing them manually, etc., just to get to the same spot.

The original data source is the Baseball Databank (aka "The Lahman Database"). I use the Boxball distributions, which include both retrosheet and Baseball Databank data.

You know, I thought about that after posting, that the dailies might not be available directly from retrosheet. I use the Boxball distributions, which take the retrosheet data and process it into database images. These distributions include a "daily" table with the daily logs. So apparently Boxball is generating that table.

I used to think the same thing, and then I found the Boxball pre-built images. You still have to be a bit handy to get them working, but it's way easier than starting from scratch with raw retrosheet data. I initially used the postgres image and worked in SQL, and then switched to python/pandas and the parquet files.

I've always been good with data, math, Excel, etc., but decided to take it to another level during the pandemic. The most difficult thing about Retrosheet data is getting into a usable format. Luckily, the boxball project has created a number of ready-to-download images with all of the data, for different database technologies.

For Retrosheet data, I've been using pre-built images from Boxball, rather than going through the whole process of downloading and converting the files from retrosheet. The Boxball images are awesome. I mostly use the Parquet files for Pandas, but they have other formats (e.g., postgres Docker images, CSV, etc) there.

What exactly are you missing? I haven't really written "boring corporate backend stuff" in a few years but something like sqlx provides everything I've ever needed there.

Badges are the little rectangles you typically see at the top of a crate's README: https://github.com/launchbadge/sqlx/blob/main/README.md

SQLX has entered the chat [1].

[1] https://github.com/launchbadge/sqlx

Database : SqLite (using sqlx).

The current (well, removed) MSSQL sqlx driver also doesn't support encrypted connections (https://github.com/launchbadge/sqlx/issues/1552), so it might work locally on an unnamed instance but not over the network until they add in the newer driver (which will be closed source but for an OSS project you should be able to request a free license I think)

I'm considering making my own wrapper over lower-level database drivers. There are a lot of features in sqlx that I don't need, and the latest version seems to have removed useful data structures that SQLPage is using. It also removed support for SQL Server.

add Ed25519 to AuthPlugin enum on https://github.com/launchbadge/sqlx/blob/061fdcabd72896d9bc3abb4ea4af6712a04bc0a8/sqlx-core/src/mysql/connection/auth.rs and implement them using these crates: