baseball_sql VS boxball

You'll pick up on this stuff as you read and write more code, and your code will become simpler. Inspired by your work, I took my own pass at implementing Marcels today. Feel free to take a look and ask me any questions.

If you do check it out, and use python/pandas, here is some code I wrote to make the Retrosheet game log data more usable (cells 1-6 specifically). It corrects some inconsistencies in column names, and adds useful columns like year, game_id, game type.

If anybody is curious, my code and details are here.

I started with SQL and then moved to python/pandas. And basically it just took a lot of practice; any time I hear a fun fact or trivia question, I see if I can replicate it myself. Even if I know something is easy to do with Stathead, I've forced myself to code it myself. I moved on to harder and more interesting stuff, like component park factors, aging curves, umpire effects, etc. Some of my code is here.

I haven't worked with BI tools, but I'll make the same recommendation for Retrosheet data that I always do: check out the Boxball distributions, specifically the CSV files (or the parquet files if you prefer). You can load those right in, rather than futzing with Retrosheet's event files and processing them manually, etc., just to get to the same spot.

The original data source is the Baseball Databank (aka "The Lahman Database"). I use the Boxball distributions, which include both retrosheet and Baseball Databank data.

You know, I thought about that after posting, that the dailies might not be available directly from retrosheet. I use the Boxball distributions, which take the retrosheet data and process it into database images. These distributions include a "daily" table with the daily logs. So apparently Boxball is generating that table.

I used to think the same thing, and then I found the Boxball pre-built images. You still have to be a bit handy to get them working, but it's way easier than starting from scratch with raw retrosheet data. I initially used the postgres image and worked in SQL, and then switched to python/pandas and the parquet files.

I've always been good with data, math, Excel, etc., but decided to take it to another level during the pandemic. The most difficult thing about Retrosheet data is getting into a usable format. Luckily, the boxball project has created a number of ready-to-download images with all of the data, for different database technologies.

For Retrosheet data, I've been using pre-built images from Boxball, rather than going through the whole process of downloading and converting the files from retrosheet. The Boxball images are awesome. I mostly use the Parquet files for Pandas, but they have other formats (e.g., postgres Docker images, CSV, etc) there.