scrapeOP VS boxball

For my first web scraping project, I wanted to use an existing program from github: https://github.com/Seb943/scrapeOP

I found this code: https://github.com/Seb943/scrapeOP and it works well. But it is setup to grab odds data from one specific page of every event and I need it from another one.

I haven't worked with BI tools, but I'll make the same recommendation for Retrosheet data that I always do: check out the Boxball distributions, specifically the CSV files (or the parquet files if you prefer). You can load those right in, rather than futzing with Retrosheet's event files and processing them manually, etc., just to get to the same spot.

The original data source is the Baseball Databank (aka "The Lahman Database"). I use the Boxball distributions, which include both retrosheet and Baseball Databank data.

You know, I thought about that after posting, that the dailies might not be available directly from retrosheet. I use the Boxball distributions, which take the retrosheet data and process it into database images. These distributions include a "daily" table with the daily logs. So apparently Boxball is generating that table.

I used to think the same thing, and then I found the Boxball pre-built images. You still have to be a bit handy to get them working, but it's way easier than starting from scratch with raw retrosheet data. I initially used the postgres image and worked in SQL, and then switched to python/pandas and the parquet files.

I've always been good with data, math, Excel, etc., but decided to take it to another level during the pandemic. The most difficult thing about Retrosheet data is getting into a usable format. Luckily, the boxball project has created a number of ready-to-download images with all of the data, for different database technologies.

For Retrosheet data, I've been using pre-built images from Boxball, rather than going through the whole process of downloading and converting the files from retrosheet. The Boxball images are awesome. I mostly use the Parquet files for Pandas, but they have other formats (e.g., postgres Docker images, CSV, etc) there.