npbc VS SQLite

Well, there will probably be problems in your life that have a programming solution. For example, we subscribe to something like 5 newspapers billed monthly but with different prices per paper per weekday. We verify the vendor's calculations and it's bit of a chore. Perfect thing to automate with a script!

Another good example is command line arguments. I got started with argv and argc and wrote a rudimentary application with that. At some point I decided to migrate it to Python, and continued to use sys.argv there. Now, we (family) rely on that application, and I use argparse (Python) most of the time. In the context of this application, I'm currently learning about deployment and distribution (hence you'll find my makeshift "installation" instructions in the README). Once I figure that out, I'll switch to a existing tried and tested system.

I'm currently on a new version of the newspaper bill calculator (https://github.com/eccentricOrange/npbc) and it's still teaching me tons (more file i/o, good/bad practices, databases, regex, CI/CD, different kinds of UI like CLIs etc etc)...

The primary purpose of this project is for me to learn to use Flask. It usually takes me 4-5 ~rewrites~ revisions of a project to make it have some structure, and this one is only in v2. I've also had to learn a lot of stuff (Flask, virtually all of JS, some 80% of the CSS, the whole concept of a front- and back-end being split between a CLI and a browser, datetime module), so it's been somewhat tough to build as well. I don't always code like this.

The full codebase is available. At this point, I've completed most of the changes I need to make to the actual code, and push it to GitHub. https://github.com/eccentricOrange/npbc/tree/5c529dacbef0f9a1f8915a49dcca47834204aa09

Link: https://github.com/eccentricOrange/npbc/tree/efd5f37b82a42437a9ed0d61d20a8455dce6f0e0

This post is not a request for review, but if you want to get a sense of the size/number of files, here is a GitHub repo at a specific commit (so that all discussion is consistent). Is it okay to make a post requesting a review of this code? If not, do you know a place I could request someone to take a look?

Make sure that the "complex" problem you're solving is the same as your end goal. That's my main motivator: if I need an app to calculate newspaper bills, I need it. Learning about OOP (for example) outside of a problem where I really benefit from having it doesn't work so well for me.

If you are into alternative storage engines for SQLite, there is also an LSM (Log-Structured Merge-tree) extension in the main repository that is not announced nor documented but seems to work. It’s based on the SQLite 4 project.

https://github.com/sqlite/sqlite/tree/master/ext/lsm1

https://www.charlesleifer.com/blog/lsm-key-value-storage-in-...

The sqlite code base is really well done. Lots of documentation.

https://github.com/sqlite/sqlite

Especially the VM part: https://github.com/spandanb/learndb-py/blob/master/learndb/v...

Compare it with this: https://github.com/sqlite/sqlite/blob/master/src/vdbe.c

That's said, I'm curious how complete this LearnDB is. SQLite is hard to read not only it's old but also it covers a lot of SQL and following SQL spec makes hings complicated. SQLite has great test suite so it's nice if you run the suit against this implementation.

Correct, see the github mirror[1]. I don't know how well supported that feature is compared to main branch. If it was completely stable, then it would have already landed in the main stable branch. Clarity about the roadmap of that branch would be nice.

1. https://github.com/sqlite/sqlite/tree/begin-concurrent

Including comprehensive comments, documentation and tests in a codebase takes time and effort.

Failing to do so creates code that is very difficult to maintain or for someone new to the codebase to understand.

However, time and effort may not be what the organization wants to pay for, and individuals may view their own incomprehensible code as something like job security, as they can't be replaced by someone else easily.

As an example of complicated code that's still well-documented, the open-source sqlite code is a good example, about 1/4 of the B-tree file is comments, every time a variable is defined there's a short note explaining what it's used for, every function has a comment header that's comprehensive, such that someone new to the codebase could construct a map of how it all works fairly quickly. It's a good model for how to avoid the problem:

https://github.com/sqlite/sqlite/blob/master/src/btree.c