pydantic VS SQLAlchemy

The solution? Leverage Pydantic, a Python library that enables runtime data validation using type annotations.

Pydantic author here. We have plans for an improvement to pydantic where JSON is parsed iteratively, which will make way for reading a file as we parse it. Details in https://github.com/pydantic/pydantic/issues/10032.

Our JSON parser, jiter (https://github.com/pydantic/jiter) already supports iterative parsing, so it's "just" a matter of solving the lifetimes in pydantic-core to validate as we parse.

This should make pydantic around 3x faster at parsing JSON and significantly reduce the memory overhead.

Across this five-post series, we’ve journeyed from Pydantic’s basics—type validation and nested models—to advanced integrations with FastAPI, SQLAlchemy, and scalable techniques. You’ve learned how to build declarative, type-safe models, handle complex APIs, and optimize performance. To deepen your knowledge, explore the Pydantic documentation, contribute to the open-source project, or experiment with real-world use cases. Check out our GitHub repo for code samples and a Pydantic cheat sheet. Thank you for joining us—happy coding!

To dive deeper, check out the Pydantic documentation for advanced features like nested models, custom validators, and settings management. Happy coding!

Pydantic is bundled with FastAPI and is excellent for modelling, validating, and serialising API responses.

Pydantic Settings is a python library that extends 🚀 Pydantic for dealing with settings management.

And here is where FastAPI ✨ shines: It leverages Pydantic powerful data validation to offer out of the box JSON Schema and OpenAPI specs via Swagger and Redoc.

If you carefully think about it, the system that extracts raw text from the PDF needs to both detect and render PDF form elements like checkboxes and radiobuttons in a way that LLMs can understand. In this example, we’ll use LLMWhisperer to extract PDF raw text representing checkboxes and radiobuttons. You can use LLMWhisperer completely free for processing up to 100 pages per day. As for structuring the output from LLMWhisperer, we’ll use GPT3.5-Turbo and we’ll use Langchain and Pydantic to help make our job easy.

Two good examples of "builders to SQL" are

https://www.jooq.org/

and

https://www.sqlalchemy.org/

JooQ isn't everybody's taste but I use it for my job and I think it's great particularly in that you can reuse expressions and write generators for complex queries. We have a powerful search interface that combines full-text with other kinds of queries ("Is about topic T", "Data was collected between S and E") that is beautiful. I think it's funny how JooQ has that lispy f(a,b) style (no accident it is like ferocity) and how Sqlalchemy is really fluent and takes advantage of operator overloading.

6. SQLAlchemy - Database Interaction using ORM

Object-Relational Mapping frameworks like Hibernate (Java), SQLAlchemy (Python), and Sequelize (Node.js) typically use parameterized queries by default and abstract direct SQL interaction. These frameworks help eliminate common developer errors that might otherwise introduce vulnerabilities.

SQLAlchemy [0] is still arguably the most featureful and robust ORM in Python (although it is much more than an ORM). It is likely the best default choice unless you're already working within a framework that offers its own ORM.

[0]: https://www.sqlalchemy.org

SQLAlchemy Documentation. "SQLAlchemy ORM." https://www.sqlalchemy.org/.

Let’s start with a simple example of a data pipeline that reads data from a CSV file, processes it, and stores the result in a database. We’ll use Pandas for data manipulation and SQLAlchemy to interact with an SQL database.

We will define our database schema using SQLAlchemy. First, we will store a subset of the information in a single table called documents. This is to test that our database configuration is correct and avoid storing nested data now. The code is fairly simple to create a SQLite3 database with SQLAlchemy:

We've put a special focus on making sure Drizzle, Prisma, SQLAlchemy and Django ORMs work well with our platform. Common administrative and data exploration tools like DataGrip, pgAdmin and TablePlus have been put through the wringer to resolve any compatibility hiccups seen over the past few months.

SQLAlchemy: Python SQL toolkit and ORM (Object Relational Mapper). Use it to work with databases with Python objects.