pydantic VS CPython

The solution? Leverage Pydantic, a Python library that enables runtime data validation using type annotations.

Pydantic author here. We have plans for an improvement to pydantic where JSON is parsed iteratively, which will make way for reading a file as we parse it. Details in https://github.com/pydantic/pydantic/issues/10032.

Our JSON parser, jiter (https://github.com/pydantic/jiter) already supports iterative parsing, so it's "just" a matter of solving the lifetimes in pydantic-core to validate as we parse.

This should make pydantic around 3x faster at parsing JSON and significantly reduce the memory overhead.

Across this five-post series, we’ve journeyed from Pydantic’s basics—type validation and nested models—to advanced integrations with FastAPI, SQLAlchemy, and scalable techniques. You’ve learned how to build declarative, type-safe models, handle complex APIs, and optimize performance. To deepen your knowledge, explore the Pydantic documentation, contribute to the open-source project, or experiment with real-world use cases. Check out our GitHub repo for code samples and a Pydantic cheat sheet. Thank you for joining us—happy coding!

To dive deeper, check out the Pydantic documentation for advanced features like nested models, custom validators, and settings management. Happy coding!

Pydantic is bundled with FastAPI and is excellent for modelling, validating, and serialising API responses.

Pydantic Settings is a python library that extends 🚀 Pydantic for dealing with settings management.

And here is where FastAPI ✨ shines: It leverages Pydantic powerful data validation to offer out of the box JSON Schema and OpenAPI specs via Swagger and Redoc.

If you carefully think about it, the system that extracts raw text from the PDF needs to both detect and render PDF form elements like checkboxes and radiobuttons in a way that LLMs can understand. In this example, we’ll use LLMWhisperer to extract PDF raw text representing checkboxes and radiobuttons. You can use LLMWhisperer completely free for processing up to 100 pages per day. As for structuring the output from LLMWhisperer, we’ll use GPT3.5-Turbo and we’ll use Langchain and Pydantic to help make our job easy.

import requests import time import threading # List of URLs to download (example sites) urls = [ "https://www.example.com", "https://www.python.org", "https://httpbin.org/delay/2", # artificial delay "https://www.github.com", "https://www.wikipedia.org", ] # Function to download content from a URL def download_url(url): print(f"Starting download: {url}") response = requests.get(url) print(f"Finished downloading {url} (size={len(response.text)} characters)") # Run sequentially (one after another) def run_sequential(): start = time.time() for url in urls: download_url(url) end = time.time() print(f"\nSequential time: {end - start:.2f} seconds\n") # Run with threads (concurrently) def run_with_threads(): start = time.time() threads = [] for url in urls: thread = threading.Thread(target=download_url, args=(url,)) threads.append(thread) thread.start() # Wait for all threads to finish for thread in threads: thread.join() end = time.time() print(f"\nThreaded time: {end - start:.2f} seconds\n") # Run the two scenarios if __name__ == "__main__": print("=== Sequential ===") run_sequential() print("=== With Threads ===") run_with_threads()

Also, you say "temporary variable" but it's no more temporary than any other local variable, because lexical scoping is evidently for children

I have always hated that comprehension variables leak for that same reason

1: https://github.com/python/cpython/blob/v3.13.7/Grammar/pytho...

Python, the programming language used in Flask. This tutorial is meant for version 3 or later.

Python installed on your computer.

AI starts with math and coding. You don’t need a PhD—just high school math like algebra and some geometry. Linear algebra (think matrices) and calculus (like slopes) help understand how AI models work. Python is the main language for AI, thanks to tools like TensorFlow and NumPy. If you know JavaScript from Vue.js, Python’s syntax is straightforward.

It's unreasonable to ask me to chase down a reference in a book but I did it anyway because I was really curious.

It doesn't support what you said on your previous comment. It was a bit of a word salad but seemed to suggest that any calls to your overridden methods might magically use the base class instead, even when director called from application code. But the book says something very different:

> The code of the built-ins (written in C) usually does not call methods overridden by user-defined classes.

But that is nothing to do with built in classes or even being written in C. Any class, even written in pure Python, is not guaranteed to implement any of its method in terms of other public methods, even when it's possible to do that. It just depends on how it's implemented.

Indeed that's true even for UserDict. It's no longer implemented in pure Python, but if you look at the 2.7 version [1], which was, you can see some methods implemented in terms of others (e.g. get() uses `key not in self` and `self[key]`) and others that aren't (e.g. keys() just uses self.dict.keys() rather than being implemented in terms of self.items()).

There was even a breaking change in Python 3.12 to which other methods UerDict.__getitem__() uses in its implementation [2]. So it really makes no difference: either way, you'll have to override all methods that you want to behave differently (and use).

[1] https://github.com/enthought/Python-2.7.3/blob/master/Lib/Us...

[2] https://github.com/python/cpython/issues/105524

I requested an ISO 8601 date parser in the Python "datetime" library in 2012.[1] "datetime" could format into ISO 8601, but not parse strings. There were five ISO 8601 parsers available, all bad. After six years of bikeshedding, it was was fixed in 2018.

That's what it took to not write my own date parsing library.

[1] https://github.com/python/cpython/issues/60077

Install Python3.11 from python.org

https://rr-project.org/ is a great Time-Travel debugger and has the capability to record and replay the execution of the CPython interpreter, but since it depends on GDB (which just knows how to inspect compiled programs) it struggles to present the state of the Python program running inside of the interpreter.

I've found that the best way to inspect Python programs with GDB is to use the GDB extensions (https://github.com/python/cpython/blob/3.13/Tools/gdb/libpyt...) that the CPython project maintains. It's a bunch of code that tells GDB how to navigate the interpreter internals and how to find things like the local variables and the backtrace of the currently executing function. You can actually see quite a lot using this! It falls short if you need to do any kind of navigation or stepping, but it's great for understanding what your program is doing "right now".

At my employer https://undo.io we makes a Time-Travel debugger called UDB which is similar to rr and we've had some success introducing time-travel into this Python debugging approach. We can record and replay Python processes, and with some additions on top of the GDB extensions from CPython we can navigate around the execution history. It's a bit rough around the edges, but it's really powerful to be able to evaluate Python expressions at arbitrary points in time, and we're finding it valuable to use internally. Here are some more details if you're interested (you should be able to adapt this to rr): https://undo.io/resources/how-i-debug-python-code-with-a-tim....

I expect that this new remote debugging protocol will be useful to some of the approaches we're currently testing out.

Working knowledge of a backend language like Node.js or Python