Most performance problems in web applications come down to one thing: the database.
With Django, the situation is no different — in fact, in some ways it’s even worse than that of our ORM-hating colleagues. Database performance tuning in Django involves a dizzying number of ORM API methods (
iterator, and many more), mysterious alphabet letters (
F?!), and all of this rests on top of the already complex machinery of SQL queries and indexes.
I’m going to make a bold claim here: The only way to master Django is to master database performance. The difference between junior, mid-level, and senior Django engineers is usually a function of this mastery.
And to fully master database performance with Django, reading through API documentation is not enough. You need to become an expert. Here are some examples of what an expert knows that this book will teach you:
How to use profiling and measurement tools like New Relic, Django Debug Toolbar, and database query plans to see exactly how badly a query performs in production
How to eyeball a query and know exactly the right index to speed it up (a covering index with
INCLUDE? a partial index? a GIN index? You’re going to tell me by the end of the book because YOU will be the expert)
How to use advanced techniques like server-side cursors with
iterator(), custom Func() expressions, and keyset pagination with row comparisons to churn through millions of rows of data with ease
You will learn all of this and more through over 140 pages, 60 examples, and 30 quiz questions. By the end, you won’t ever think of a Django ORM query the same again: it will exist simultaneously in your mind as an operational metric, ORM syntax, generated SQL, and a database execution plan. Because you will be an expert.
I can teach this because I have over ten years of experience working with Django and relational databases, both as an application developer and a platform engineer building databases as a service for Compose.com and IBM Cloud.
And did I mention that you’re going to have fun? Yes, you read that right. This book is chock full of awesome fantasy art including a map of each chapter’s core concepts. Why fantasy art? Because I play tabletop role-playing games and I require books to be fun.
So gaze upon the Sacrificial Cliff of Profiling. Get lost in the Labyrinth of Indexing. Ransack the Crypt of Querying. Then get back to kicking ass and saving the world from unbounded queries!
This book is written for Django developers who use a relational database and want to push their database-related code as fast as it can go. You should have experience with Python and Django. We will generally not explain Django features unless they intersect with database performance.
This means that concepts like URL routing, class-based views, and template rendering appear in the text without much (if any) explanation for how they work. On the other hand, we dive into the
QuerySet API and look at how methods like
iterator() translate into SQL statements and consume Python process memory. We also explore ways to use these API methods and others more effectively from templates, views, and management commands.
The focus of this book is on relational databases like PostgreSQL, MySQL, and SQLite — databases accessible through the core models and migrations systems provided with Django.
You should have experience with relational databases and SQL. However, the reason this book exists is to level up your knowledge of these two topics where they intersect with Django, so we will spend a good amount of time discussing them. Over the next 100+ pages, we will talk often about databases and SQL, discussing how to build efficient database indexes, uses of different types of index, partial indexes, covering indexes, table clustering, types of
JOIN statements, and more.
This book contains technical content like conceptual overviews, explanations of performance problems, and annotated source code examples using Python and SQL code.
This content is the meat of the book: everything you need to know to become an expert in database performance with Django.
But that’s not all! One of the most effective learning tools verified by scientific research is spaced repetition, a memorization technique. That is why this book comes with quiz questions at the end of each chapter that you can turn into index cards or add to your spaced repetition regime. Quizzing yourself is extremely effective at helping you recall the material!
Having only technical content would be fine, but why not spice things up a bit?
To that end, interleaved with the technical content are references to a fantasy adventure in which you explore the Temple of Django Database Performance. As you proceed through the book, you will encounter references to different areas of the Temple and its grounds:
The Sacrificial Cliffs of Profiling
The Labyrinth of Indexing
The Crypt of Querying
Each of these areas relates to a chapter in the book. Every chapter has an illustrated fantasy map that touches on the core technical concepts within the chapter and also relates to the fantasy adventure.
And just as they might in a fantasy adventure, these areas contain Creatures, Traps, and Artifacts, each of which is illustrated with a piece of fantasy art and some text within the chapter.
These aspects of the book are not merely included for fun. They add an important learning dimension in the form of visualizations and metaphors to make the content more engaging and memorable.
Instead of abstract code examples, this book uses a Django application called Quest.
Quest has a mix of some pages rendered purely by Django (e.g., the login form and admin pages) and others rendered partly by Django and partly by React components. It has a REST API powered by Django Rest Framework that the React components consume.
The focus in this book is on code that interacts with the database. That will keep us solidly in Django models and views. At times, we will dip into template rendering and touch on client interactions with the API, but that will be the extent of our work in the front-end.
Data for the application is stored in a PostgreSQL database and accessed through Django models. Many of the SQL and ORM examples used in this book will be database-agnostic, but at times we will use PostgreSQL-specific features. At these times, we will discuss how the feature works in other databases, but you may need to look up the details for your particular database.
Perhaps most importantly, most of the code examples are unit tested and these tests appear in the examples when they help explain complexity.
The Temple of Django Database Performance is a must read for anyone working with Django. You’ll learn about profiling, query plans, indexing, and the Django ORM. Very approachable for those with only modest experience but even the experts should learn something new.
This section lists corrections that new editions have made to errors in older editions.