Django ORM - built -in ORM DJango

Introduction

When developing web applications in Python, one of the most popular frameworks is Django. A key feature of Django is its built‑in Object‑Relational Mapping (ORM) system, which allows developers to interact with databases using high‑level Python methods instead of writing raw SQL queries. This simplifies development and makes the code more readable and maintainable.

Django ORM is a powerful data‑handling tool that automatically translates Python code into SQL queries, providing protection against SQL injection and cross‑platform compatibility with various database management systems.

What Is an ORM and How Django ORM Works

Object‑Relational Mapping Principles

Object‑Relational Mapping (ORM) is a technology that bridges databases with object‑oriented programming languages such as Python. ORM lets developers work with a database using objects, classes, and methods, making data interaction more intuitive and natural.

Core ORM principles include:

• Abstraction from a specific DBMS
• Automatic connection management
• Protection against SQL injection
• Transaction support
• Query caching

Django ORM as the Bridge Between Python and SQL

Django ORM automatically converts Python‑written queries into corresponding SQL statements executed against the database. This lets developers focus on application logic without worrying about low‑level database interactions.

Benefits of Django ORM:

• Ease of use and learning
• Automatic SQL generation
• Schema migration support
• Built‑in data validation
• Integration with the Django admin interface

Key Components of Django ORM

Models: Define data structures as Python classes, where each class maps to a database table.

Managers: Provide an interface for executing database queries through model methods.

QuerySets: Represent collections of records retrieved from the database and support filtering, ordering, and other operations.

Migrations: Version‑control system for database schemas, tracking changes in data structures.

Django ORM Architecture

Active Record Pattern

Django ORM follows the Active Record pattern, where each model encapsulates both the data structure and the methods to manipulate it. This means model instances contain both data and logic for saving, updating, and deleting.

Lazy Loading

QuerySets in Django ORM use lazy evaluation, meaning the SQL query is executed only when the data is actually needed. This optimizes performance and allows complex queries to be built step‑by‑step.

Migrations System

Django ORM includes a robust migrations system that lets you version schema changes and apply them automatically across development environments.

Installation and Configuration

Installing Django

To start working with Django ORM, install Django and create a new project:

pip install django
django-admin startproject myproject
cd myproject

Creating an App

Then create an app inside the project:

python manage.py startapp myapp

Database Configuration

In settings.py, configure the database connection:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'myproject_db',
        'USER': 'myuser',
        'PASSWORD': 'mypassword',
        'HOST': 'localhost',
        'PORT': '5432',
    }
}

Registering the App

Add the newly created app to settings.py:

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'myapp',
]

Creating Data Models

Basic Model Structure

In your app’s models.py, define the models:

from django.db import models
from django.contrib.auth.models import User

class Author(models.Model):
    name = models.CharField(max_length=100, verbose_name="Author Name")
    birth_date = models.DateField(verbose_name="Birth Date")
    email = models.EmailField(unique=True, verbose_name="Email")
    bio = models.TextField(blank=True, verbose_name="Biography")
    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)
    
    class Meta:
        verbose_name = "Author"
        verbose_name_plural = "Authors"
        ordering = ['name']
    
    def __str__(self):
        return self.name

class Book(models.Model):
    title = models.CharField(max_length=200, verbose_name="Book Title")
    author = models.ForeignKey(Author, on_delete=models.CASCADE, verbose_name="Author")
    published_date = models.DateField(verbose_name="Publication Date")
    isbn = models.CharField(max_length=13, unique=True, verbose_name="ISBN")
    pages = models.PositiveIntegerField(verbose_name="Number of Pages")
    rating = models.DecimalField(max_digits=3, decimal_places=2, null=True, blank=True)
    
    class Meta:
        verbose_name = "Book"
        verbose_name_plural = "Books"
        ordering = ['-published_date']
    
    def __str__(self):
        return self.title

Running Migrations

After defining the models, run migrations to create the corresponding tables in the database:

python manage.py makemigrations
python manage.py migrate

Core CRUD Operations

Creating Records

Django ORM offers several ways to create records:

# Method 1: Instantiate and save
author = Author(name='Leo Tolstoy', birth_date='1828-09-09', email='tolstoy@example.com')
author.save()

# Method 2: Use create()
author = Author.objects.create(
    name='Fyodor Dostoevsky',
    birth_date='1821-11-11',
    email='dostoevsky@example.com'
)

# Method 3: Use get_or_create()
author, created = Author.objects.get_or_create(
    email='pushkin@example.com',
    defaults={'name': 'Alexander Pushkin', 'birth_date': '1799-06-06'}
)

Reading Records

# Retrieve all records
authors = Author.objects.all()

# Retrieve a single record
tolstoy = Author.objects.get(name='Leo Tolstoy')

# Filter with condition
young_authors = Author.objects.filter(birth_date__year__gt=1850)

# First / last record
first_author = Author.objects.first()
last_author = Author.objects.last()

# Existence check
exists = Author.objects.filter(name='Leo Tolstoy').exists()

Updating Records

# Update a single record
tolstoy = Author.objects.get(name='Leo Tolstoy')
tolstoy.bio = 'Great Russian writer'
tolstoy.save()

# Bulk update
Author.objects.filter(birth_date__year__lt=1800).update(bio='Literary classic')

# Update using F()
from django.db.models import F
Book.objects.filter(rating__lt=3.0).update(rating=F('rating') + 1)

Deleting Records

# Delete a single record
author = Author.objects.get(name='Leo Tolstoy')
author.delete()

# Bulk delete
Author.objects.filter(birth_date__year__lt=1800).delete()

# Delete all records
Author.objects.all().delete()

Working with QuerySets

Fundamental QuerySet Methods

QuerySets represent collections of database rows and support method chaining:

# Method chaining example
popular_books = Book.objects.filter(
    rating__gte=4.0
).exclude(
    published_date__year__lt=2000
).order_by('-rating')[:10]

# Count records
count = Book.objects.filter(rating__gte=4.0).count()

# Existence check
exists = Book.objects.filter(title__icontains='war').exists()

# Retrieve field values
titles = Book.objects.values_list('title', flat=True)
book_data = Book.objects.values('title', 'rating')

Filtering and Searching

# Exact match
books = Book.objects.filter(title='War and Peace')

# Case‑insensitive contains
books = Book.objects.filter(title__icontains='war')

# Date range
from datetime import date
books = Book.objects.filter(
    published_date__range=[date(1850, 1, 1), date(1900, 12, 31)]
)

# Lookups across relationships
books = Book.objects.filter(author__name='Leo Tolstoy')

# Complex queries with Q objects
from django.db.models import Q
books = Book.objects.filter(
    Q(title__icontains='war') | Q(title__icontains='peace')
)

Model Relationships

One‑to‑Many (ForeignKey)

class Publisher(models.Model):
    name = models.CharField(max_length=100)
    address = models.TextField()

class Book(models.Model):
    title = models.CharField(max_length=200)
    publisher = models.ForeignKey(Publisher, on_delete=models.CASCADE)
    
# Usage
publisher = Publisher.objects.create(name='Eksmo', address='Moscow')
book = Book.objects.create(title='New Book', publisher=publisher)

# Reverse lookup
books = publisher.book_set.all()

Many‑to‑Many (ManyToManyField)

class Genre(models.Model):
    name = models.CharField(max_length=50)

class Book(models.Model):
    title = models.CharField(max_length=200)
    genres = models.ManyToManyField(Genre, blank=True)

# Usage
book = Book.objects.create(title='War and Peace')
genre1 = Genre.objects.create(name='Novel')
genre2 = Genre.objects.create(name='Historical Fiction')

book.genres.add(genre1, genre2)
book.genres.remove(genre1)
book.genres.clear()

One‑to‑One (OneToOneField)

class AuthorProfile(models.Model):
    author = models.OneToOneField(Author, on_delete=models.CASCADE)
    website = models.URLField(blank=True)
    social_media = models.JSONField(default=dict)

# Usage
author = Author.objects.create(name='Leo Tolstoy')
profile = AuthorProfile.objects.create(
    author=author,
    website='https://tolstoy.ru'
)

Advanced Query Capabilities

Aggregation and Annotation

from django.db.models import Count, Avg, Sum, Max, Min

# Aggregation (returns a dict)
stats = Book.objects.aggregate(
    avg_rating=Avg('rating'),
    max_pages=Max('pages'),
    total_books=Count('id')
)

# Annotation (adds fields to each object)
authors_with_counts = Author.objects.annotate(
    book_count=Count('book'),
    avg_rating=Avg('book__rating')
)

# Grouping by fields
from django.db.models import TruncYear
books_by_year = Book.objects.annotate(
    year=TruncYear('published_date')
).values('year').annotate(count=Count('id'))

Subquery‑Based Complex Queries

from django.db.models import OuterRef, Subquery

# Subquery to fetch latest book per author
latest_books = Book.objects.filter(
    author=OuterRef('pk')
).order_by('-published_date')

authors_with_latest_book = Author.objects.annotate(
    latest_book_title=Subquery(latest_books.values('title')[:1])
)

Conditional Expressions

from django.db.models import Case, When, Value, CharField

# Conditional category based on rating
books_with_category = Book.objects.annotate(
    category=Case(
        When(rating__gte=4.5, then=Value('Excellent')),
        When(rating__gte=3.5, then=Value('Good')),
        When(rating__gte=2.5, then=Value('Average')),
        default=Value('Poor'),
        output_field=CharField()
    )
)

Performance Optimization

Eager Loading of Related Objects

# select_related for FK and OneToOne
books = Book.objects.select_related('author', 'publisher').all()

# prefetch_related for M2M and reverse FK
authors = Author.objects.prefetch_related('book_set').all()

# Combined usage
books = Book.objects.select_related('author').prefetch_related('genres').all()

Using only() and defer()

# Load only selected fields
books = Book.objects.only('title', 'rating').all()

# Defer loading of specific fields
books = Book.objects.defer('bio').all()

Database Indexes

class Book(models.Model):
    title = models.CharField(max_length=200, db_index=True)
    
    class Meta:
        indexes = [
            models.Index(fields=['title', 'published_date']),
            models.Index(fields=['rating'], name='book_rating_idx'),
        ]

Comprehensive Django ORM Methods Table

Core QuerySet Methods

Relationship Management Methods

Field Lookup Operators

Aggregation Functions

Date Functions

Custom Managers and QuerySets

Creating a Custom Manager

class PublishedBookManager(models.Manager):
    def get_queryset(self):
        return super().get_queryset().filter(published_date__isnull=False)
    
    def by_author(self, author_name):
        return self.get_queryset().filter(author__name=author_name)
    
    def popular(self):
        return self.get_queryset().filter(rating__gte=4.0)

class Book(models.Model):
    title = models.CharField(max_length=200)
    author = models.ForeignKey(Author, on_delete=models.CASCADE)
    published_date = models.DateField(null=True, blank=True)
    rating = models.DecimalField(max_digits=3, decimal_places=2)
    
    objects = models.Manager()               # Default manager
    published = PublishedBookManager()       # Custom manager

Custom QuerySet

class BookQuerySet(models.QuerySet):
    def published(self):
        return self.filter(published_date__isnull=False)
    
    def by_author(self, author_name):
        return self.filter(author__name=author_name)
    
    def popular(self):
        return self.filter(rating__gte=4.0)
    
    def recent(self):
        return self.filter(published_date__year__gte=2020)

class Book(models.Model):
    title = models.CharField(max_length=200)
    author = models.ForeignKey(Author, on_delete=models.CASCADE)
    published_date = models.DateField(null=True, blank=True)
    rating = models.DecimalField(max_digits=3, decimal_places=2)
    
    objects = BookQuerySet.as_manager()

Working with Raw SQL

Using raw()

# Execute a raw SQL query
books = Book.objects.raw('SELECT * FROM myapp_book WHERE rating > %s', [4.0])

# Map to custom column names
books = Book.objects.raw('SELECT id, title AS book_title FROM myapp_book')

Using connection

from django.db import connection

def get_book_statistics():
    with connection.cursor() as cursor:
        cursor.execute("""
            SELECT 
                author_id,
                COUNT(*) AS book_count,
                AVG(rating) AS avg_rating
            FROM myapp_book 
            GROUP BY author_id
        """)
        return cursor.fetchall()

Transactions

Transaction Management

from django.db import transaction

# Function decorator
@transaction.atomic
def create_book_with_author():
    author = Author.objects.create(name='New Author')
    book = Book.objects.create(title='New Book', author=author)
    return book

# Context manager
def transfer_books(from_author, to_author):
    with transaction.atomic():
        books = Book.objects.filter(author=from_author)
        books.update(author=to_author)
        
# Savepoint example
def complex_operation():
    with transaction.atomic():
        # Main operation
        author = Author.objects.create(name='Author')
        
        # Create a savepoint
        savepoint = transaction.savepoint()
        try:
            # Risky operation
            book = Book.objects.create(title='Book', author=author)
            transaction.savepoint_commit(savepoint)
        except Exception:
            transaction.savepoint_rollback(savepoint)

Django Signals

Core Signals

from django.db.models.signals import post_save, pre_save, post_delete
from django.dispatch import receiver

@receiver(post_save, sender=Book)
def book_saved(sender, instance, created, **kwargs):
    if created:
        print(f"New book created: {instance.title}")
    else:
        print(f"Book updated: {instance.title}")

@receiver(pre_save, sender=Book)
def book_pre_save(sender, instance, **kwargs):
    # Logic before saving
    if not instance.slug:
        instance.slug = slugify(instance.title)

@receiver(post_delete, sender=Book)
def book_deleted(sender, instance, **kwargs):
    print(f"Book deleted: {instance.title}")

Query Caching

QuerySet‑Level Caching

from django.core.cache import cache

def get_popular_books():
    cache_key = 'popular_books'
    books = cache.get(cache_key)
    
    if books is None:
        books = list(Book.objects.filter(rating__gte=4.0).select_related('author'))
        cache.set(cache_key, books, 3600)  # Cache for 1 hour
    
    return books

Model‑Level Caching

class Book(models.Model):
    title = models.CharField(max_length=200)
    rating = models.DecimalField(max_digits=3, decimal_places=2)
    
    @property
    def cached_rating_category(self):
        cache_key = f'book_rating_category_{self.id}'
        category = cache.get(cache_key)
        
        if category is None:
            if self.rating >= 4.5:
                category = 'Excellent'
            elif self.rating >= 3.5:
                category = 'Good'
            else:
                category = 'Average'
            cache.set(cache_key, category, 1800)  # 30 minutes
        
        return category

Testing Models

Creating Model Tests

from django.test import TestCase
from django.core.exceptions import ValidationError
from myapp.models import Author, Book

class AuthorModelTest(TestCase):
    def setUp(self):
        self.author = Author.objects.create(
            name='Test Author',
            birth_date='1900-01-01',
            email='test@example.com'
        )
    
    def test_author_creation(self):
        self.assertEqual(self.author.name, 'Test Author')
        self.assertEqual(str(self.author), 'Test Author')
    
    def test_author_email_unique(self):
        with self.assertRaises(ValidationError):
            Author.objects.create(
                name='Another Author',
                birth_date='1900-01-01',
                email='test@example.com'  # Duplicate email
            )
    
    def test_book_creation(self):
        book = Book.objects.create(
            title='Test Book',
            author=self.author,
            published_date='2023-01-01'
        )
        self.assertEqual(book.author, self.author)
        self.assertEqual(self.author.book_set.count(), 1)

Testing QuerySets

class BookQuerySetTest(TestCase):
    def setUp(self):
        self.author = Author.objects.create(
            name='Test Author',
            birth_date='1900-01-01',
            email='test@example.com'
        )
        Book.objects.create(
            title='Popular Book',
            author=self.author,
            rating=4.5,
            published_date='2023-01-01'
        )
        Book.objects.create(
            title='Regular Book',
            author=self.author,
            rating=3.0,
            published_date='2023-01-01'
        )
    
    def test_popular_books_filter(self):
        popular_books = Book.objects.filter(rating__gte=4.0)
        self.assertEqual(popular_books.count(), 1)
        self.assertEqual(popular_books.first().title, 'Popular Book')

Database Migrations

Creating Migrations

# Generate migration files for model changes
python manage.py makemigrations

# Create an empty migration
python manage.py makemigrations --empty myapp

# Apply migrations
python manage.py migrate

# View the SQL generated by a migration
python manage.py sqlmigrate myapp 0001

Custom Migrations

from django.db import migrations
from django.db.models import F

def update_book_ratings(apps, schema_editor):
    Book = apps.get_model('myapp', 'Book')
    Book.objects.filter(rating__isnull=True).update(rating=0)

def reverse_update_book_ratings(apps, schema_editor):
    Book = apps.get_model('myapp', 'Book')
    Book.objects.filter(rating=0).update(rating=None)

class Migration(migrations.Migration):
    dependencies = [
        ('myapp', '0001_initial'),
    ]
    
    operations = [
        migrations.RunPython(
            update_book_ratings,
            reverse_update_book_ratings
        ),
    ]

Frequently Asked Questions

How can I avoid duplicate queries when accessing related objects?

Use select_related() for ForeignKey relationships and prefetch_related() for ManyToMany relationships:

# Inefficient – N+1 queries
books = Book.objects.all()
for book in books:
    print(book.author.name)  # Triggers a query each iteration

# Efficient – single query
books = Book.objects.select_related('author').all()
for book in books:
    print(book.author.name)  # Data already loaded

How do I perform bulk operations efficiently?

Use bulk_create(), bulk_update(), and update():

# Bulk creation
books = [Book(title=f'Book {i}', author=author) for i in range(1000)]
Book.objects.bulk_create(books)

# Bulk update
Book.objects.filter(rating__isnull=True).update(rating=0)

How can I work with JSON fields in Django?

Use JSONField to store JSON data:

class Book(models.Model):
    title = models.CharField(max_length=200)
    metadata = models.JSONField(default=dict)

# Creation
book = Book.objects.create(
    title='Book',
    metadata={'tags': ['science fiction', 'novel'], 'pages': 300}
)

# Querying JSON
books = Book.objects.filter(metadata__tags__contains=['science fiction'])

How do I create a composite index?

Define composite indexes in the model’s Meta.indexes:

class Book(models.Model):
    title = models.CharField(max_length=200)
    author = models.ForeignKey(Author, on_delete=models.CASCADE)
    published_date = models.DateField()
    
    class Meta:
        indexes = [
            models.Index(fields=['author', 'published_date']),
            models.Index(fields=['title'], name='book_title_idx'),
        ]

How can I optimize queries on large datasets?

Use pagination, iterator(), and chunk_size:

# Pagination
from django.core.paginator import Paginator

books = Book.objects.all()
paginator = Paginator(books, 25)
page = paginator.get_page(1)

# Iterating over a huge queryset
for book in Book.objects.iterator(chunk_size=1000):
    # Process each book
    process_book(book)

Comparison with Other ORMs

Best Practices

Model Organization

1. Use meaningful names: Name models, fields, and methods clearly.
2. Add Meta information: Use verbose_name, ordering, indexes, etc.
3. Validate data: Implement custom validators where needed.
4. Define __str__: Always provide a readable representation.

Query Optimization

1. Leverage select_related and prefetch_related: Avoid N+1 queries.
2. Limit the selected fields: Use only(), defer(), values().
3. Index frequently queried columns: Add db_index=True where appropriate.
4. Monitor queries: Use Django Debug Toolbar or similar tools.

Security

1. Prevent SQL injection: Always use parameterised queries.
2. Validate input data: Clean data before saving.
3. Restrict access: Apply permissions and filters appropriately.

When to Use Django ORM

Django ORM is ideal for:

• Rapid development: Quickly prototype or build an MVP.
• Standard web applications: Blogs, e‑commerce sites, CRM systems.
• Administrative interfaces: Built‑in Django admin.
• API development: Together with Django REST Framework.
• Educational projects: Easy to learn and use.

It may be less suitable for:

• High‑traffic systems: Where maximum performance is critical.
• Complex analytical queries: Requiring specialized SQL constructs.
• Microservices: When a minimalistic architecture is preferred.
• NoSQL workloads: Limited support for document databases.

Conclusion

Django ORM is a powerful and convenient tool for handling databases in Python applications. It offers a high level of abstraction, security, and performance, making web development more efficient and enjoyable.

Key advantages of Django ORM include ease of use, automatic protection against SQL injection, a built‑in migrations system, and seamless integration with the rest of the Django ecosystem. While some scenarios may require more specialized solutions, Django ORM remains an excellent choice for the majority of web projects.

Mastering Django ORM opens up vast opportunities for building modern, scalable, and secure web applications with Python.