Скачать или смотреть Solving django-oscar Query Overhead: How to Optimize Product Serialization

Learn how to reduce query counts in `django-oscar` when adding custom attributes to your ProductSerializer, ensuring better performance for your API.
---
This video is based on the question https://stackoverflow.com/q/76341365/ asked by the user 'Kevin Renskers' ( https://stackoverflow.com/u/403425/ ) and on the answer https://stackoverflow.com/a/76430078/ provided by the user 'Kevin Renskers' ( https://stackoverflow.com/u/403425/ ) at 'Stack Overflow' website. Thanks to these great users and Stackexchange community for their contributions.

Visit these links for original content and any more details, such as alternate solutions, latest updates/developments on topic, comments, revision history etc. For example, the original title of the Question was: django-oscar / django-oscar-api: huge number of queries to render the ProductSerializer

Also, Content (except music) licensed under CC BY-SA https://meta.stackexchange.com/help/l...
The original Question post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license, and the original Answer post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license.

If anything seems off to you, please feel free to write me at vlogize [AT] gmail [DOT] com.
---
Optimizing Query Performance in django-oscar for Custom Product Attributes

When developing with Django and especially using the django-oscar framework, performance can sometimes be a concern—particularly when customizing product serializers with additional attributes. In this post, we’ll explore a common issue faced by developers: a dramatic increase in the number of queries executed when rendering a product serializer. We will also provide an effective solution to keep your query counts in check.

The Problem: Excessive Queries

In a typical scenario using django-oscar, you may define a custom ProductSerializer that includes properties stored as attributes in your product model. Here’s an example of how this might look:

[[See Video to Reveal this Text or Code Snippet]]

However, when a custom property like legacy_title is added, something unexpected happens: the number of database queries skyrockets.

Initially, you might see 3 queries, but the moment you add legacy_title, this can jump to 70 queries or more, especially when additional fields like children, recommendations, and images are included.

With 21 products, this becomes a significant issue, slowing down your API endpoint considerably.

Why Does This Happen?

This excessive querying can be attributed to how Django ORM fetches related objects. Each time a property is accessed that isn't eagerly loaded via select_related or prefetch_related, a new query is sent to the database. This can lead to N+ 1 query problems.

The Solution: Overriding the ProductList View

To gain control over the number of queries, we can override the ProductList view provided by django-oscar-api. This allows us to prefetch related objects intelligently, dramatically reducing the total queries executed.

Step-by-Step Implementation

Create a Custom ProductList Class:
We'll extend the existing ProductList class and modify its get_queryset method to include prefetching.

[[See Video to Reveal this Text or Code Snippet]]

Modify the Product Class to Access Prefetched Attributes:
Utilize the prefetched attributes in your product model to avoid unnecessary database hits when properties are accessed.

[[See Video to Reveal this Text or Code Snippet]]

Function to Handle Prefetched Values

Here’s how you can create a function—get_prefetched_attribute—that returns the prefetched attribute values or falls back to regular querying if necessary:

[[See Video to Reveal this Text or Code Snippet]]

Conclusion

By implementing the above changes, you can significantly reduce the number of queries generated when rendering products with custom attributes in django-oscar. This not only optimizes your API performance but also enhances the overall user experience.

In summary:

Identify attributes that result in excessive queries.

Override the ProductList view.

Use select_related and prefetch_related effectively.

Access prefetched attributes in your product model efficiently.

With these strategies, you can ensure your API is quick and responsive, even as you scale.