Скачать или смотреть How to Create New Fields in MongoDB by Mapping Values Across Collections

Learn how to efficiently update documents in MongoDB using PyMongo by mapping values from one collection to another with aggregation pipeline.
---
This video is based on the question https://stackoverflow.com/q/67480283/ asked by the user 'OJT' ( https://stackoverflow.com/u/10634126/ ) and on the answer https://stackoverflow.com/a/67480571/ provided by the user 'Joe' ( https://stackoverflow.com/u/2282634/ ) at 'Stack Overflow' website. Thanks to these great users and Stackexchange community for their contributions.

Visit these links for original content and any more details, such as alternate solutions, latest updates/developments on topic, comments, revision history etc. For example, the original title of the Question was: Mongo create new field by mapping value from one field to value in a field in another collection

Also, Content (except music) licensed under CC BY-SA https://meta.stackexchange.com/help/l...
The original Question post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license, and the original Answer post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license.

If anything seems off to you, please feel free to write me at vlogize [AT] gmail [DOT] com.
---
Creating New Fields in MongoDB by Mapping Values Between Collections

MongoDB is a powerful NoSQL database that offers flexibility and scalability for handling data. One common problem developers face is the need to update documents in one collection based on the values stored in another collection. In this guide, we'll explore how to achieve this by creating a new field in one collection, using values from another collection.

The Problem Statement

Suppose you have two collections in your MongoDB database:

Collection c1: Contains documents with fields such as {'Race': 'blck'}.

Collection c2: Maps raw values to standardized ones, like {'raw': 'blck', 'mapped_race': 'black'}.

The goal is to update documents in c1 by adding a new field called Race_Standardized, containing the corresponding mapped value from c2. Using the above example, the updated document in c1 should look like this: {'Race': 'blck', 'Race_Standardized': 'black'}.

To accomplish this, an aggregation pipeline could be created, but there’s a more efficient method that involves querying c2 and processing the updates using bulk operations in PyMongo.

Solution Overview

Step 1: Querying Collection c2

The first step is to retrieve the mapping documents from collection c2. You'll use a loop to process each document and prepare an update operation for c1.

Step 2: Prepare Bulk Write Operations

Using the retrieved documents from c2, you can create a list of update operations. For each document in c2, you will look for matching Race values in c1 and set the Race_Standardized field accordingly.

Step 3: Execute Bulk Write to c1

Once your updates are prepared, you can execute them in bulk against c1. This approach is much more efficient than updating documents one at a time.

Implementation Using PyMongo

Here’s how you can implement this solution in Python using the PyMongo library:

[[See Video to Reveal this Text or Code Snippet]]

Explanation of the Code

Initialize updates List: Start with an empty list that will hold all your update operations.

Iterate over Collection c2: Use a for loop to go through each document in c2.

Construct UpdateMany Operations: For each document, create an UpdateMany operation that:

Matches all documents in c1 where the Race field equals raw from c2.

Sets the Race_Standardized field to the value in mapped_race.

Bulk Write Execution: Finally, execute all the collected operations with bulk_write, which executes them in a single batch.

Conclusion

By using PyMongo's bulk write capabilities, you can efficiently update documents in one collection based on mappings derived from another collection. This not only streamlines the update process but also significantly enhances performance, especially with larger datasets.

Now that you understand how to create new fields in MongoDB by mapping values from one collection to another, feel free to apply this technique to your own data management projects. Happy coding!