Скачать или смотреть Finding Rows in a Pandas DataFrame with List Values that Match a Given List using Regex

Learn how to filter a Pandas DataFrame that contains lists in its columns to find rows matching a given list using regex.
---
This video is based on the question https://stackoverflow.com/q/65306502/ asked by the user 'Monta' ( https://stackoverflow.com/u/9288498/ ) and on the answer https://stackoverflow.com/a/65308255/ provided by the user 'cookesd' ( https://stackoverflow.com/u/13716967/ ) at 'Stack Overflow' website. Thanks to these great users and Stackexchange community for their contributions.

Visit these links for original content and any more details, such as alternate solutions, latest updates/developments on topic, comments, revision history etc. For example, the original title of the Question was: Checking if a pandas dataframe column(that has lists as values) has one element of another list

Also, Content (except music) licensed under CC BY-SA https://meta.stackexchange.com/help/l...
The original Question post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license, and the original Answer post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license.

If anything seems off to you, please feel free to write me at vlogize [AT] gmail [DOT] com.
---
How to Filter a Pandas DataFrame for Specific List Elements with Regex

When working with data in Pandas, you might come across scenarios where your DataFrame contains list values in certain columns. This becomes tricky when you want to filter rows based on another list of keywords. In this guide, we’ll tackle this problem step by step.

The Problem

Imagine you have a DataFrame named files_to_export with the following structure:

[[See Video to Reveal this Text or Code Snippet]]

You also have a list of companies, for instance:

[[See Video to Reveal this Text or Code Snippet]]

Your goal is to filter files_to_export to include only those rows where the Assignee lists contain at least one of the keywords from your Companies list. This means that if a list contains "Ford global tech," it should match due to the presence of "Ford."

The Initial Attempt

You might have tried a piece of code similar to this:

[[See Video to Reveal this Text or Code Snippet]]

However, this might have returned an empty DataFrame, which is not the result you were expecting.

The Solution

Step 1: Set Up Your Data

First, ensure your DataFrame and list are created correctly:

[[See Video to Reveal this Text or Code Snippet]]

Step 2: Create a Regex Pattern

Next, we need to create a regex pattern from the list of companies. This can be done by joining the companies with the 'or' (|) operator:

[[See Video to Reveal this Text or Code Snippet]]

Step 3: Convert Lists to Strings

Since you're dealing with a column that contains lists, you will need to convert these lists into a format that can be matched against the regex pattern. The following code transforms the lists into comma-separated strings:

[[See Video to Reveal this Text or Code Snippet]]

Step 4: Filter the DataFrame

Finally, use the str.contains() method to filter the DataFrame based on the regex pattern. The complete code looks like this:

[[See Video to Reveal this Text or Code Snippet]]

Summary of the Code

Here’s the complete breakdown of the steps combined:

[[See Video to Reveal this Text or Code Snippet]]

Conclusion

By using the code provided, you will successfully filter your DataFrame to only include rows that contain at least one of the specified companies in the Assignee lists. This method leverages the power of regex and the flexibility of Pandas, making it an effective solution for dealing with complex data structures.

Don’t hesitate to try this approach in your own projects, and feel free to reach out if you have further questions or encounter challenges along the way!