Скачать или смотреть How to Drop Rows from a DataFrame Based on a String in Python Pandas

In this guide, we explore how to drop rows from a DataFrame in Python Pandas that contain a specific string in a column. We also cover how to create a new DataFrame with just those rows.
---
This video is based on the question https://stackoverflow.com/q/77748848/ asked by the user 'Quentin Phillips' ( https://stackoverflow.com/u/23187337/ ) and on the answer https://stackoverflow.com/a/77748897/ provided by the user 'Corralien' ( https://stackoverflow.com/u/15239951/ ) at 'Stack Overflow' website. Thanks to these great users and Stackexchange community for their contributions.

Visit these links for original content and any more details, such as alternate solutions, comments, revision history etc. For example, the original title of the Question was: Dropping rows from dataframe if they contain a string within a series

Also, Content (except music) licensed under CC BY-SA https://meta.stackexchange.com/help/l...
The original Question post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license, and the original Answer post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license.

If anything seems off to you, please feel free to write me at vlogize [AT] gmail [DOT] com.
---
A Guide to Dropping Rows from a DataFrame in Pandas

Working with data in Python can sometimes be a bit tricky, especially when you're trying to filter out unwanted information. One common requirement is to drop rows from a DataFrame based on certain content in a specific column. In this post, we'll dive deep into how to remove rows containing a particular string using the Pandas library in Python, along with how to create a new DataFrame that only contains those rows.

The Problem

Imagine you have a DataFrame with various columns including one named text, which consists of lists of strings. For example, it may look something like this:

Index
Label
Score
Text
ID
0
NEGATIVE
0.98
[perrosloja, JoeBiden, ...]
0
1
NEGATIVE
0.99
[MolinaPvanya, JoeBiden]
1
2
NEGATIVE
0.86
[Iowa4Nikki, ...]
2
...
...
...
...
...

You need to remove any row where the column text contains the name "Joe". You've tried using the str.contains method, but it didn't yield the expected results.

The Solution

To solve this problem, we'll walk through the following steps:

Join Words into a String: The first step is to concatenate the list of strings in the text column into a single string for each row.

Filter Rows Based on Contents: Next, we'll check if the string contains the specified substring ("Joe" in this case).

Create New DataFrames: Finally, we will create two DataFrames: one with the rows containing "Joe" and one without.

Step 1: Joining Words in Each Row

You can use the map function to join the lists of words into single strings. Here’s the code:

[[See Video to Reveal this Text or Code Snippet]]

This transforms each list in the text column into a string sentence. For instance:

[[See Video to Reveal this Text or Code Snippet]]

This would produce:

[[See Video to Reveal this Text or Code Snippet]]

Step 2: Filter Rows

Now that we have our words joined as strings, we can use the str.contains method to filter the rows that contain "Joe":

[[See Video to Reveal this Text or Code Snippet]]

Step 3: Creating New DataFrames

With the boolean mask m, we can easily create two new DataFrames: one that contains rows where text includes "Joe" and another that excludes those rows.

[[See Video to Reveal this Text or Code Snippet]]

Final Output

At this point, you will have your original DataFrame split into two:

joe: Contains all rows that have the word "Joe".

oth: Contains all rows that do not have the word "Joe".

Conclusion

Using Pandas, it's quite straightforward to filter rows from a DataFrame based on specific strings found within a column. By concatenating lists into single strings and leveraging the str.contains method, you can efficiently retire unwanted data and create new, organized DataFrames to work with.

By following the steps outlined in this guide, you can simplify your data manipulation tasks and enhance your workflow in Python. Happy coding!