Скачать или смотреть How to Extract Only Text in a DIV Tag Using Python Selenium

Discover how to retrieve pure text from a `div` tag while excluding child elements using Python Selenium in this comprehensive guide. Learn practical code solutions and troubleshooting tips.
---
This video is based on the question https://stackoverflow.com/q/73217132/ asked by the user 'CHeN' ( https://stackoverflow.com/u/19629213/ ) and on the answer https://stackoverflow.com/a/73217282/ provided by the user 'Parolla' ( https://stackoverflow.com/u/14803340/ ) at 'Stack Overflow' website. Thanks to these great users and Stackexchange community for their contributions.

Visit these links for original content and any more details, such as alternate solutions, latest updates/developments on topic, comments, revision history etc. For example, the original title of the Question was: How to get only text in div Tag except child element by Python selenium?

Also, Content (except music) licensed under CC BY-SA https://meta.stackexchange.com/help/l...
The original Question post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license, and the original Answer post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license.

If anything seems off to you, please feel free to write me at vlogize [AT] gmail [DOT] com.
---
Extracting Only Text in a DIV Tag Using Python Selenium

When working with web scraping and automation using Python Selenium, you might come across scenarios where you need to extract only specific text from an HTML element, particularly a div tag. In many cases, this div may contain both text nodes and child elements, and the challenge lies in isolating the text you need without including the content from child elements. Let's dive into how to achieve this!

Understanding the Problem

Imagine you have a specific example of HTML code:

[[See Video to Reveal this Text or Code Snippet]]

In this example, the div tag contains the text “APPLE” but also has a child span element containing the text “BANANA”. If we attempt to fetch the text directly from the div with Selenium, both "APPLE" and "BANANA" will be returned. This is because the .text method extracts all text within the tag, including that from child elements.

The Attempted Solution

The initial attempt to isolate the text might involve using XPath like this:

[[See Video to Reveal this Text or Code Snippet]]

However, this approach returns:

[[See Video to Reveal this Text or Code Snippet]]

This isn't what you want. You need to isolate the text “APPLE” only.

A Common XPath Mistake

You might have tried another approach with XPath that looks for the text directly:

[[See Video to Reveal this Text or Code Snippet]]

This results in an error:

[[See Video to Reveal this Text or Code Snippet]]

XPath can't directly return text nodes in this manner, leading to confusion.

The Effective Solution

To achieve the desired result of retrieving only the text “APPLE” from the div, you can utilize the JavaScript execution feature provided by Selenium alongside Python. Follow this approach:

Step-by-Step Instructions

Locate the div Element:
First, identify the div element that contains your target text. You can use CSS selectors for a straightforward approach.

Execute JavaScript to Fetch the Text:
Utilize execute_script to run JavaScript that returns only the first text node.

Here’s the code that encapsulates these steps:

[[See Video to Reveal this Text or Code Snippet]]

Explanation

driver.find_element('css selector', 'div.fr-view-article'): This line grabs the desired HTML div element using its class name.

execute_script('return arguments[0].firstChild;', div): This JavaScript command obtains the first child of the selected element, which in this case is the text node "APPLE".

['textContent']: Finally, it retrieves the content of this text node.

strip(): It helps to remove any leading or trailing whitespace.

Conclusion

By following these steps, you can efficiently extract pure text from a specified div tag using Python Selenium while ignoring any child elements. This technique is particularly useful for web scraping when you need precise content from web pages that have nested HTML structures.

Utilizing JavaScript with Selenium, as shown in this guide, will help you tackle similar challenges effectively and make your automation tasks smoother.

Happy coding!