9 web scraping techniques to access any type of web data

Mastering Web Scraping is essential for those who want to scrape data from the internet. Real estate data, e-commerce and stock prices are just a few examples.

But large websites try to keep this data for themselves, and build in all kind of techniques to prevent you from getting the data you need.

In this video I share 9 essential techniques every scraping professional should know. Spoofing headers, using a proxy server and proxy rotation are just a few examples.

👇 Get $15 Bright Data Credit for Free 👇
https://brdta.com/tomstechacademy

👇 Download the 9 scripts 👇
https://tomstechacademy.com/code-exam...

👇 Bright Data Resources 👇
Proxy server example: https://docs.brightdata.com/proxy-net...

👇 All example websites 👇
Example 1 - Fake housing data: https://www.python-unlimited.com/webs...
Example 2 - Spoofing headers: https://www.scrapethissite.com/pages/...
See your headers: https://httpbin.org/headers
Example 3: - Get your IP: https://ident.me/
Free proxy list: https://free-proxy-list.net/
Example 7: AJAX Scrolling: https://webscraper.io/test-sites/e-co...
Example 8: AJAX Clicking: https://webscraper.io/test-sites/e-co...
Example 9: Download Excel file: https://tomstechacademy.com/wp-conten...

👇 Timestamps 👇
00:00 - Intro
01:19 - Basics of web scraping with Python
10:54 - Spoofing headers
13:49 - Using a proxy server
15:52 - Rotating proxy servers
19:33 - How to use Bright Data residential proxy
21:18 - Scraping with a headful browser and Selenium
23:33 - How to use Selenium for websites with lazy loading
26:05 - How to use Selenium for websites with "more.." button
29:20 - Scraping data with a Web Scraping API - Bright Data

👇 Follow me 👇
💼 LinkedIn:   / thomas--janssen  
✅ Instagram:   / tomstechacademy  

❗ Some of my videos contain affiliate links. When you click on these kind of links, I get a small percentage in case you make a purchase via the website. Be aware that this doesn't cost you anything - you never pay more than the standard retail price.

#webscraping #python #selenium