Скачать или смотреть How to scrape amazon data using Scala and JSoup - A step by step guide

Learn how to build an Amazon web scraper using Scala, complete with code explanations, error handling, and data extraction techniques.

Disclaimer: This is for educational purposes only. Amazon, and by extension, all other prominent websites have anti-bot, anti-scraping mechanisms in place to discourage you and me from nicking data from them. So, be sure to use techniques like proxy rotation and delayed requesting. This code is not suitable for large scale data scraping, and one should use the REST APIs provided by Amazon for said purpose.

In this tutorial, we’ll guide you through each step of the process—from setting up the scraper, using the JSoup library for HTML parsing, handling errors with Try/Option, to writing data into CSV files. Perfect for beginners and intermediate Scala users, this tutorial provides clear explanations and hands-on coding to help you master web scraping in Scala. By the end, you'll have a fully functional Amazon web scraper that fetches product data like titles, prices, and availability directly from the Amazon website."

Code covered:
Using Scala Options, Try, Success, and Failure for safe web scraping
Fetching product links and data with JSoup selectors
Writing extracted data to CSV in Scala
Whether you’re just getting into web scraping or want to add Scala to your web scraping toolkit, this video is for you.

Github repo of full code:
https://github.com/mineshmelvin

Chapters:
00:00 - 00:51 Intro
00:51 - 02:42 Importing quickstart scala archetype
02:42 - 04:25 The JSoup library
04:25 - 08:42 Building the AmazonWebScraper
08:42 - 09:49 Writing the fetchPage method
09:49 - 12:45 What is Option and why use it
12:45 - 16:11 What is Try and why use it
16:11 - 18:13 Writing the extractProductLinks method
18:13 - 25:08 What is the JSoup document and how to get data from it
25:08 - 34:01 Writing the extractProductData method
34:01 - 43:22 Writing the output to a csv file
43:22 - 45:15 Running the Scraper
45:15 - 46:12 Viewing the output