Scrapy Course – Python Web Scraping for Beginners

Published: 27 April 2023
on channel: freeCodeCamp.org

610,636

12k

The Scrapy Beginners Course will teach you everything you need to learn to start scraping websites at scale using Python Scrapy.

The course covers:
Creating your first Scrapy spider
Crawling through websites & scraping data from each page
Cleaning data with Items & Item Pipelines
Saving data to CSV files, MySQL & Postgres databases
Using fake user-agents & headers to avoid getting blocked
Using proxies to scale up your web scraping without getting banned
Deploying your scraper to the cloud & scheduling it to run periodically

✏️ Course created by Joe Kearney.

⭐️ Resources ⭐️
Course Resources
Scrapy Docs: https://docs.scrapy.org/en/latest/
Course Guide: https://thepythonscrapyplaybook.com/f...
Course Github: https://github.com/orgs/python-scrapy...
The Python Scrapy Playbook: https://thepythonscrapyplaybook.com/

Cloud Environments
Scrapyd: https://github.com/scrapy/scrapyd
ScrapydWeb: https://github.com/my8100/scrapydweb
ScrapeOps Monitor & Scheduler: https://scrapeops.io/monitoring-sched...
Scrapy Cloud: https://www.zyte.com/scrapy-cloud/

Proxies
Proxy Plan Comparison Tool: https://scrapeops.io/proxy-providers/...
ScrapeOps Proxy Aggregator: https://scrapeops.io/proxy-api-aggreg...
Smartproxy: https://smartproxy.com/deals/proxyser...

❤️ Try interactive Python courses we love, right in your browser: https://scrimba.com/freeCodeCamp-Python (Made possible by a grant from our friends at Scrimba)

⭐️ Contents ⭐️
⌨️ (0:00:00) Part 1 - Scrapy & Course Introduction
⌨️ (0:08:22) Part 2 - Setup Virtual Env & Scrapy
⌨️ (0:16:28) Part 3 - Creating a Scrapy Project
⌨️ (0:28:17) Part 4 - Build your First Scrapy Spider
⌨️ (0:55:09) Part 5 - Build Discovery & Extraction Spider
⌨️ (1:20:11) Part 6 - Cleaning Data with Item Pipelines
⌨️ (1:44:19) Part 7 - Saving Data to Files & Databases
⌨️ (2:04:33) Part 8 - Fake User-Agents & Browser Headers
⌨️ (2:40:12) Part 9 - Rotating Proxies & Proxy APIs
⌨️ (3:18:12) Part 10 - Run Spiders in Cloud with Scrapyd
⌨️ (4:03:46) Part 11 - Run Spiders in Cloud with ScrapeOps
⌨️ (4:20:04) Part 12 - Run Spiders in Cloud with Scrapy Cloud
⌨️ (4:30:36) Part 13 - Conclusion & Next Steps

🎉 Thanks to our Champion and Sponsor supporters:
👾 davthecoder
👾 jedi-or-sith
👾 南宮千影
👾 Agustín Kussrow
👾 Nattira Maneerat
👾 Heather Wcislo
👾 Serhiy Kalinets
👾 Justin Hual
👾 Otis Morgan

--

Learn to code for free and get a developer job: https://www.freecodecamp.org

Read hundreds of articles on programming: https://freecodecamp.org/news

On this page of the site you can watch the video online Scrapy Course – Python Web Scraping for Beginners with a duration of hours minute second in good quality, which was uploaded by the user freeCodeCamp.org 27 April 2023, share the link with friends and acquaintances, this video has already been watched 610,636 times on youtube and it was liked by 12 thousand viewers. Enjoy your viewing!

4,309

161