Django + Celery + Selenium to Scrape Anything with Python

Publicado el: 18 marzo 2024
en el canal de: CodingEntrepreneurs

62,429

508

🚀 Sign up for Bright Data Now https://brdta.com/justin

Selenium with Bright Data can scrape nearly any website regardless of Captcha or Login required. In this series, you are going to use Django and Celery to ensure you can scrape on your schedule and store the data with excellent structure. We will be tracking Amazon product price fluctuations.

Topics Covered:
⦿ Integrating Django with Celery for scheduled and background task processing
⦿ First-hand experience with Selenium in Jupyter for live browser automation
⦿ Advanced proxy scraping techniques using Bright Data and Selenium including handling complex challenges like Amazon Captchas
⦿ Building helper functions to streamline your scraping module
⦿ Combining Django and Jupyter for an enhanced data-driven development experience
⦿ Data parsing with BeautifulSoup4 to effectively find and extract the data you need
⦿ Efficient data storage using Django models
⦿ Offloading heavy scraping tasks with Celery for improved application performance
⦿ Automating and scheduling scraping tasks within the Django admin using Celery
⦿ Plus, much more in-depth knowledge and practical tips!

Recommended to Watch
⦿ Try Django 3.2 (or similar): • Try Django 3.2 - Python Web Developme...
⦿ 30 Days of Python (up to about day 15): • 30 Days of Python ~ Python 3.8+
⦿ Some HTML / CSS Experience is ideal

Resources
⦿ Code: https://github.com/codingforentrepren...
⦿ Subscribe: https://cfe.sh/youtube
⦿ Django: https://djangoproject.com
⦿ Bright Data: https://brdta.com/justin

🚀 Next up? Web Scraping with Python + AI our Free Udemy Course: https://www.udemy.com/course/smarter-...

00:00:00 - Welcome
00:02:23 - Tools
00:06:07 - Rapid Fire Django + Celery Integration
00:19:06 - Setup Project
00:21:23 - First Look at Selenium with Jupyter
00:25:08 - Proxy Scraping with Bright Data + Selenium
00:32:08 - Helper Functions for Bright Data Module
00:35:55 - Django + Jupyter Integration
00:39:00 - Parsing Data with BeautifulSoup4
00:52:02 - Amazon Captcha + Prepare Parser Helper Functions
00:58:30 - Saved Scraped Data with Django Models
01:10:51 - Celery Task to Offload Scrape-to-Save
01:22:29 - Schedule Scraping Events with Celery in the Django Admin
01:33:16 - Thank you and next steps

En esta página del sitio puede ver el video en línea Django + Celery + Selenium to Scrape Anything with Python de Duración hora minuto segunda en buena calidad , que subió el usuario CodingEntrepreneurs 18 marzo 2024, comparta el enlace con amigos y conocidos, en youtube este video ya ha sido visto 62,429 veces y le gustó 508 a los espectadores. Disfruta viendo!

21,717