Python and Requests-HTML - Web Scraping Dynamic Content from JavaScript applications
In this video, we'll learn how to scrape content that is NOT present in initial page loads, but instead is loaded dynamically by JavaScript.
This is a common problem with scraping the modern web: the initial response contains minimal HTML and a SPA-based JavaScript app (React, Vue, Angular, etc). The data that we want to scrape on the page is therefore not present, but is rendered later via API calls from the SPA application.
We will look at how we can use requests-html to solve this issue in Python when scraping such sites. We'll also look at using this with BeautifulSoup in order to find data on the page.
This video makes use of the following sample website (a React application):
https://react-amazon-bestsellers-book...
📌 𝗖𝗵𝗮𝗽𝘁𝗲𝗿𝘀:
00:00 Intro
02:15 Sending GET request using Python requests library
04:00 Finding objects with BeautifulSoup
05:15 Installing requests-html
06:38 Executing JavaScript on page using requests-html
☕️ 𝗕𝘂𝘆 𝗺𝗲 𝗮 𝗰𝗼𝗳𝗳𝗲𝗲:
To support the channel and encourage new videos, please consider buying me a coffee here:
https://ko-fi.com/bugbytes
𝗦𝗼𝗰𝗶𝗮𝗹 𝗠𝗲𝗱𝗶𝗮:
📖 Blog: https://bugbytes.io/posts/
👾 Github: https://github.com/bugbytes-io/
🐦 Twitter: / bugbytesio
📚 𝗙𝘂𝗿𝘁𝗵𝗲𝗿 𝗿𝗲𝗮𝗱𝗶𝗻𝗴 𝗮𝗻𝗱 𝗶𝗻𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻:
requests-html: https://pypi.org/project/requests-html/
BeautifulSoup: https://beautiful-soup-4.readthedocs....
requests: https://pypi.org/project/requests/
Sample website: https://react-amazon-bestsellers-book...
#python #webscraping #datascience
En esta página del sitio puede ver el video en línea Python and Requests-HTML - Web Scraping Dynamic Content from JavaScript applications de Duración hora minuto segunda en buena calidad , que subió el usuario BugBytes 30 marzo 2023, comparta el enlace con amigos y conocidos, en youtube este video ya ha sido visto 24,199 veces y le gustó 566 a los espectadores. Disfruta viendo!
