Python and Requests-HTML - Web Scraping Dynamic Content from JavaScript applications
In this video, we'll learn how to scrape content that is NOT present in initial page loads, but instead is loaded dynamically by JavaScript.
This is a common problem with scraping the modern web: the initial response contains minimal HTML and a SPA-based JavaScript app (React, Vue, Angular, etc). The data that we want to scrape on the page is therefore not present, but is rendered later via API calls from the SPA application.
We will look at how we can use requests-html to solve this issue in Python when scraping such sites. We'll also look at using this with BeautifulSoup in order to find data on the page.
This video makes use of the following sample website (a React application):
https://react-amazon-bestsellers-book...
📌 𝗖𝗵𝗮𝗽𝘁𝗲𝗿𝘀:
00:00 Intro
02:15 Sending GET request using Python requests library
04:00 Finding objects with BeautifulSoup
05:15 Installing requests-html
06:38 Executing JavaScript on page using requests-html
☕️ 𝗕𝘂𝘆 𝗺𝗲 𝗮 𝗰𝗼𝗳𝗳𝗲𝗲:
To support the channel and encourage new videos, please consider buying me a coffee here:
https://ko-fi.com/bugbytes
𝗦𝗼𝗰𝗶𝗮𝗹 𝗠𝗲𝗱𝗶𝗮:
📖 Blog: https://bugbytes.io/posts/
👾 Github: https://github.com/bugbytes-io/
🐦 Twitter: / bugbytesio
📚 𝗙𝘂𝗿𝘁𝗵𝗲𝗿 𝗿𝗲𝗮𝗱𝗶𝗻𝗴 𝗮𝗻𝗱 𝗶𝗻𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻:
requests-html: https://pypi.org/project/requests-html/
BeautifulSoup: https://beautiful-soup-4.readthedocs....
requests: https://pypi.org/project/requests/
Sample website: https://react-amazon-bestsellers-book...
#python #webscraping #datascience
Auf dieser Seite können Sie das Online-Video Python and Requests-HTML - Web Scraping Dynamic Content from JavaScript applications mit der Dauer stunde minuten sekunde in guter Qualität ansehen, das der Benutzer BugBytes 30 März 2023 hochgeladen hat, den Link mit Freunden und Bekannten teilen, dieses Video wurde auf Youtube bereits 24,199 Mal angesehen und es wurde von 566 den Zuschauern gefallen. Viel Spaß beim Betrachtenden Zuschauern gefallen!
