Web Scraping | Python & SQL | How to track and Compare data changes over time
Comparing Scraped data often involves checking the difference between your current scrape and the previous one. Storing the data in a temp table or 'staging' table can require regular removal of old/redundant data. In this video I show how you can track and remove data based on the order in which it was collected.
If you watch just past halfway you may see an unexpected event as well.
The data i am working with on this project was collected via Selenium to a CSV and I am now in the process of inserting into the correct tables based on specific conditions, AND only if the value of "status" has changed since the previous scrape.
Hope it's useful / interesting?
Dr P.
Thanks to Stackoverflow for this : https://stackoverflow.com/questions/1...
Visit redandgreen blog for more Tutorials
=========================================
🌏 http://redandgreen.co.uk/about/blog/
Subscribe to the YouTube Channel
=================================
🌏 / drpicode
Follow on Twitter - to get notified of new videos
=================================================
🌏 / rngweb
Buy Dr Pi a coffee (or Tea)
☕ https://www.buymeacoffee.com/DrPi
Proxies
=================================================
If you need a good, easy to use proxy, I was recommended this one, and having used ScraperAPI for a while I can vouch for them. If you were going to sign up anyway, then maybe you would be kind enough to use the link and the coupon code below?
You can also do a full working trial first as well, (unlike some other companies). The trial doesn't ask for any payment details either so all good! 👍
🌏 10% off ScraperAPI : https://www.scraperapi.com?fpr=ken49
◼️ Coupon Code: DRPI10
(You can also get started with 1000 free API calls. No credit card required.)
Thumbs up yeah? (cos Algos..)
#webscraping #tutorials #python
Sur cette page du site, vous pouvez voir la vidéo en ligne Web Scraping | Python & SQL | How to track and Compare data changes over time durée heure minute seconde en bonne qualité , qui a été Téléchargé par l'utilisateur jobstr 11 mars 2021, Partagez le lien avec vos amis et connaissances, sur youtube cette vidéo a déjà été regardée 262 fois et il a aimé 5 téléspectateurs. Bon visionnage!
