A Comprehensive Guide to Cross-Validation with Scikit-Learn and Python

Publicado em: 28 Agosto 2023
no canal de: Ryan & Matt Data Science

11,169

323

🧠 Don’t miss out! Get FREE access to my Skool community — packed with resources, tools, and support to help you with Data, Machine Learning, and AI Automations! 📈 https://www.skool.com/data-and-ai-aut...

In this comprehensive tutorial, we dive into the world of model validation using cross-validation techniques with Python and Scikit-Learn. Cross-validation plays a pivotal role in assessing the true performance of machine learning models, helping us build robust and reliable solutions.

🚀 Hire me for Data Work: https://ryanandmattdatascience.com/da...
👨‍💻 Mentorships: https://ryanandmattdatascience.com/me...
📧 Email: ryannolandata@gmail.com
🌐 Website & Blog: https://ryanandmattdatascience.com/
🖥️ Discord:   / discord
📚 *Practice SQL & Python Interview Questions: https://stratascratch.com/?via=ryan
📖 *SQL and Python Courses: https://datacamp.pxf.io/XYD7Qg

🍿 WATCH NEXT
Scikit-Learn and Machine Learning Playlist:    • Scikit-Learn Tutorials - Master Machine Le...
Column Transformer:    • Simplify Data Preprocessing with Python's ...
Cross Validation:    • A Comprehensive Guide to Cross-Validation ...
Feature Scaling:    • Python Feature Scaling in SciKit-Learn (No...

Learn how to implement cross-validation in Python using scikit-learn to build more reliable machine learning models. In this comprehensive tutorial, I break down cross-validation techniques from beginner to intermediate level, showing you why a single train-test split can give biased results and how cross-validation solves this problem.

We start with a simple train-test split example that demonstrates how different random states can produce vastly different accuracy scores (0.71 vs 0.80), highlighting the limitations of this approach. Then I walk you through implementing basic cross-validation with cross_val_score, which provides multiple accuracy measurements instead of just one, giving you a much more reliable assessment of your model's performance.

Throughout the video, you'll learn how to calculate average scores and standard deviations to better understand your model's consistency. I demonstrate how to define K-Fold cross-validation with custom parameters including n_splits, shuffle options, and random states for reproducible results. We also explore stratified K-Fold validation, which maintains class distribution percentages across folds for more balanced testing.

As a bonus, I show you how to use different scoring metrics like F1 and accuracy scores, and most importantly, how to combine cross-validation with scikit-learn pipelines. This advanced technique lets you validate entire preprocessing and modeling workflows together, which is essential for real-world machine learning projects.

By the end of this tutorial, you'll understand when to use cross-validation over simple train-test splits, how to interpret multiple validation scores, and how to integrate these techniques into your machine learning pipeline for more robust model evaluation.

TIMESTAMPS

00:00 Introduction to Cross-Validation
01:12 Setting Up Data and Imports
03:19 Creating X and Y Variables
05:06 Train Test Split Example
07:42 Logistic Regression Model
09:42 Testing with Different Random States
11:52 Basic Cross-Validation Score
14:01 Defining K-Fold
16:10 Stratified K-Fold
19:13 Pipeline Example with Cross-Validation
22:02 Results Overview and Recap

OTHER SOCIALS:
Ryan’s LinkedIn:   / ryan-p-nolan
Matt’s LinkedIn:   / matt-payne-ceo
Twitter/X: https://x.com/RyanMattDS

Who is Ryan
Ryan is a Data Scientist at a fintech company, where he focuses on fraud prevention in underwriting and risk. Before that, he worked as a Data Analyst at a tax software company. He holds a degree in Electrical Engineering from UCF.

Who is Matt
Matt is the founder of Width.ai, an AI and Machine Learning agency. Before starting his own company, he was a Machine Learning Engineer at Capital One.

*This is an affiliate program. We receive a small portion of the final sale at no extra cost to you.

Nesta página do site você pode assistir ao vídeo on-line A Comprehensive Guide to Cross-Validation with Scikit-Learn and Python duração hora minuto segundo em boa qualidade , que foi baixado pelo usuário Ryan & Matt Data Science 28 Agosto 2023, compartilhe o link com seus amigos e conhecidos, no youtube este vídeo já foi visto 11,169 vezes e gostou 323 espectadores. Boa visualização!