Python Tutorial : Filtering DataFrames

Publié le: 22 mars 2020
sur la chaîne: DataCamp

359

Want to learn more? Take the full course at https://learn.datacamp.com/courses/ma... at your own pace. More than a video, you'll learn hands-on coding & quickly apply skills to your daily work.

---

We have previously seen how indexing and slicing works with Pandas DataFrames and Series.

Filtering is more general tool for selecting part of the data not based on labels or positions, but based on properties of interest of the data itself.

DataFrames have several special methods for filtering.

Underlying filtering is the idea of a Boolean Series.

Here, we ask, "Where in the salt Series is the number of sales recorded greater than 60?"

The Series returned has True or False values according to whether the salt sold in that month was more than 60 units.

In itself, a Boolean Series isn't all that useful, but, when it is used to select from a Series or DataFrame or array, it is very powerful.

When used to make a selection, a Boolean Series is called a filter.

A filter may be used directly between brackets as a logical expression or it may be assigned to another variable name (in this case, the explainer variable enough_salt_sold).

Filters can be combined using standard logical operators like and, or, and not.

Here, we combine the filter (salt bigger than or equal to 50) and the filter (eggs strictly less than 200), first using and (so both must be True), second using or (so either can be True).

Notice the use of parentheses to enforce order of logical operations.

Missing values are commonly encountered in data analysis.

These missing values are sometimes represented by zeros, at other times by the special NaN (Not a Number) value.

Here, let's copy our DataFrame and add a column bacon with some zero elements.

We'll use this new DataFrame df2 to explore Pandas methods for selecting or excluding data slices with zeros or NaNs.

We might ask which columns have all non-zero values with the method all.

This selection excludes the bacon because it has zero entries.

We might also ask which columns have any non-zeros entries.

In this case, there are no zero columns, so all of df2 is returned.

A related question is which rows or columns contain NaN.

We can ask that with dot isnull and notnull.

Combining dot any and dot isnull, returns any columns that have a NaN value.

Conversely, we might want all columns where all data is present, that is, there are no NaN values.

We can combine dot all and dot notnull to get this from the original DataFrame df.

We can also remove rows with missing data is using dropna.

Using how='any', the row 'May' is dropped from df because it does have an NaN entry.

By contrast, how=all would keep this row.

Filtering can be powerful when used to assign values.

We might select the egg sales for those months in which salt sales are high.

Explicitly, we extract the 'eggs' column from df and filter the indices where the corresponding entry of the salt column is larger than 55.

The result is a Series subselected from the eggs column.

This filtering allows us to caculate one column based on another, in whole, or in part.

Based on a filter on one column, we can modify the value in another column.

For example, perhaps with every large salt sale, a few eggs are thrown in for free.

It's your turn now to practice using filters with Pandas Series & DataFrames in the exercises.

#DataCamp #PythonTutorial #ManipulatingDataFrameswithpandas

Sur cette page du site, vous pouvez voir la vidéo en ligne Python Tutorial : Filtering DataFrames durée heure minute seconde en bonne qualité , qui a été Téléchargé par l'utilisateur DataCamp 22 mars 2020, Partagez le lien avec vos amis et connaissances, sur youtube cette vidéo a déjà été regardée 359 fois et il a aimé 3 téléspectateurs. Bon visionnage!

446