5. Traversing DataFrame Elements using
Part 2 : Data Frame Continued ................................
5. Traversing DataFrame Elements using
i. iterrows() , iteritems() and itertuples()
To iterate over the rows of the DataFrame, we can use the following functions -
iteritems() - to iterate over the (key,value) pairs
iterrows() - iterate over the rows as (index,series) pairs
itertuples() - iterate over the rows as named tuples
Lets take an example >>>import pandas as pd >>>import numpy as np >>>d={'Name':['Shalini','Varsha','Shanti','Madhu'],'Age':[23,56,54,34]} >>> print df
Name Age 0 Shalini 23 1 Varsha 56 2 Shanti 54 3 Madhu 34
5.1 USE of iterrows()
>>>print('ITER ROWS') >>>for key,values in df.iterrows():
for val in values: print('Hello',val)
ITER ROWS Hello Shalini Hello 23 Hello Varsha Hello 56 Hello Shanti Hello 54 Hello Madhu Hello 34
5.2 USE of iteritems()
>>>print('ITER ITEMS')
>>>for key,values in df.iteritems():
for val in values: print('Hello',val)
ITER ITEMS Hello Shalini Hello Varsha Hello Shanti Hello Madhu Hello 23 Hello 56 Hello 54 Hello 34
pythonclassroomdiary.
by Sangeeta M Chauhan, PGT CS , KV NO.3 Gwalior
5.3 USE of itertuples()
>>>print('ITER TUPLES')
>>>for rows in df.itertuples():
print(rows)
ITER TUPLES Pandas(Index=0, Name='Shalini', Age=23) Pandas(Index=1, Name='Varsha', Age=56) Pandas(Index=2, Name='Shanti', Age=54) Pandas(Index=3, Name='Madhu', Age=34)
6. Binary Operations in a DataFrame (add, sub, mul, div, radd , rsub) : Lets take a DataFrames with numeric data
>>> s1=[[1,2,3],[4,5,6]]
>>> s2=[[3,2,5],[5,7,8]]
>>> s3=[[5,5,5],[4,4,4]]
>>> dfr1=pd.DataFrame(s1)
>>> dfr2=pd.DataFrame(s2)
>>> dfr3=pd.DataFrame(s3)
6.1
ADDITION
>>> dfr1
0 1 2 0 1 2 3 1 4 5 6
Created three data frames namely dfr1, dfr2 and dfr3
>>> dfr2 0 1 2
0 3 2 5 1 5 7 8
>>> dfr3 0 1 2
0 5 5 5 1 4 4 4
An individual value or a Data frame can be added to another Dataframe
>>> dfr1+2
0 1 2 0 3 4 5 1 6 7 8
Here 2 is added to each element of Data Frame dfr2
pythonclassroomdiary. by Sangeeta M Chauhan, PGT CS , KV NO.3 Gwalior
>>> dfr1+dfr2
0 1 2 04 4 8 1 9 12 14
>>> dfr1.add(dfr2)
0 1 2 04 4 8 1 9 12 14
>>> dfr1.radd(dfr2)
0 1 2 04 4 8 1 9 12 14
>>> dfr3+dfr1+dfr2
0 1 2 0 9 9 13 1 13 16 18
6.2
SUBTRACTION
>>> dfr1-dfr2
0 1 2 0 -2 0 -2 1 -1 -2 -2
>>> dfr1.sub(dfr2)
0 1 2 0 -2 0 -2 1 -1 -2 -2
>>> dfr1.rsub(dfr2)
0 1 2 0 2 0 2 1 1 2 2
>>> dfr1-2
0 1 2 0 -1 0 1 1 2 3 4
Corresponding element of dfr1 and dfr2 is added It will add Corresponding elements of dfr2 with dfr1 (dfr2+dfr1)
Here `r' stands for reverse it will add Corresponding elements
of dfr2 with dfr1 (dfr2+dfr1) It will add Corresponding elements of dfr1, dfr2 and dfr13
It will subtract Corresponding elements of dfr1 with dfr2
It will subtract Corresponding elements of dfr1 with dfr2
Here `r' stands for reverse it will subtract Corresponding
elements of dfr2 with dfr1 (dfr2 - dfr1)
Here 2 is subtracted with each element of Data Frame dfr1
pythonclassroomdiary. by Sangeeta M Chauhan, PGT CS , KV NO.3 Gwalior
In the Same way Multiplication can be done with * operator and mul() function and Division can be done with / operator and div() function
7. Matching and Broadcasting Operations:
7.1 Matching : Whenever we perform arithmetic operations on dataframe data is aligned on the basis of matching indexes and then performed arithmetic ; for non-overlapping indexes the arithmetic operations result as a NaN . This default behavior of data alignment on the basis of matching indexes is known as MATCHING
import pandas as pd s1=[[21,52,43],[41,55,66]] s2=[[34,4],[4,6]] dfr1=pd.DataFrame(s1) dfr2=pd.DataFrame(s2) print('Data Frame 1') print(dfr1) print('Data Frame 2') print(dfr2) print('Matching is done') print(dfr1+dfr2)
Data Frame 1 0 1 2
0 21 52 43 1 41 55 66
Data Frame 2 0 1
0 34 4 1 4 6
Matching is done 0 1 2
0 55 56 NaN 1 45 61 NaN
Output
7.2 Broadcasting : Enlarging the smaller object in a binary operation by replicating its elements so as to match the shape of larger object.
import pandas as pd s1=[[21,52,43],[41,55,66]] s2=[[34,4],[4,6]] dfr1=pd.DataFrame(s1) dfr2=pd.DataFrame(s2) print('Data Frame 1') print(dfr1) print('Data Frame 2') print(dfr2) print('Matching is done') print(dfr1+dfr2) print('Broadcasting is done') s3=[3,4,5] print(dfr1+s3)
Data Frame 1
01 2
0 21 52 43
1 41 55 66
Data Frame 2
0
1
0 34
4
1 4
6
Broadcasting is done
01 2
0 24 56 48
1 44 59 71
Output
1
pythonclassroomdiary. by Sangeeta M Chauhan, PGT CS , KV NO.3 Gwalior
8. Handling Missing Data :
As data comes in many shapes and forms, pandas aims to be flexible with regard to handling missing data. While NaN is the default missing value marker for reasons of computational speed and convenience, we need to be able to easily detect this value with data of different types: floating point, integer, boolean, and general object. In many cases, however, the Python None will arise and we wish to also consider that "missing" or "not available" or "NA".
Function Name isnull()
notnull()
dropna()
Use Returns True or False for each value in pandas object if it is a missing value or not Returns True or False for each value in pandas object if it is a data value or not It will remove(drop) all the rows which contain NaN values anywhere in row
Dropna(how='all') fillna()
It will remove nly those rows that have all NaN values It will fill missing Values with the value specified
>>>import pandas as pd >>>KV_shift1={'Computer':[20,25,22,50],'Projectors':[1,1,1,14],'iPad':[1,1,1,7],'AppleTv':[1,1,1,7]} >>>dfr1=pd.DataFrame(KV_shift1,index=['SrCompLab','SecCompLab','PriLab','Others']) >>>print(dfr1) >>>KV_shift2={'Computer':[20,25,22,50],'Visualizers':[1,1,1,14],'iPad':[1,1,1,7],'AppleTv':[1,1,1,7]} >>>dfr2=pd.DataFrame(KV_shift2,index=['SrCompLab','SecCompLab','PriLab','Others']) >>>print(dfr2) >>>KV3Gwl=dfr1+dfr2 >>>print(KV3Gwl)
Computer Projectors iPad AppleTv
SrCompLab
20
1 1
1
SecCompLab
25
1 1
1
PriLab
22
1 1
1
Others
50
14 7
7
DataFrame : dfr1
Computer Visualizers iPad AppleTv
SrCompLab
20
1 1
1
SecCompLab
25
1 1
1
PriLab
22
1 1
1
Others
50
14 7
7
DataFrame : dfr2
AppleTv Computer Projectors Visualizers iPad
SrCompLab
2
40
NaN
NaN 2
SecCompLab 2
50
NaN
NaN 2
PriLab
2
44
NaN
NaN 2
Others
14 100
NaN
NaN 14
8.1 Use of isnull() and notnull()
DataFrame : KV3Gwl
print('ISNULL ()' )
print(KV3Gwl.isnull())
print('NOTNULL ()' ) print(KV3Gwl.notnull())
Isnull() Will Give True If
notnull() Will Give True If Corresponding Element
Corresponding Element
Contains an data
pythonclassroomdiary.wordpreCsso.ncotamins bNyaSNangeeta M Chauhan, PGT CS , KV NO.3 Gwalior
Isnull() Will Give True If
Corresponding Element
Contains NaN
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related download
- 15 110 hw6 tweet analytics
- programming in python lecture xvii exception handling
- advanced tabular data processing with pandas
- python tutorial for cse 446 university of washington
- 5 traversing dataframe elements using
- lecture 11 computational efficiency
- pandas groupby iterate error getting images 1 pandas
- pandas under the hood
- analysis of unstructured data
- sas and python the perfect partners in crime