NEXT - UMD
[Pages:31]NEXT
Data collection
Data processing
Exploratory analysis & Data viz
Analysis, hypothesis testing, &
ML
Insight & Policy
Decision
1
NEXT:
NUMPY, SCIPY, AND DATAFRAMES
2
DATA MANIPULATION AND COMPUTATION
Data Science == manipulating and computing on data Large to very large, but somewhat "structured" data
We will see several tools for doing that this semester Thousands more out there that we won't cover
Need to learn to shift thinking from:
Imperative code to manipulate data structures
to:
Sequences/pipelines of operations on data
Should still know how to implement the operations themselves, especially for debugging performance
3
DATA MANIPULATION AND
COMPUTATION Indexing
1. Data Representation, i.e., what is Slicing/subsetting
the natural way to think about given data
One-dimensional Arrays, Vectors
Filter `map' ! apply a function to every element 'reduce/aggregate' ! combine values to get a single scalar (e.g., sum,
median)
0.1 2 3.2 6.5 3.4 4.1
Given two vectors: Dot and cross
products
"data"
"representation"
"i.e."
2. Data Processing Operations, which take one or more datasets as input and produce one or more datasets as output
4
DATA MANIPULATION AND
COMPUTATION
1. Data Representation, i.e., what is the natural way to think about
given data
n-dimensional arrays
Indexing Slicing/subsetting Filter `map' ! apply a function to every element 'reduce/aggregate' ! combine
values across a row or a column (e.g., sum, average, median etc..)
2. Data Processing Operations, which take one or more datasets as input and produce one or more datasets as output
5
DATA MANIPULATION AND COMPUTATION
1. Data Representation, i.e., what is the natural way to think about given data
Matrices, Tensors
n-dimensional array operations + Linear Algebra
Matrix/tensor multiplication Transpose Matrix-vector multiplication Matrix factorization
2. Data Processing Operations, which take one or more datasets as input and produce one or more datasets as output
6
DATA MANIPULATION AND
COMPUTATION
1. Data Representation, i.e., what is the natural way to think about given data
Sets: of Objects
Filter Map Union
Reduce/Aggregate
Sets: of (Key, Value Pairs)
(juexu@cs.umd.edu,(email1, email2,...)) (nayeem@cs.umd.edu,(email3, email4,...))
Given two sets, Combine/Join using "keys"
Group and then aggregate
2. Data Processing Operations, which take one or more datasets as input and produce one or more datasets as output
7
DATA MANIPULATION AND
COMPUTATION
1. Data Representation, i.e., what is the natural way to think about given data
Filter rows or columns
Tables/Relations == Sets of Tuples
"Join" two or more relations
"Group" and "aggregate" them
Relational Algebra formalizes some of them
Structured Query Language (SQL) Many other languages and constructs, that look very similar
2. Data Processing Operations, which take one or more datasets as input and produce one or more datasets as output
8
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related download
- numpy tutorialspoint
- using gpu power for numpy syntax calculations nvidia
- 3 case study with keras jordi
- cs 357 numerical methods lecture 2 basis and numpy
- numpy tutorial complete guide to learn python numpy
- introduction to netcdf4 binary le with python c and r
- lecture 6 florida state university
- numerical computing in python cornell university
- lab 2 numpy and scipy brigham young university
- numpy primer cornell university
Related searches
- sentai next 2019
- the next question is
- next super sentai series 2019
- stock market outlook for next 6 months
- next equinox date
- top careers for the next 10 years
- mortgage rates next 30 days
- next states to legalize weed
- kid n play next question
- next question see ya
- next states for recreational marijuana
- russell westbrook next question