Big data tutorial w2 spark
EECS E6893 Big Data Analytics Spark 101
Yvonne Lee, yl4573@columbia.edu
9/17/21
1
Agenda
Functional programming in Python
Lambda
Crash course in Spark (PySpark)
RDD Useful RDD operations
Actions Transformations
Example: Word count
2
Functional programming in Python
3
Lambda expression
Creating small, one-time, anonymous function objects in Python
Syntax: lambda arguments: expression
Any number of arguments Single expression
Could be used together with map, filter, reduce
Example:
Add: add = lambda x, y : x + y
def add (x, y): return x + y
type(add) =
add(2,3)
4
Crash course in Spark
5
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related download
- built in functions
- a tutorial introduction to the lambda calculus
- working with functions using python
- python the full monty
- lecture 09 filter map reduce and lambda
- 1 functions in python
- ppyytthhoonn ffuunnccttiioonnss tutorialspoint
- booleans if loops lambda functions
- tutorial 6 introduction to lambda iii serverless databases
- big data tutorial w2 spark