Distributed GPU Computing with Dask
Distributed GPU Computing with Dask
Nick Becker RAPIDS Engineering
RAPIDS
Scaling RAPIDS with Dask
Data Preparation
cuDF cuIO Analytics
Dask
cuML Machine Learning
Model Training
cuGraph Graph Analytics
GPU Memory
Visualization
PyTorch Chainer MxNet
Deep Learning
cuXfilter pyViz Visualization
2
Dask
What is Dask?
? Distributed compute scheduler built to scale Python
? Scales workloads from laptops to supercomputer clusters
? Extremely modular: disjoint scheduling, compute, data transfer and out-of-core handling
? Multiple workers per node allow easier one-worker-per-GPU model
3
Why Dask?
PyData Native
? Easy Migration: Built on top of NumPy, Pandas
Scikit-Learn, etc.
? Easy Training: With the same APIs ? Trusted: With the same developer community
Deployable
? HPC: SLURM, PBS, LSF, SGE ? Cloud: Kubernetes ? Hadoop/Spark: Yarn
Easy Scalability
? Easy to install and use on a laptop ? Scales out to thousand-node clusters
Popular
? Most common parallelism framework today in
the PyData and SciPy community
4
Combine Dask with cuDF
Many CPU DataFrames form a distributed CPU DataFrame
5
Combine Dask with cuDF
Many GPU DataFrames form a distributed GPU DataFrame
cuDF
6
Combine Dask with CuPy
Many CPU arrays form a Distributed CPU array
7
Combine Dask with CuPy
Many GPU arrays form a Distributed GPU array CuPy
8
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related download
- dask parallel computation with blocked algorithms and
- lecture 4 dask github pages
- distributed gpu computing with dask
- scalable machine learning with dask
- 126 proc of the 14th python in science conf scipy
- dask processing and analytics for large datasets
- lecture notes to big data management and analytics winter
- harnessing the power of anaconda for scalable data science
- comparative evaluation of big data systems on scientific
Related searches
- financial management distributed learning center
- examples of normally distributed variables
- not normally distributed data examples
- fm distributed learning center
- java gpu library
- desktop windows manager high gpu windows 10
- desktop windows manager gpu usage
- distributed logistic regression
- jointly distributed random variables examples
- dask dataframe to pandas dataframe
- dask dataframe from dictionary
- dask dataframe merge