MapReduce - Duke University

[Pages:36]Map--Reduce

Everything Data CompSci 290.01 Spring 2014

2

Announcements (Thu. Feb 27)

? Homework #8 will be posted by noon tomorrow.

? Project deadlines:

? 2/25: Project team formation ? 3/4: Project Proposal is due.

? 3/4: 2 minute presentation in class

3

2,161,530,000,000 searches in 2013

4

Size of the entire corpus??

131,000,000 pages mentioning Einstein

5

Size of the entire corpus??



6

Trend 1: Data centers

353991/Where-The-InternetLives-A-Glimpse-Inside-Google-sPrivate-Data-Centers/

7

The TFuturerofend Computing 2: Performance: MuGameltOveriorcNext ore Level?

Moore'sP O LWaERwIS:N O#W tLrIaMnITsINiGstGoRrOsW ToHnI N inCOteMgPUrTaItNeGdP E cRiFrOcRuMiAtNsC dE oubles e9v1ery 2 years

10,000

cy (MHz) over time (1985-2010).

1985

1990

1995

2000

2005

Year of Introduction

URE 3.3 Microprocessor-clock frequency (MHz) over time (1985-2010).

1,000

100

10

1985

1990

ected performance. One might think that it should therefore be e to continue to scale performance by doubling the number of pro cores. And, in fact, since the middle 1990s, some researchers h ued that chip multiprocessors (CMPs) can exploit capabilities of C nology more effectively than single-processor chips.21 However, the 1990s, the performance of single processors continued to scal rate of more than 50 percent per year, and power dissipation was a limiting factor, so those efforts did not receive wide attention. le-processor performance scaling slowed down and the air-coo er-dissipation limit became a major design constraint, researchers ustry shifted toward CMPs or multicore microprocessors.22

1995

Year of Introduction

2000

2005

unle Olukotun, Basem A. Nayfeh, Lance Hammond, Ken Wilson, and Kunyung Ch , The case for a single-chip multiprocessor, Proceedings of 7th International Confer rchitectural Support for Programming Languages and Operating Systems, Cambri s., October 1-5, 1996, pp. 2-11. isted here are some of the references that document, describe, and analyze this hael Bedford Taylor, Walter Lee, Jason Miller, David Wentzlaff, Ian Bratt, Ben Green ry Hoffmann, Paul Johnson, Jason Kim, James Psota, Arvind Saraf, Nathan Shnid er Strumpen, Matt Frank, Saman Amarasinghe, and Anant Agarwal, 2004, Evalu e raw microprocessor: An exposed-wire-delay architecture for ILP and streams, ings of the 31st Annual International Symposium on Computer Architecture, Mu

2010

Copyright ? National Academy of Sciences. All rights reserved.

8

Need to think "parallel"

? Data resides on different machines ? Split computation onto different

machines/cores

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download