Revised: December 28, 2011



Revised: January 2015

Stevens Institute of Technology

Howe School of Technology Management

Syllabus

BIA 678

Big Data Seminar

|Spring 2015 |Tuesday 6:15 pm |

|David Belanger |Office Hours: |

|Babbio 409 |Monday 2:00 and 5:00 pm |

|Tel: 201-216-3392 |Also by appointment |

|Fax: 201-216-5385 | |

|dbelange@stevens.edu | |

| |Course Room/Web Address: |

| |Babbio304 / |

Overview

|The field of Big Data is emerging as one of the transformative business processes of recent times. It utilizes classic techniques |

|from Business Intelligence & Analysis, along with a new tools and processes to deal with the volume, velocity, and variety |

|associate with big data. As they enter the workforce, a significant percentage of BIA students will be directly involved with big |

|data either as technologists, managers, or users. This course will build on their understanding of the basic concepts of BI&A to |

|provide them with the background to succeed in the evolving data centric world, not only from the point of view of the technologies|

|required, but in terms of management, governance, and organization. Tools will include Hadoop, Hbase, and related software. |

|Prerequisites: Admission requirements for the BI&A program.Course ObjectivesThe objective of this course is to study key |

|technological, management, and governance techniques for application of big data. This will be done through a series of readings |

|and lectures, some by outside experts; case studies of the application of big data; application of technologies typical of the |

|field (e.g. Map/Reduce); and a semester long, small team project applying what has been learned. They will learn how to apply |

|selected tools in areas such as data management, data analysis, and data visualization, and also learn how to deal with the issues |

|related to the management of large sets of data. The course will concentrate on what is different in a big data environment, from |

|what they have already learned about standard BIA environments.. Finally, through the analysis and discussion of case studies they |

|get useful insights on how to optimize the value of big data processes and operations, to streamline the goals and to design |

|flexible systems. Students taking the course will be expected to have some background in areas such as multivariate statistics, |

|data mining, data management, and programming. |

|Additional learning objectives include the development of: |

|Written and oral communications skills: the individual project proposal will be used to assess written skills and the final |

|presentations will be used to assess presentation skills. |

|Technical Reading Capability: Students will be required to read, and lead discussions on, seminal papers in the field of big data.|

|Team skills: The final project for the course will involve student teams; an online survey instrument will be used to measure |

|individual contributions to team performance. |

List of Course Outcomes:

After taking this course, students will be able to:

1. Understand and discuss what big data is, and how it differs from traditional approaches to BI&A

2. Plan and use the primary tools associated with big data in creating systems to take advantage of big data.

3. Extract knowledge and intelligence from datasets which exhibit high volume, velocity, and/or variety.

4. Plan and execute a project that includes the use of at least one big data dataset.

5. Understand and discuss the meta issues around big data such as governance, security, privacy, and OAM&P.

6. Understand and be able to execute analyses oriented to streaming data.

7. Have a framework with which to understand new advances in the field, and distinguish hype from reality.

8. Understand and discuss organizational issues related to big data.

Pedagogy

|The course will employ lectures, class discussion, in-class individual assignments, an individual term paper and a team project. In|

|the team project, students will analyze an industrial problem using real data, design a solution approach using big data techniques|

|along with other statistical and machine learning techniques, program and execute the solution, and interpret the solution for |

|management. In the term paper, students will be required to describe and address issues of importance in modern big data systems. |

Readings

|Required Text |

| |

|Soares, Sunil, “Big Data Governance – An Emerging Imperative.” Boise ID, MC Press, 2012. |

| |

|Supplementary Reading: |

| |

| |

|Wu, et. al., “Data Mining with Big Data”, IEEE Transactions on Knowledge and Data Engineering, 1/2014 |

| |

| |

|Lin & Ryaboy, “Scaling Big Data Mining Infrastructure: The Twitter Experience”, SIGKDD Explorations, V14 I2 |

| |

| |

|McKinsey Global Institute, “Big Data: The next frontier for innovation, competition, and productivity”, 2011 |

|

|sights%20%26%20Publications |

| |

|Dean & Ghemawat, “MapReduce:Simplified Data Processing on Large Clusters”, |

|, 2004 |

| |

|Ghemawat, et al, “Google File System”,

|, 2003 |

| |

|Compression vcodex, , |

| |

|Cortes, et al., Communities of Interest, |

| |

| |

|CAP, IEEE Computer V45 N2 2/2012 pp. 21-58., esp: 21, 23, 30, 37, 43. |

|Lynch & Gilbert, “Perspectives on the CAP Theorem”, , 2012 |

| |

|Abadi, et al, “Column-Stores vs. Row-Stores: How Different are they Really, |

|, |

| |

|Chang et al, “Bigtable: A Distributed Storage System for Structured Data”, |

| , 2006 |

| |

|Decandia, et al, “Amazon’s Highly Available Key Value Store”, |

| , |

| |

|Hbase Basics (Cassandra Basics) O’reilly ; |

| |

|Widom, et. al, “STREAM: The Stanford Data Stream Management System”, , 2004 |

| |

|, 2004, Cranor, et al. “Gigascope: A Stream |

|Database for Network Applications” |

| |

|Johnson, “Stream Warehouseing” , , An application to Darkstar |

| |

|IBM Infosphere Streams, |

| |

|Wu, et al, “Top 10 Algorithms in Data Mining”, Knowledge Systems 2007, |

|, |

| |

|Marai,Liz |

| |

|Shneiderman, Ben, Extreme Visualization: Squeezing a Billion Records into a Million Pixels |

| |

| |

|Scheidegger, et al, “Visual Embedding, a Model for Visualization”, , 1/2014 |

| |

|

|e.PDF |

| |

|NIST Big Data Public Working Group and Standardization activities, , |

| |

|Privacy Policies, for example: jpmorgan, at&t, google, smaller folks, … |

| |

|Johnson, et al, “Bistro Data Feed Management System”, , ,, |

| |

|MIT, “an evaluation framework for data quality tools”, |

|, |

| |

| |

Assignments

Class Discussion Leadership (10%)

Each student will be required to lead the class discussion of one or more of the assigned readings. This will be done in front of the class. All students are expected to have read the assigned readings, and to take part in the discussison.

Biweekly homework assignments including programming using map/reduce, compression, etc. along with weekly reading assignments (33%)

1 INDIVIDUAL Term Paper (33%).

Each student will be required to write a term paper of approximately 5 – 10 pages on a topic of their choice within the domain of big data.

TEAM PROJECT REPORT & PRESENTATION (33%)

The class will be divided into teams of approximately 5 students each. Each team will be expected to select a data set appropriate to big data, to conduct a variety of analyses on the data using big data associated tools, to present the project results to the class, and to create a written report on the project.

Ethical Conduct

|The following statement is printed in the Stevens Graduate Catalog and applies to all students taking Stevens courses, on and off |

|campus. |

| |

|“Cheating during in-class tests or take-home examinations or homework is, of course, illegal and immoral. A Graduate Academic |

|Evaluation Board exists to investigate academic improprieties, conduct hearings, and determine any necessary actions. The term |

|‘academic impropriety’ is meant to include, but is not limited to, cheating on homework, during in-class or take home examinations |

|and plagiarism.“ |

| |

|Consequences of academic impropriety are severe, ranging from receiving an “F” in a course, to a warning from the Dean of the |

|Graduate School, which becomes a part of the permanent student record, to expulsion. |

| |

|Reference: The Graduate Student Handbook, Academic Year 2003-2004 Stevens |

|Institute of Technology, page 10. |

|Consistent with the above statements, all homework exercises, tests and exams that are designated as individual assignments MUST |

|contain the following signed statement before they can be accepted for grading. |

|____________________________________________________________________ |

|I pledge on my honor that I have not given or received any unauthorized assistance on this assignment/examination. I further pledge|

|that I have not copied any material from a book, article, the Internet or any other source except where I have expressly cited the |

|source. |

|Signature _________________________ Date: _____________ |

|Please note that assignments in this class may be submitted to , a web-based anti-plagiarism system, for an |

|evaluation of their originality. |

| |

Course/Teacher Evaluation

Continuous improvement can only occur with feedback based on comprehensive and appropriate surveys. Your feedback is an important contributor to decisions to modify course content/pedagogy which is why we strive for 100% class participation in the survey. 

All course teacher evaluations are conducted on-line.  You will receive an e-mail one week prior to the end of the course informing you that the survey site () is open along with instructions for accessing the site.  Login using your Campus (email) username and password. This is the same username and password you use for access to Moodle. Simply click on the course that you wish to evaluate and enter the information. All responses are strictly anonymous.  We especially encourage you to clarify your position on any of the questions and give explicit feedbacks on your overall evaluations in the section at the end of the formal survey which allows for written comments.  We ask that you submit your survey prior to end of the examination period.  

COURSE SCHEDULE

The course is divided into modules, some of which will extend across more than a single class meeting.

|1. Introduction to Big Data |

|Overview: An introduction to Big Data, Definitions, Applications, Tools, and Governance. |

|Readings:Wu, et. al., 2014 |

|Lin & Ryaboym, 2012 |

|McKinsey Global Institute, 2011 |

|2. Core Technologies for Distribution and Scale |

|An introduction to the core technologies for scale and distribution, including map/reduce, Hadoop, compression, GFS and HDFS |

|Readings: Dean & Ghemawat, 2004 |

|Ghemawat, et. al., 2003 |

|Vcodex |

|Cortes, et. al. |

|Cloudera Tutorial |

|3. Data Base Management |

|CAP, NoSQL, Column Store, Hbase, Xquery, |

|Readings: Lynch & Gilbert, 2012 |

|Abadi, et al, 2008 |

|Chang, et al, 2006 |

|Decandia, et al, 2008 |

|4. Data Stream Management |

| Internet of Things, Data Stream Management Systems, Infosphere Stream, STREAM, Gigascope, Analytics |

|Readings: Widom, et al, 2004 |

|Cranor, et al, 2004 |

|Johnson, 2012 |

|Infosphere Speaker |

|5. Data Analytics |

|Data analytics in a big data, distributed world. R over Hadoop |

|Readings: Wu et al, 2007 |

|6. Visualization in a big data world |

|Issues and techniques in visualizing large, or fast moving, datasets. |

|Readings: Marai, 2004 |

|Sheiderman, 2008 |

|Scheidigger, et al, 2012 |

|7. Data Governance |

|Issues related to the governance of large data sets, including: security, privacy, integrity, quality, and OA&M |

|Readings: Soares Parts 1, 2, and 3 |

|8. Meta Issues in Big Data Governance |

|More detailed discussion of the issues of security, privacy, integrity, quality, OA&M, and management of big data, including |

|related technologies. (Visiting Speaker.) |

|Readings: NIST Documents |

|Privacy Policies of selected companies (e.g. JPMorgan, AT&T) |

|MIT |

|9. Applications |

|Detailed discussion of selected applications of big data in a few different industries. (Visiting Speaker) |

|10. Student Presentations of Term Projects |

|Each team presents their term project: written report plus oral presentation |

| |

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download