Schedule

Week 1

Wednesday | 2018.1.17

• First class! Course overview, syllabus discussion, and start of course material.
• Slides: Lecture 1

MP

• No MP Assigned

Week 2

Monday | 2018.1.22

• MapReduce Part 1
• Slides: Lecture 2
• Relevant Readings/Resources:
  • MapReduce: The programming model and practice
  • MapReduce Abstraction (UW Coursera)
  • Google MapReduce White Paper (Sections 1-2)

Wednesday | 2018.1.24

• MapReduce Part 2
• Slides: Lecture 3
• Relevant Readings/Resources:
  • Hadoop Streaming Documentation
  • Hadoop MapReduce Documentation
  • MRJob Documentation
  • Google MapReduce White Paper (Sections 3-5)

MP

• MP1 - Introduction to MapReduce

Week 3

Monday | 2018.1.29

• Hadoop and HDFS
• Slides: Lecture 4
• Relevant Readings/Resources:
  • HDFS Architecture Guide
  • Apache Hadoop YARN
  • Untangling Apache Hadoop YARN (Cloudera)

Wednesday | 2018.1.31

• [Optional]
• Troubleshooting Cluster Access & Hadoop Usage
• Extended Office hours

MP

• MP2 - MapReduce on Hadoop

Week 4

Monday | 2018.2.5

• Spark
• Slides: Lecture 5
• Relevant Readings/Resources:
  • Spark RDD White Paper (UC Berkeley)
  • Spark Programming Guide
  • PySpark Package Documentation

Wednesday | 2018.2.7

• [Optional]
• Spark Programming and Usage Demos
• Office hours

MP

• MP3 - Spark Core

Week 5

Monday | 2018.2.12

• Spark Streaming
• Final Project Details
• Slides: Lecture 6
• Relevant Readings/Resources:
  • Spark Streaming Programming Guide
  • “Diving Into Apache Spark Streaming Execution Model” (DataBricks)
  • PySpark Streaming API Documentation
  • DStreams White Paper

Wednesday | 2018.2.14

• [Optional]
• Spark Streaming Programming and Usage Demos
• Office hours

MP

• MP4 - Spark Streaming

Week 6

Monday | 2018.2.19

• Spark SQL & DataFrames
• Slides: Lecture 7
• Relevant Readings/Resources:
  • Spark SQL Programming Guide
  • Spark SQL Python Documentation

Wednesday | 2018.2.21

• GCP Guest Lecture

MP

• MP5 - Spark SQL and DataFrames

Week 7

Monday | 2018.2.26

• Spark MLlib
• Slides: Lecture 8
• Relevant Readings/Resources:
  • Spark MLlib RDD Programming Guide
  • Spark MLlib RDD API
  • Spark MLlib DataFrame Programming Guide
  • Spark MLlib DataFrame API

Wednesday | 2018.2.28

• [Optional]
• Spark MLlib Office Hours
  • Example RDD MLlib Application
  • Example DataFrame MLlib Application
• Project Proposal Office Hours

MP

• MP6 - Spark MLlib (Due March 13th)

Week 8

Monday | 2018.3.5

• NoSQL and Key-Value Stores
• Slides: Lecture 9
• Relevant Readings/Resources:
  • CAP Theorem
  • ACID Properties
  • Apache Cassandra

Wednesday | 2018.3.7

• [Optional]
• Spark MLlib Office Hours
• Final Project Office Hours

MP

• MP6 - Spark MLlib (Due March 13th)

Final Project

• Group Selection due on March 10th at 11:59pm

Week 9

Monday | 2018.3.12

• Data Sourcing, Licensing, and Cleaning
• Slides: Lecture 10
• Relevant Readings/Resources:
  • Open Source License Comparison
  • BeautifulSoup
  • AWS Snowball

Wednesday | 2018.3.14

• [Optional]
• Project Proposal Office Hours

Final Project

• Project Proposal due on March 16th at 11:59pm (Submission on Moodle)

Week 10

Spring Break

Week 11

Monday | 2018.3.26

• Commercial Clouds and Cloud Security
• Slides: Lecture 11
• Relevant Readings/Resources:
  • AWS Cloud Offerings
  • Azure Cloud Offerings
  • GCP Cloud Offerings

Wednesday | 2018.3.28

• [Optional]
• Final Project Office Hours

Week 12

Monday | 2018.4.2

• Docker & Containerization
• Slides: Lecture 12
• Relevant Readings/Resources:
  • Docker Overview
  • Dockerfile Reference Documentation
  • Docker Compose
  • Docker Swarm

Wednesday | 2018.4.4

• [Optional]
• Docker Tutorial and Demo
• Office Hours

MP

• MP7 - Docker

Week 13

Monday | 2018.4.9

• Terraform & Infrastructure as Code
• Slides: Lecture 13
• Relevant Readings/Resources:
  • “What is Infrastructure as Code?”
  • “Why Infrastructure as Code?”
  • Terraform Docs: Writing and Using Modules
  • Terraform Docs: Google Cloud Provider

Wednesday | 2018.4.11

• [Optional]
• Terraform Demo; Troubleshooting GCP Access
• Office Hours

MP

• MP8 - Terraform

Week 14

Monday | 2018.4.16

• Kubernetes
• Slides: Lecture 14
• Relevant Readings/Resources:
  • Kubernetes Documentation / Tutorial
  • “Kubernetes in 5 Minutes”

Wednesday | 2018.4.18

• [Optional]
• Kubernetes Office Hours
• Final Project Office Hours

MP

• MP9 - Kubernetes

Week 15

Monday | 2018.4.23

• Final Project Group Presentations
  • See Piazza for Group Schedule

Wednesday | 2018.4.25

• Final Project Group Presentations
  • See Piazza for Group Schedule

Week 16

Monday | 2018.4.30

• Final Project Group Presentations
  • See Piazza for Group Schedule

Wednesday | 2018.5.2

• Closing Lecture; Final Remarks

Final Project

• Project Report due on May 2nd at 11:59pm