| Date | Study | Workout |
|---|---|---|
| 07/01 | Job Hunting | / |
| 07/02 | Job Hunting | / |
| 07/03 | Job Hunting | / |
| 07/04 | Learn pyspark streaming | / |
| 07/05 | Learn pyspark streaming | / |
| 07/06 | (Travel in Portland~~) | / |
| 07/07 | (Travel in Portland~~) | Hiking |
| 07/08 | Finish udemy's course Apache Spark Streaming with Python and PySpark | / |
| 07/09 | Write pyspark project code and blog; | / |
| 07/10 | Write twitter pyspark streaming project | / |
| 07/11 | Coffee chat and write email all day | / |
| 07/12 | Work on new spotify project | / |
| 07/13 | (///nothing~~) | / |
| 07/14 | Coding for project; learn AWS DynamoDB, S3 | Ping Pong |
| 07/15 | Work on project; learn Apache Airflow, review Linear Regression | / |
| 07/16 | Write project report; learn Apache Airflow | / |
| 07/17 | Write project report | / |
| 07/18 | Learn Beautifulsoup, crawling and start new project | / |
| 07/19 | Write crawling code | Ping Pong |
| 07/20 | Crawling all day, deploy to EC2 | / |
| 07/21 | Write feature extraction code; rewrite resume | / |
| 07/22 | Revise resume | / |
| 07/23 | Revise resume and review probability theory | / |
| 07/24 | Sending applications all day | / |
| 07/25 | Learn Kafka and Spark, start a new project (a full stack data engineering and analysis project with realtime data) | Ping Pong |
| 07/26 | Meeting, Research on new project | / |
| 07/27 | Learn Kafka, play with multiple realtime data API | / |
| 07/28 | Learn Cassandra and CQL | / |
| 07/29 | Study bokeh, write streaming visualization module | / |
| 07/30 | Crawl yahoo finance for realtime data | / |
| 07/31 | Study bokeh, write visualization module for project | Ping Pong |
| Categories | Content | Progress |
|---|---|---|
| Project | Realtime Twitter Data Analysis using Spark Streaming | ■■■■■■■■■■ |
| Project | Data Analysis of K-POP: Playing with Spotify API | ■■■■■■■■■■ |
| Project | Realtime financial market data visualization and analysis | ■■■■■■□□□□ |
| Project | Crawling project | ■■■■■■■■■■ |
| Big Data | Udemy: Apache Spark Streaming with Python and PySpark | ■■■■■■■■■■ |
- Youtube: Advanced Apache Spark Training - Sameer Farooqui (Databricks)
- Blog & Github: Data Engineering Workshop 2018 from Netflix & A Typical Data Engineering Project — Sharing From Netflix Data Engineering Team