Data Integration and Large-Scale Analysis WS2023/24
(VU, 706.520 Data Integration and Large-Scale Analysis)

DIA is a 5 ECTS bachelor and master course, applicable to the bachelor programs computer science or software engineering and management, as well as the master catalog 'Data Science'. This course covers major data integration architectures, key techniques for data integration and cleaning, as well as methods for large-scale, i.e., distributed, data storage and analysis.

Lectures

In detail, the course covers the following topics, which also reflects the course calendar. All slides will be made available prior to the individual lectures, which take place Friday's 3pm in HS-i5 or virtually.

A: Data Integration and Preparation

  • 01 Introduction and Overview [Oct 06, pdf, pptx]
  • 02 Data Warehousing, ETL, and SQL/OLAP [Oct 13, pdf, pptx]
  • 03 Message-oriented Middleware, EAI, and Replication [Oct 20, pdf, pptx]
  • 04 Schema Matching and Mapping [Oct 27, pdf, pptx]
  • 05 Entity Linking and Deduplication [Nov 3, pdf, pptx]
  • 06 Data Cleaning and Data Fusion [Nov 10, pdf, pptx]

B: Large-Scale Data Management and Analysis

  • 07 Cloud Computing Fundamentals [Nov 17, pdf, pptx]
  • 08 Cloud Resource Management and Scheduling [Dec 01, pdf, pptx]
  • 09 Distributed Data Storage [Dec 15, pdf, pptx]
  • 10 Distributed, Data-Parallel Computation [Jan 12, pdf, pptx]
  • 11 Distributed Stream Processing [Jan 19,pdf, pptx]
  • 12 Distributed Machine Learning Systems [Jan 19,pdf, pptx]

Exercises

The lectures are accompanied by mandatory programming exercises (to the extend of 2 ECTS, i.e, roughly 50 working hours), preferably in Python or Java language.

Exercise Description and Data (PredictX, goldY, yelp_err)

Organization

  • Lecturer: M.Sc. Shafaq Siddiqi, ISDS
  • Final written exams: 02 Feb 2024, 15:00 - 17:00 in HS i11
  • Grading: 30% project, 70% final exam