Systems Infrastructure for Data Science

Main page |Lecture Notes | Exercise Sheets 

Lecturer

Prof. Dr. Peter Fischer

Organization

Lecture:

• Tuesday, 16:15 - 18:00, SR 00-006, Building 082
• Thursday, 14:15-15, SR 00-036, Building 082

Exercises: Thursday, 15 - 16, SR 00-006, Building 082

Content

Recently, the term "big data" has become an important buzzword: Massive amounts of complex data are being produced by businesses, scientific applications, government agencies and social applications. This data can be utilized to gain new insights for decision support, scientific insights, advertising or just entertainment.

In addition to the increasing amount of available data, the architectures and methods to store and analyze this data have changed drastically in the last decade.

The course covers the fundamentals of different data infrastructure
systems, among them classical databases, main-memory databases, data
stream systems and cloud computing frameworks.

In the first part of the course (until around the Christmas break), the fundamentals of database management systems are covered

1. Architecture of classical Database Management Systems
   
   1. Data Storage - Storage Hierarchies, Storage Management
   2. Indexing: ISAM, B-Trees, Hash-Based Indexing
   3. Spatial Indexes: Quad Trees, k-d-Trees, R-Trees
   4. Query Processing: Operators, Execution Model
   5. Query Optimizations: Query Translation Stages, Cost Models, Plan Enumeration
   6. Performance Measurement and Tuning
2. Distributed Databases
   1. General Concepts and Fundamental Architectures
   2. Data Placement and Fragmentation
   3. Distributed Query Processing
3. Parallel Databases

On this basis, the second stage of the course will provide further insights how design assumptions change when such systems are used in contexts which require extreme scalability, very short response times or complex analytical operations. Relevant Topics include:

1. Hadoop and the Map-Reduce Framework
2. Web-Scale "Databases": Key/Value-Stores
3. Main-Memory Databases
4. Data Stream Systems

Literature

The classical data management areas are covered in the following books:

• - Database Management Systems”, Raghu Ramakrishnan and Johannes Gehrke, 3rd edition, 2002
• - Kemper und Eickler. Datenbanksysteme. Eine Einführung. Oldenbourg-Verlag. (in German)

The key reference for distributed databases is

• Öszu/Valduirez: Principles of Distributed Database Systems, 3rd edition

Modern techniques are mostly available in research papers, which are
provided during the lecture.