Systems Infrastructure for Data Science
- Monday, 14:15 - 16, SR 00-007, Building 106
- Friday, 14:15-15, SR 00-007, Building 106
Exercises: Friday, 15 - 16, SR 00-007, Building 106
Oral exam dates:
- February 26th./27th
- March 25th
Recently, the term "big data" has become an important buzzword: Massive amounts of complex data are being produced by businesses, scientific applications, government agencies and social applications. This data can be utilized to gain new insights for decision support, scientific insights, advertising or just entertainment.
In addition to the increasing amount of available data, the architectures and methods to store and analyze this data have changed drastically in the last decade.
The course covers the fundamentals of different data infrastructure
systems, among them classical databases, main-memory databases, data
stream systems and cloud computing frameworks.
In the first part of the course (until around the Christmas break), the fundamentals of database management systems are covered
- Architecture of classical Database Management Systems
- Data Storage - Storage Hierarchies, Storage Management
- Indexing: ISAM, B-Trees, Hash-Based Indexing
- Spatial Indexes: Quad Trees, k-d-Trees, R-Trees
- Query Processing: Operators, Execution Model
- Query Optimizations: Query Translation Stages, Cost Models, Plan Enumeration
- Performance Measurement and Tuning
- Distributed Databases
- General Concepts and Fundamental Architectures
- Data Placement and Fragmentation
- Distributed Query Processing
- Parallel Databases
On this basis, the second stage of the course will provide further insights how design assumptions change when such systems are used in contexts which require extreme scalability, very short response times or complex analytical operations. Relevant Topics include:
- Hadoop and the Map-Reduce Framework
- Web-Scale "Databases": Key/Value-Stores
- Main-Memory Databases
- Data Stream Systems
The following books should be helpful:
- Database Management Systems”, Raghu Ramakrishnan and Johannes Gehrke,3rd edition, 2002.
- Kemper und Eickler. Datenbanksysteme. Eine Einführung. Oldenbourg-Verlag. (in German)
Modern techniques are mostly available in research papers, which are
provided during the lecture.