Reliability and Optimization of Data Stream Systems
Overview
With the proliferation of dynamically generated data, we have witnessed the emergence of data stream processing as a new data management paradigm. Many Stream Processing Engine (SPE) prototypes have been built and several business applications have started to use data stream management systems.
However, as industry gears toward using data stream processing in complex applications, there are challenges which still need to be addressed.
A major aspect of these challenges concerns two aspects
- Reliability and Assurance of Stream Processing Applications: Since stream processing is used for complex applications with high reliability requirements, assurances need to be given how data is being produced (lineage) and how queries are instantiated and adapted (query lifecycle)
- The complexity of stream applications requires new methods to model and optimize streaming query workloads.
Project Members and Collaborators
- Prof. Dr. Peter Fischer
- Prof. Dr. Boris Glavic (Illinois Institute of Technology)
- Prof. Dr. Nesime Tatbul (ETH Zurich)
- Dr. Kyumars Sheykh Esmaili (Nanyang Technological University, Singapore)
Funding
Publications
- Tahmineh Sanamrad, Kyumars Sheykh Esmaili, Peter M. Fischer, Nesime Tatbul: Changing Flights in Mid-air: A Model for Safely Modifying Continuous Queries, ACM SIGMOD 2011, 613-624
- Boris Glavic, Kyumars Sheykh Esmaili, Peter M. Fischer, Nesime Tatbul: The Case for Fine-grained Stream Provenance, Workshop “Data Streams and Event Processing” at BTW 2011
- Peter M. Fischer, Kyumars Sheykh Esmaili, Renée J. Miller: Stream Schema: Providing and Exploiting Static Metadata for Data Stream Processing. 13th International Conference on Extending Database Technology, Lausanne, Switzerland, March 22-26, 2010, Proceedings: 207-218