Lightweight modeling of complex state dependencies in stream processing systems

Abstract

Over the last few years, Real-Time Calculus has been used extensively to model and analyze embedded systems processing continuous data/event streams. Towards this, bounds on the arrival process of streams and bounds on the processing capacity of resources serve as inputs to the model, which are used to calculate end-to-end delays suffered by streams, maximum backlog, utilization of resources, etc. This "functional" model, although amenable to computationally inexpensive analysis methods, has limited modeling capability. In particular, "state-based" processing, e.g. blocking write - where the processing depends on the "state" or fill-level of the buffer - cannot be modeled in a straightforward manner. This has led to a number of recent proposals on using automata-theoretic models for stream processing systems (e.g. Event Count Automata [RTSS 2005]). Although such models offer better modeling flexibility, they suffer from the usual state-space explosion problem. In this paper we show that a number of complex state-dependencies can be modeled in a lightweight manner, using a feedback control technique. This avoids explicit state modeling, and hence the state-space explosion problem. Our proposed modeling and analysis therefore extend the original Real-Time Calculus-based functional modeling in a very useful way, and cover much larger problem domain compared to what was previously possible without explicit state-modeling. We illustrate its utility through two case studies and also compare our analysis results with those obtained from detailed system simulations (which are significantly more time consuming). © 2009 IEEE.