ALGORITHMS FOR CONVEX SWITCHING SYSTEMS

Abstract

In this thesis we introduce the convex switching system, a specific structured model in Markov decision theory. The main feature is that it contains convex reward functions and the underlying states follow linear dynamics. Many problems in engineering, finance and statistics can be modelled under the scheme of convex switching systems. Mathematically, an exact solution for such problems can be difficult sometimes, especially when the dimension of states is high. Here in this thesis we introduce an approximate algorithm for the convex switching problems. It is also shown that the algorithms can be applicable for problems with non-convex rewards and POMDPs/HMMs with nonlinear dynamics. Some numerical illustrations and simulations in real-life problems are shown.