Heavy Traffic and Markov Modulated Models for Wireless Queuing Systems and Numerical Methods for Associated Resource Allocation Problems

Abstract

This dissertation is concerned with heavy traffic and Markov modulated diffusion models that are applied to resource allocation problems in wireless communication system and the numerical analysis for their associated continuous time stochastic control problems. To be specific, the heavy traffic model is a two-dimensional stochastic differential equation with reflection (SDER), and the other model is a second-order Markov modulated diffusion process.

With the proliferation of wireless applications having large capacity requirements, such as multimedia, internet, gaming, etc., and the limitations of realizing spectral efficiency gains, wireless queueing systems will be operating a near-capacity levels, so called "Heavy traffic". Under this assumption, SDER has been developed as an approximation model for a multi-buffer and various channel state wireless communication system. Building on the seminal work of Buche and Kushner [13], we study how the reflection process can affect the solution of the SDER and the resource (reserve power) allocation theoretically and numerically. We have shown that Multi-Completely S is a necessary condition for the existence and uniqueness for the SDER instead of the well known Completely S in the wireline system [69]. The whole resource (reserve transmission power) allocation is modeled as a stochastic control problem subject to the SDER. Using Markov Chain Approximation (MCA) method [51], various effects of factors, especially the reflection processes (nominal power reallocation) are studied via numerical experiments. After optimal control policies are obtained via MCA method under an appropriate grid size setting, Monte Carlo and real time simulation experiments are done using heavy traffic policies v.s. heuristic wedge control policies. The performance of heavy traffic policies is better than that of wedge policies under various traffic patterns including aggregated ON⁄OFF process (Long Range Dependence & Heavy Tailed) which is an active research area in mathematical and engineering communities.

Usually, the common objective of power control problem in mobile system is to minimize power consumption while maintaining the signal-to-interference-plus-noise ratio (SINR) above a predesigned threshold determined by the QoS requirement ( [88], [30]). Static or quasi static channel gain is assumed (i.e., for most of the users, the channel gains remain approximately constant over sufficiently long periods of time). But we propose a queue-based power control model where the channel gain process is modeled by a finite-state continuous-time Markov chain and embedded explicitly in the queue dynamics. Relatively few wireless power control literatures take a queue-based approach. In [16] Chisci and his coworkers introduce Queue-Based Distributed Power Control algorithm but the queue dynamics are described by a discrete-time system in quasi-static channel gain environment. Buche and Kushner [13] study the forward-link queueing system with time- varying channels using a heavy traffic method. A limit queueing model is obtained by weak convergence methods, where an "averaging" occurs through taking the limit, the channel process is modelled in the prelimit process, however the channel process does not appear explicitly in the limit model due to the averaging. Huang and his coworkers [36] propose a reverse-link stochastic control framework, in their case, the channel gain (power attenuation process) is described by a stochastic differential equation, the control objective is to try to minimize the total transmission power as well as maintain acceptable levels for the SINR. We consider queue dynamics described by a continuous time Markov-modulated diffusion process due to the time-varying channel gains. Furthermore, we model the affect of the SINR on the queue dynamics through a bit error rate (BER) function. Similarly, we discusse the associated power allocation problem in a stochastic control framework, the corresponding HJB function is derived and numerical method is discussed.