A Stationary Stochastic Approximation Algorithm for Estimation in the GLMM

Abstract

Estimation in generalized linear mixed models is challenging because the marginal likelihood is an integral without closed form. In many of the leading approaches such as Laplace approximation and Monte Carlo integration, the marginal likelihood is approximated, and the maximum likelihood estimate (MLE) can only be reached with error. An alternative, the simultaneous perturbation stochastic approximation (SPSA) algorithm is designed to maximize an integral and can be employed to find the exact MLE under the same circumstances. However, the SPSA does not directly provide an error estimate if the algorithm is stopped in a number of finite steps. In order to estimate the MLE properly with an statistical error bound, we propose the stationary SPSA (SSPSA) algorithm. Assuming that the marginal likelihood, objective function, is quadratic around the MLE, the SSPSA takes the form of a random coefficient vector autoregressive process. Under mild conditions, the algorithm yields a strictly stationary sequence where the mean of this sequence is asymptotically unbiased to the MLE and has a closed-form variance. Also, the SSPSA sequence is ergodic providing certain constraints on the step size, a parameter of the algorithm, and the mechanism that directs the algorithm to search the parameter space. Sufficient conditions for the stationarity and ergodicity are provided as a guideline for choosing the step size. Several implementation issues are addressed in the thesis: pairing numerical derivative, scaling, and importance sampling. Following the simulation study, we apply the SSPSA on several GLMMs: Epilepsy seizure data, lung cancer data, and salamander mating data. For the first two cases, SSPSA estimates are similar to published results whereas, for the salamander data, our solution greatly differs from others.