This dissertation presents a generative model-based gait representation framework for video-based human motion estimation. In this work, we advocate an important new concept, gait manifold, which is used to capture the gait variability among different individuals and enables gait interpolation for an unknown subject from a limited training gaits. Specifically, we propose two manifold structures, a closed-loop and a torus, along which the continuous gait variable is defined. Both gait manifolds are used to develop visual gait generative models for human motion estimation.;The learning of the closed-loop gait manifold involves a non-linear tensor decomposition by which we can learn a kinematic gait generative model (KGGM) and a visual gait generative model (VGGM). KGGM and VGGM represent gait kinematics and appearances by a few latent variables, respectively. Then a new manifold learning technique is proposed to learn a closed-loop gait manifold by which KGGM and VGGM can be integrated together to estimate gait kinematics from gait appearances. Additionally, we extend this framework to the part-level gait modeling that involves two gait manifolds, one for the lower body and one for the upper-body. Experimental results show our algorithms are competitive with state-of-art algorithms considering that a single camera is used, and the part-level gait modeling further improves results.;The toroid joint gait-pose manifold (JGPM) is proposed to jointly represent the pose and gait factors into a single latent space that captures the motion variability across gaits and poses simultaneously. We propose a new Gaussian processes (GP) based dimensionality reduction (DR) algorithm to learn a torus-like JGPM that balances the desired manifold structure with the actual intrinsic structure among data. JGPM is further used to learn a visual gait generative model for motion estimation. Experimental results show that the proposed JGPM provides superior performance on human motion modeling compared with other GP-based methods, and the results on video-based motion estimation are also among the best in the literature.;The major contribution of this dissertation is on the structure-guided manifold learning. It is a critical issue when we are dealing with a sparse and unorganized (without explicit topology) data set and when the prior knowledge of the expected manifold structure is involved. This idea can be applied to other manifold learning applications that may be encumbered by a limited training data set without a clear topology.
展开▼
