Body shape and movement are related to human health. Our shape tells us about our body fat for example and our movement tells us something about the health of our motor system. We explore how 3D body shape and movement are related to health. We bring our 3D body models to this task and by using computer vision we work towards non-invasive and lightweight methods for analyzing human health.
For example, Cerebral Palsy (CP) is a motor disorder that can have lifelong impact but, if detected early, effective therapies can minimize the impact in later life. CP can be diagnosed early in infants using General Movements Assessment (GMA), a non-invasive, inexpensive technique based on the observation of the infants' spontaneous, undirected movements. Unfortunately, GMA requires expert training that is not widely available. If we can automatically track infant movement using computer vision, we could possibly automate GMA and the early detection of CP. The vision community has made great progress in 3D human tracking but has focused on adults. Infants have a very different body shape from adults (see figure), which makes it difficult to directly extend work on adults to infants. To address this, we learn a model of infant body shape [ ] and use it to track 3D movement in RGB-D sequences. Previous models of 3D humans [ ] have been learned from thousands of high quality 3D scans. However, scanning infants is challeging, as they can not be instructed to strike poses on demand, and have additional strict requirements, such as hygiene and warmth. Consequently we deployed a low-cost, non-invasive scanning device in infants hospitals, and scanned over 30 infants. We develop a novel method that learns infant body shape directly from these low quality, uncomplete RGB-D scan sequences.
Another example involves the distribution of body fat in the body. Not all fat is the same. Visceral adipose tissue (fat around the organs) is highly correlated with health problems like diabetes and cardiovascular disease. In contrast, sub-cutaneous adipose tissue (fat under the skin) is relatively benign. Today an analysis of this fat distribution requires and MRI scan to reveal where fat is stored. In our work, we are exploring whether we can estimate this fat distribution purely from the surface shape of the body. To that end, we have developed methods to fit our 3D body models to full-body MRI scans [ ]. We fit the 3D body surface to both the external surface and the internal fat layer. This allows us to analyze where the body fat is and what percentage is under the skin. We are collecting a dataset of matched MRI data and 3D surface shape and our ongoing work is focused on predicting what is inside solely from the surface.