Proving the Business Case for the Internet of Things

Carnegie exercise camera works better than wearables

Steve Rogerson
September 17, 2019

Researchers at Carnegie Mellon University have developed a camera system they claim monitors gym exercises better than wearable sensors and smart watches, and can blur out identifiable faces that would intrude on privacy.
The problem, they say, is that while being a popular motivational tool for fitness enthusiasts, gadgets do not sense all exercises equally; they have found that a stationary camera is a better choice for gym exercises.
The vision-based system, called GymCam, detects repetitive motions. By doing so, Rushil Khurana and Karan Ahuja, both PhD students in the Pennsylvania university’s Human-Computer Interaction Institute (HCII), found they could detect exercises in a gym. Moreover, they could recognise the type of exercise and reliably count repetitions.
"In a gym, the repetitive motion almost always is an exercise," said Mayank Goel, assistant professor in the HCII and Institute for Software Research. "If you are moving both your arms, you tend to move them together in time. However, if two people are exercising next to each other and performing the same exercise, they are usually not in sync, and we can tell the difference between them."
Because the system only needs motion information, the camera feed can be reduced to pixel-by-pixel changes and eliminate identifiable faces that would intrude on privacy.
Khurana said reliance on motion information also addressed a problem for single-camera systems in a crowded gym environment – the inability to see a person's whole body. Gym equipment or other people can often obscure the camera's view. GymCam, however, can detect exercise as long as its camera can see any body part moving repetitively.
Khurana and Ahuja presented their findings at last week’s International Joint Conference on Pervasive & Ubiquitous Computing in London.
Ahuja said smart watches and other wearables do a reasonable job of tracking many cardio exercises and some strength-training exercises. But their effectiveness depends on where the wearables are worn. A smartwatch might sense a dumbbell lift, but is useless for leg presses. Moreover, it is hard for a watch to differentiate between several body motions. Instrumenting the exercise machines is an option, but an expensive one. A camera, however, is relatively cheap and provides spatial as well as motion information.
The system can also learn the location of types of exercise machines or certain exercise stations in a gym. It can then use an individual's location, in addition to their movements, to determine the exercise they're doing.
The researchers tested their algorithm in a crowded gym. But Goel said the same algorithm worked perfectly on a smartphone as well, so a person could use their phone to record and track their workouts at home. Some companies have already expressed interest in using the system for tracking in-home exercises.
The system also might have uses beyond physical exercise. Goel said the camera system, combined with smart watches worn by individuals, might help people with visual disabilities navigate shopping malls, airports and other public spaces. Instead of using the person's face as their identity, the system will use their motion as their signature. It allows people to opt-out easily of being tracked or located.
In addition to Khurana, Ahuja and Goel, the research team included Chris Harrison, associate professor in the HCII; former HCII faculty member Jennifer Mankoff, now at the University of Washington; and Zac Yu, a former visiting researcher in Goel's lab who is now at Google. The Carnegie Bosch Institute sponsored part of this work.