What is Motion Capture? What does Qinematic focus on and why?
What is mocap?
Motion capture, often shortened to ‘mocap’, is what we call the technology for recording the movement of objects, including people, animals, vehicles etc.
These days, most people have heard of self-driving cars that can see other objects, recognise them as a person or a tree, and react in lighting fast time. And whether you know it or not, most action films these days are not putting the lives of Hollywood stars at risk in remote jungles, they are dressing them up in mo-cap suits and having them jump around in a green room. The computer generated character uses 3D modelling, and digitally animated jungle backdrops and wild explosions in a studio in London or LA rather that on location in Costa Rica.
Types main types of mocap
1. 3D optical time-of-flight camera recording
2. 3D inertial device recording
3. artificial intelligence that processes video input
There are pros and cons with each method. Some considerations include: hardware costs, operating costs, time to set-up and calibrate, storage space, post-processing of data and images, missing data due to occlusion of body parts during movements, frame-rate limitations, accuracy of the tracking models, ease of use, ease of understanding the data, portability, use indoors versus outdoors, lighting conditions, reflective light, magnetic interference, camera field of view etc.
With human mocap, some systems only show what is called a ‘rig’ or the skeletal model of the person during movement. This rig can be generated using multicamera marker based systems (reflector balls or diodes), mocap suits lined with inertial sensors, and 2D video processed by deep learning algorithms.
The digital twin
Qinematic’s Moovment show’s the actual 3D image of the person (without showing too much detail in the face to maintain anonymity). This way, the user can see the actual body in 3D and the skeletal tracking inside of the body. If the tracking and the digital twin image don’t match, it is very obvious.
The rig
Many systems that show only the rig, a rig with an animation overlay, or a 2D video and a rig will try to hide the skeletal tracking points that are jumping around by ‘smoothing the data’. Although this makes the rig look cleaner, it can reduce accuracy. To be fair, jitter is almost impossible to avoid. It can be seen when using most software development kits (SDK) that come with sensors for playing games, like the Microsoft 360 Kinect sensor. They dont care about accuracy, only having fun. Realistically, data will hop around because we are talking about pixels and milliseconds. Qinematic spent 18 months fine tuning the tracking algorithms for the knee, to get them as accurate as possible, and avoid having the knee jitter half way up the thigh, or the ankles hovering above the ground.
Occlusion
Data can also get lost altogether when movements cause occlusion of other body parts. The hip disappearing behind the knees in an overhead squat is the classic example. Thanks to Function Movement Screening (FMS) making the overhead squat a popular (although flawed) assessment test of function, a lot of mocap tools blindly implement it, at the expense of accurate data. If you want to see a lot of jitter, look at the hips and knees during an overhead squat. This happens with humans as well, so don’t be too harsh on the motion capture devices - humans cannot see the hips behind the knees either - they only see their imagination at work! Qinematic tries to avoid movements that involve occlusion (as should humans) in order to minimise the need for smoothing. Our customers like to know that actual recording is being measured, with as little ‘synthetic data’ as possible. The alternative is to have a multicamera set-up that can see most body parts almost all of the time, but this gets expensive.
Sensors
Qinematic has never manufactured sensors - that is a job for the big boys like Microsoft and Intel. They usually focus on gaming and manufacturing, where the big bucks can be made. Health and fitness is still a niche area for us smaller geek companies that are fascinated by human movement and want to change peoples’ lives for the better.
Back in 2012, we played around with all kinds of wearable and non-wearable sensors in the clinic and gym in Stockholm, Sweden. Our clients thought we were nuts. We tried attaching IMUs and other devices to the body, but it was clunky and time consuming with calibration and affixing things correctly to the body, it was not whole-body, and it seemed to change the way people behaved. We wanted to see the real ‘You’, but in a relatively standardised way. We decided quite quickly that the (Prime Sense) 3D camera was awesome. Our clients agreed - seeing themselves in 3D was mind-blowing, and educational. And therefore very motivating - the key to change in health and fitness!
What health and fitness providers want
It took 5 years to thoroughly interview health experts and end-users, design what they needed, and release a desktop software (DaaS) that used an affordable 3D sensor (XBox Kinect ). In 2017, motion capture using this sensor was old news to tech geeks and gamers, but still very new to health and fitness professionals - like ‘Star Trek’ new!
Users testing an MVP with physiotherapists and chiropractors in Holland in 2013. What we learned: the skeletal animation scared off a lot of clients, and the 15-page biomechanics report was overkill.
Digital twins in the cloud
By 2019, gyms, health centres, insurers and universities all over the world were using our software for prevention, fitness and research. Based on some research we did with EIT Digital over 18 months with some very clever people from three EU countries, we created an online console known as the Moovment Pro Portal for clients and providers to meet in the cloud, for asynchronous and online consultations. Although ‘online services’ became quite the norm in 2022, online fitness and rehab were also quite ‘Star Trek’ before the COVID pandemic.
3D is accurate, 2D is convenient
With advances in hardware technology, we added the Kinect Azure sensor integration in 2019, and the Orbbec sensor in 2024 - read our 3D time-of-flight (TOF) sensor tech blog.
During the COVID pandemic (let’s say 2020-2023), most of our bricks and mortar clients offering 3D scan services were closed or running on empty. We had to bring the Moovment scan service to the people, in their homes. Using your phone, wherever you are, is without a doubt more convenient, but it is less accurate (don’t believe otherwise), and unlike many other markerless motion capture products out there, Qinematic is all about accuracy. In late 2021, we added the 2D mobile phone scan through necessity, rather than choice. It is still not as accurate as the 3D sensor (no matter what anyone tells you), but having a camera in every persons pocket around the world is very convenient.
In 2022, B2B customers could subscribe for both 3D and 2D services, and provide scans to their clients onsite or remotely.
Who else is using mocap to measure outcomes
Some big health service companies like Kaia Health and Hinge Health have added their own mobile scan services, and it has been a game changer for them. Watch their demo videos and decide for yourself if the tracking is accurate (with all due respect).
Screenshot from the Kaia Health website showing IMU tracking on the left, and the neural network tracking applied to 2D film as used by Kaia on the right. The yellow arrows point to the location of the IMUs, and the intersection of the white lines (bones) is assumed to be the knee joint, and presumably the correct knee angle (in green). The red dots have been added to show a best guess of where the actual joints (and therefore bones) might be located at the ankle, knee and hip. Both systems do an OK job, but they are not accurate.
Artificial Intelligence or Augmented Intelligence
In 2024, the world is buzzing with talk about artificial intelligence and mixed reality. OpenAI started back in 2015 with $1B of pledged funding, was still very ‘Star Trek’ in 2022, and now it is HOT! Naturally, there are AI actors with their co-pilots popping-up all over, and in all industries, including health and fitness. The EU is still discussing what type of AI is acceptable for their citizens health data. One things is for sure, a doctor, physio, or personal trainer with AI is going to outperform anyone without AI. The power from augmentation will grow exponentially, as will the value creation for their clients/patients, payers, coaches and caregivers.
Qinematic develops Augmented Intelligence solutions. Qinematic knows human movement science, rehabilitation science, and applies it to mocap software to augment the experts - to improve performance, remove documentation, increase accuracy, improve productivity, encourage learning and boost customer satisfaction. Qinematic has been collecting ‘clean data’ for years. We started machine learning ground-work in 2018 through 2020, and we already have unique ways to classify movement using 3D data (in milliseconds and millimeters).
Now that Apple has taken the lead with a commitment to mixed reality with the Vison Pro headsets, and Meta Smart Glasses with Ray Ban for the fashion conscious, consumer self-awareness about the virtues of 3D visual information is about to rise exponentially. 2D will not be good enough in the near-future.
Which Moovment by Qinematic mocap to use
If you want top-quality tracking, with a simple user interface, that scans automatically without personnel, then Moovment Scan using a 3D TOF sensor (Orbbec) is the way to go. Don’t settle for animated visualisation - go for the real point cloud image. You never really know about the accuracy of the animation. With awareness/education in mind, clients respond much better to an image of themselves. We have tested this on people of all ages, education and cultural backgrounds. A real person works better every time.
Not quite an ultrasound of baby, but a magnificant video of a soon-to-be mother using a harmless infrared light 3D TOF sensor. This pregnant lady was experiencing back pain and poor range of motion, and provided a remote consultation in the comfort of her home.
If you want convenience at the cost of accuracy, and you don’t want the awesomeness of a 3D digital twin, then AI tracking using a 2D image from a phone is better than a human just eyeballing (yep that is what most coaches, physios and doctors still do, despite your ranking, excruciating pain or disability). A video recording beats the memory of any fitness instructor, and you get unbiased estimates of human movement that are quite good from the frontal view - and less biased than that fitness instructor, for sure. If convenience and home use is a priority, then Moovment.Pro is a good choice. It offers the same carefully selected test protocol as Moovment Scan, without single leg balance (for safety reasons), so you can compare both 3D video (taken onsite) and 2D video (taken at home). Both are visible in the online Portal. Using both modailities gives you quality Assessment for assisting in an accurate assessment, and and you get Home Monitoring for progress reporting as often as you like.
Glenn Bilby
Founding CEO, Qinematic AB
Human Movement Scientist, Physiotherapists, MBA