Gymnagic

Current tools for quantitative analysis in gymnastics rely on complex equipment like force plates and advanced multi-camera setups to capture athlete performance, compromising the ability to assess real world performance of athletes in their natural training environment and across complex sequences of movements that cannot be replicated in a lab setting.

Gymnagic provides a way to perform multi-faceted analyis of performance using consumer grade video equipment (such as iPhones) that can be easily deployed in gyms during an athlete's normal training session. The analtyical framework processes the resulting video through a series of components that individually solve the key problems needed to produce automated, unsupervised performance analysis.

• Pose Estimation (identifying athlete, body and joint positions and angles)
• Detection of key moments (identify phases of movement and critical moments e.g. impact with floor)
• Center of Mass estimation (of a moving body in contorted positions)
• Kinematic analysis (speed, forces, angular moments)

The framework is built in a modular fashion where each component can be improved upon, and swapped out, should advances in any of the relevant fields provide a more accurate computational approach for any one step.

Results and Possibilities

The Gymnagic toolbox can be leveraged to answer a variety of questions, building in complexity, that can shed light on athletes' performance as well as risk of injury.

• Normalization: What is the normal range of movement (RoM), speed, angles, and impact forces across athletes and possible correlation with other athlete characteristics (age, height, weight, static RoM).
• Quantified Self: Tracking changes in performance of a single athlete in identical movements over time e.g. across multiple reps, training sessions, or a season.
• Injury Risk: Analyze data across variety of athletes and reps to pinpoint causes or correlations with factors tha increase risk of injury e.g. Achilles tendon tear.

Below are sample videos with analytics

Components of Analytical Framework

Pose Esimation

Prior work on pose estimation in gymnastics movements has mostly relied on complex infrastructure involving synchronized multi-camera videography (to estimate 3D position and point cloud data by stitching together separate 2D video streams), burst photography, and training models by manually labeling training data sets of specific movements. Each of these approaches has severe limitations in terms of portability, resource requirements, and the ability to analyze real world videos in athletes' natural training environments.

Specific challenges in sports video analysis and in gymnastics in particular include:

• High speed of movement of joints as an athlete executes a skill, with drastic changes in joint position and orientation of the body within fractions of a second.
• Temporal moment of interest for analysis (e.g. moment of impact on the floor or apparatus) is that much more fleeting.
• Gymnasts exhibit unusually high range of movement in their joints, contorted body positions, and body orientation (with a fair amount of time in inverted positions) relative to the humans in the training data sets of pre-trained models.

Gymnagic builds on the latest in pre-trained machine vision models, combined with careful selection of camera angles, distance from athlete and video frame rates, to accurately perform pose estimation using videos taken with a consumer grade cell phone camera in an athlete's normal training environment even with a visually noisy background.

Samples of pose detection in gym environment.

Center of Mass (CoM) Estimation

Estimating CoM is a prerequisite for calculating average speed of athlete's body through the movement and in turn calculating force, angular moments, and other measures of interest.

Naive implementations of CoM estimation generally place CoM in the position it would be found in a static, standing, upright body with hands by the sides (i.e. in the horizontal center of the torso just below the belly button). However, when analyzing gymnastics movements, this basic estimate can fall very far short on accuracy since athletes' bodies are often in contorted positions as they go through their movement where the CoM is frequently outside the body.

Gymnagic computes a far more accurate CoM position by combining the results of the pose estimation above with prior anthromorphic studies to calculate a real time CoM position that is based on the specific orientation of limbs in space at that specific moment. Each body segment's individual CoM is estimated by finding the center of the pose coordinates of the segment extremities, and then full body CoM is calculated by taking the weighted average of the body segment coordinates where the weights are average body segment weights from historical anthropomorphic studies.

The results can be sanity checked by examining the estimated CoM position at specific moments in time where we have an intuitive understanding of where the CoM should be. Some examples are presented below.

Sample of CoM estimate for athlete standing upright. We intuitively expect CoM to be in the torso near the belly button and directly above the base of support i.e. feet, in the horizontal plane.

Sample of CoM estimate for athlete passing through backbend/arched position during a back handspring. We intuitively expect the CoM to be outside the body and behind the athlete's back (lower in the vertical plane than in the naive implementation).

Sample of CoM estimate for athlete passing through handstand during a back handspring. We intuitively expect CoM to be in the torso near the belly button and directly above the base of support i.e. hands, in the horizontal plane.

Sample of CoM estimate for athlete in partial forward fold before initiating a back handspring. We intuitively expecxt the CoM to be outside the body and in front the the athlete's waist.

Detection of Key Moments in Time

Once joint positions have been ascertained and CoM position computed, we must next identify the moments in the video that are of greatest consequence for kinematics calculations, or greatest interest for the purpose of our subsequent analysis.

Moments of Impact are of critical interest because calculations of the angle, speed, force of impact on the joint are a key measurements that will form the foundation of our movement analysis. These are estimated by analyzing the movement of the extremities making impact (wrist joint, ankle joint) in the vertical plane and then finding all maxima and minima. The position of the ankle joint at the start of the movement is assumed to be the position of the floor in the vertical plane, and subsequent moments when the wrist joint or ankle joint is at a minima and reaches the level of the floor in the vertical plane represents the moment of impact.

Plot of the position of the wrist and ankle joints in the vertical plane over time, identification of minima/maxima, proximity to floor, and estimating impact.

Frame identified as a moment of wrist impact during a back handspring following the process above.

Frame identified as a moment of ankle impact at the completion of a back handspring.

Moment of Apex is the moment when the athlete's CoM reaches the maximum height after taking off from the ground. Combined with the the moment of the next impact, this can be used to estimate the total time that the athlete spent airborne, and hence calculate the speed with which the CoM is moving in the vertical plane, as a result of the pull of gravity, at the time of impact. Similar to calculation above of the moment of impact, we estimate this moment by tracking the position of the CoM along the Y axis over time and identifying the maxima. The maximum height reached between two subsequent moments of impact is the moment of apex.

Kinematics Analysis

Armed with the parameters calculated above, and some particulars about the athlete (such as weight and height), we can now go about our kinematic analysis.

Speed at the Time of Impact in the vertical plane can be estimated from the time spent airborne and using some basic physics. Once the athlete is airborne, the only force acting on the athlete is gravity. Gravity progressively decelerates the upward motion of the CoM until its velocity reaches zero and the CoM is momentarily at a standstill in the vertical plane (this corresponds to the Moment of Apex calculated earlier), before falling back towards the ground under the pull of gravity until the athlete makes impact (this corresponds to the Moment of Impact calculated earlier). The time between these two moments is the total time (t) the CoM has spent in freefall so we can estimate the terminal velocity (v) using the equation:

where g is the acceleration due to gravity = 9.8m/s^2)

Force of Impact can be estimated next by using the veloctiy estimated above (v), the athlete's mass (m), and an assumed duration of impact (e.g. 0.2 seconds) by using the equation below:

Future Work

While we have started with simple skills (e.g. back handspring) in a single event (floor) with a small number of athletes and facilities, the same framework can be built upon to generalize the analysis, and expand the scope to cover more athletes, gyms, skills, and events.