Instruments of the U is an article series covering the incredible research tools and equipment used by leading researchers. In this edition, we’re covering the Division of Games’ Motion Capture Lab—its history and its applications for student projects, including games, art and more.

The Motion Capture Lab is a 400-square-foot space housed within Building 72 in the Division of Games (Utah Games). It operates using a suite of tools and equipment, including a dedicated desktop system, a constellation of 12 OptiTrack cameras, specialized software, and the iconic piece of motion capture—the suit worn by actors. The suits are made of form-fitting spandex and secured with Velcro. Spherical reflective markers are attached to the suit, which reflect the infrared light picked up by the lab’s camera system.

Together, the cameras build “points” based on each marker through triangulation. These points are arranged in 3-D space to form a skeleton, which serves as the structure for virtual characters.

This system is fully dynamic and captures the actors’ movements with precision. Whether it’s a static pose, a delicate dance or martial arts, motion capture records an amazing level of detail. However, the caveat is that the post-production process is extensive.

“Motion capture isn’t necessarily plug and play,” said Brian Salisbury, professor of Utah Games within the College of Architecture and Planning. “[After production], you’ll need to trim for brevity, make adjustments or add embellishments, like speeding up a punch or making a kick a little higher. There are also noises, like a little bit of wiggling or bouncing on a marker, that need to be smoothed out.”

Another challenge for motion capture artists is occlusion: when markers overlap or are blocked by the actors. This often happens in scenarios with multiple actors or intricate movements like wrestling demos. In these instances, an animator—oftentimes the students involved in the project—steps in and fills in the blanks. Animators also work on creating animation sets that are used for a variety of purposes, including demos, cutscenes and interactive gameplay. As games are interactive, a key design philosophy for animators involves fine-tuning the timing of each animation to ensure the right “feel” for the player.

“It’s why games are distinct from film. A player has a controller, and when they press the jump or kick button—they want the character to perform that jump or kick right now,” said Salisbury.

Depending on the project, students may need to incorporate “anticipation” into the animations. These are motions that indicate incoming actions, such as a wind-up for a punch. Presenting this to the player, while still following the philosophy of responsive movements, is an added obstacle for students.

“Game players want instant response while retaining the essence of the animation,” said Salisbury.

Lastly, animation work in the Motion Capture Lab also involves cyclical and static motion, usually taken from a motion sequence. Movements such as walking or standing in one place involve little activity and are edited to form neat loops where the start and end of the animation flow seamlessly. The complex part of this lies in shifting between different animations, such as a character executing a combat move like a punch from a stance and then returning to that stance. Students can incorporate this using motion trees or state machines within the animation software.

Beyond software work, the lab space also involves interpersonal, project management and other technical exercises. The motion capture system requires careful calibration for each actor to ensure the proportions of their character model are accurate. Each shoot also necessitates detailed preparation in choreography for the actors. A director must advise actors and motivate them to portray specific emotions or behaviors, a task that’s far easier said than done. Oftentimes, multiple takes are required, which lengthens the overall time needed for each project. This is valuable know-how in both academic and professional environments. Overall, the process highlights the variety of skills that students learn and apply in working on their projects.

“It is a hoot, a lot of fun, [and] it’s been used to help out on a lot of [different] projects,” said Salisbury.

The Motion Capture Lab offers a comprehensive platform for students to realize their creative visions in games but getting to its current status was a journey. Its origins began with machinima—creative projects like films shot entirely within a game’s engine. A class focused on developing machinima was pioneered by Robert Kessler, professor in the Division of Games. The course was well-received by students and faculty alike, and it laid the foundation for the Entertainment Arts and Engineering program and the B.S. in Games. As these programs were developed, so too was the need for contemporary game development tools. Thus, the Motion Capture Lab was created from an academic grant. It has since supported hundreds of graduate and undergraduate projects in over 11 animation and machinima courses, as well as a dedicated motion capture class—GAMES 3650.

Though developing the academic infrastructure was a process, the lab’s technology has largely remained similar.

“Optical capture really hasn’t changed a whole lot in the last 30 years. Cameras have gotten better with higher resolutions, more powerful computers and the software is more sophisticated, but the prevalent technology is still the use of optical markers,” Salisbury said.

However, new innovations have been incorporated. Real-time streaming of captured movement directly into a game engine has significantly improved project efficiency for students. Animators now use AI to process video, which is known as marker-less capture whereby actors no longer wear the spandex suit with optical markers. While it has yet to match the precision of the marker-based method, it is a promising development for the technology.

An exciting expansion of the Motion Capture Lab is also on the horizon. “We intend to grow. More cameras with way higher resolution, and not only that—we’ll have the ability to capture finger [movements]. It’ll be state of the art,” said Salisbury.

Overall, the Motion Capture Lab enriches the education and experience of hundreds of students by equipping them with skills in animation, technology and creative problem solving. Its impact is seen not only in the quality of their projects, but in the skills and confidence they carry into their professional careers.

• Paulina Sylvester assistant director of communications, School of Architecture + Planning
  p.sylvester@utah.edu