[Overview]

This project is a news reporting project to which I chose to explore the combination of Metahumans, animation, photogrammetry, and motion capture to create a dynamic scene. I recreated myself as a Metahuman by 3D scanning my face using the photogrammetry, retargeting it to a Metahuman ID, and training the facial rig for realistic animation using different facial poses. Using the UE Live Link app, I animated the Metahuman to deliver a news report with my facial expression and motion capture.

The project shows a 3D scene where the news report is displayed on TVs, inspired on the original movie TV report scene. The backdrop for the Metahuman reporter uses animated textures created with Mega lights, while the audio was edited in Pro Tools for effects. The final scene involved embedding the news report video as an animated texture into a separate Unreal Engine environment, where I modeled and textured a TV using Maya Autodesk and Substance Painter.

[Challenges & Growth]

Metahuman Creation:

• retargeting a 3D scan of my face to a Metahuman ID and training the facial rig for natural expressions.

• Syncing Live Link animation with audio for natural lip-syncing and gestures.

  
  

Environment Design:

• Creating an immersive backdrop using animated Mega lights textures.

• Embedding the news report into a separate Unreal Engine environment and ensuring the video played correctly on TV screens.

  
  

Asset Creation:

• Modeling and texturing a TV in Maya Autodesk and Substance Painter to match the environment’s aesthetic.

• Ensuring consistency between the Metahuman animation and the 3D environment.

  
  

Audio Integration:

• Tweaking and syncing audio in Pro Tools to match the Metahuman’s animation and the overall pacing of the news report.

3D Scan & retargeting

I used the Polycam app to create a 3D scan of my face, ensuring high detail and accuracy. The scan was then retargeted to a Metahuman ID, preserving my bone structure while enabling advanced animation capabilities within the Metahuman rig.

Customization

I adjusted the Metahuman’s body proportions and features to match my body and face through the Metahuman Creator page. This included tweaking the facial structure, skin tone, hair, eyes, and clothing to match mine as best possible.

Achieving Textures

After customizing my Metahuman, I brought the Metahuman blueprint out onto the scene and imported my own normal and cavity map out onto the scene to simulate peach fuzz and pores on the face. I then adjusted the specularity to 0.7 for some added sharpness.

Live Link Animation

I animated the Metahuman using the UE Live Link app, syncing facial expressions and gestures with the audio. I filmed myself using the Metahuman animator app and recited the news broadcast, then imported it onto Unreal Engine to sync the expression with the Metahuman rig.

Front View

Posing & Animation

To clean up the Metahuman broadcast I posed it in a reporter-like stance and leaning forward to convey sense of emergency and authority. The animation was refined to ensure smooth transitions and natural movements throughout the motion capture performance.

[The result]

Breaking News: Monster demonstrates the potential of creating digital twins using Metahumans, 3D animation, photogrammetry, and motion capture. The Metahuman version of me, dynamic backdrop, and motion capture integration demonstrate my ability to blend technical skills with creative storytelling.