Unveiling the innovative MUSIC project – a tech marvel employing discreet inertial measurement units (IMUs) to seamlessly capture facial expressions. Wave goodbye to camera-based methods and embrace enhanced privacy; MUSIC doesn’t require video footage, safeguarding your personal space while adeptly tracking minute facial movements.

Here’s what sets MUSIC apart:

  • Enhanced Privacy: IMUs pick up expressions without ever recording your face, ensuring a high privacy standard.
  • Unhindered by Obstruction: MUSIC can decipher your smiles and frowns, even with facial obstructions that would stump traditional cameras.
  • Spot-On Expression Detection: The sensitivity of IMUs to tiny muscle shifts ensures that even the slightest of expressions won’t slip by unnoticed.
  • Convenience on the Go: Tiny and light, IMUs promise easy integration with gadgets galore, taking the power of MUSIC to entertainment, health, and beyond.

The nitty-gritty of how MUSIC serenades your expressions:

  1. Sleek Hardware Design: The strategically placed micro IMUs hug your facial contours for optimal motion tracking.
  2. Precise Data Harvest: Collating raw expressions through IMUs, the system ensures pristine data quality via rigorous calibration.
  3. A Rich Dataset: The IMU-ARKit dataset lays the ground with a reservoir of IMU-visual combos, all set for model training in capturing a gamut of expressions and performances.
  4. Predictive Model Mastery: Armed with a custom Transformer diffusion model, MUSIC predicts facial contours from IMU signals, refined through a dual-phase training strategy for top-notch expression interpretation.

Official Website

YouTube video player

Official Website