← All projects
2022Augmented Reality

Augmented Reality Communication

A browser-to-HoloLens communication prototype using webcam-driven avatars, audio and text messaging.

This project connects a remote browser user with a HoloLens 2 participant through an animated avatar. It investigates how facial and upper-body movement can be represented in a shared session without displaying a conventional video feed.

  • Augmented Reality
  • Communication Systems
  • User Interface Design

Project objective

The objective was to implement remote communication between a person using a standard web browser and another using a HoloLens 2. The headset participant sees an animated representation of the remote person, while both participants can exchange audio and text messages.

The prototype combines webcam-based movement estimation, avatar rendering and a session that can be viewed on a conventional screen or in augmented reality.

Image placeholderaugmented-reality-communication__image_01

Design rationale

An avatar provides a way to transmit selected nonverbal cues while changing how a participant is visually represented. The project focused on head orientation, facial expression and upper-body movement as inputs for this representation.

The implementation explored an alternative presentation of a remote participant. It did not establish whether avatars improve communication quality compared with video calls.

Session workflow

The remote participant starts a session in a browser. A second participant joins the same URL on the headset and can enter an augmented-reality view using the Start AR control. The shared session also remains accessible as a conventional web view.

Image placeholderaugmented-reality-communication__image_02

Avatar behaviour and communication

The avatar follows estimated head rotation and tilt, shoulder movement and facial features, including mouth, eye and eyelid movement. Audio controls and text messaging operate alongside the avatar.

A participant can also view the avatar directly in a browser. This provides a screen-based version of the interaction without requiring the headset.

Implementation and scope

MediaPipe extracts landmarks from the camera stream. Kalidokit maps those landmarks to rotations and facial animation parameters for a humanoid VRM model. VRM models use a glTF-based format with a defined humanoid structure and expression controls.

The project used downloadable avatar assets selected according to their usage conditions. Its main technical contribution was integrating movement estimation and avatar animation into the communication session.

Potential extensions identified in the project included shared object visualisation and remote maintenance guidance. These remained proposed applications rather than demonstrated outcomes.