Supporting cutting-edge volumetric video-based performance

Research and education networks in the Asia Pacific region collaborated to provide the ‘SuperSpeed’ network required to deliver an interactive live volumetric video performance between Sydney, Australia and Tokyo, Japan at SIGGRAPH Asia in December 2023.

Cutting-edge volumetric video technology

By capturing and reproducing a three-dimensional representation of the performer, volumetric technology allows the audience to experience the performance in a more immersive and interactive way than a traditional two-dimensional video experience.

This technology allowed the violinist in Tokyo and the audience in Sydney to co-create improvised music during the performance. The audience in Sydney was able to freely move the viewpoint from the front, side, back or close to the violin and the violinist was able to respond to the audience’s actions to create music.

The performance was streamed in real-time from the Volumetric Video Studio at TOKYO NODE using 57 cameras in an immersive green screen environment. The captured video was converted to a point-cloud using proprietary cutting-edge volumetric video technology.

The violin is performed from the Volumetric Video Studio in TOKYO NODE. Credit: Keio University Graduate School of Media Design (KMD).

Walking around the performer to move the viewpoint. Credit: Keio University Graduate School of Media Design (KMD).

‘SuperSpeed’ research and education network connectivity

High-speed and stable transmission of data was vital for the performance experience, and this would have been difficult to achieve using ordinary network circuits. Research and education networks like AARNet are purpose built for delivering the high bandwidth and headroom required for transmitting huge volumes of data, such as produced by volumetric video technology, and research more broadly.

The SuperSpeed research and education network connectivity between TOKYO NODE and the International Convention Centre Sydney was provided by AARNet, Australia’s Academic and Research Network, ARENA-PAC, Arterial Research and Educational Network in the Asia Pacific operated by the WIDE Project, and GOREX, The Guam Open Research and Education eXchange.

This performance was made possible through collaboration between Keio University Graduate School of Media Design (KMD), Canon Inc., Bascule Inc., and Mori Building Co., Ltd., WIDE Project with the Cyber Civilization Research Center (CCRC) at Keio University, which coordinates the research and education network.

Read the original media release

Featured image: The audience in Sydney enjoy the performance from Tokyo with point-cloud volumetric video. Credit: Keio University Graduate School of Media Design (KMD).