{"id":3573,"date":"2021-03-03T14:04:56","date_gmt":"2021-03-03T14:04:56","guid":{"rendered":"http:\/\/audienceofthefuture.live\/?page_id=3573"},"modified":"2021-03-03T14:04:56","modified_gmt":"2021-03-03T14:04:56","slug":"poster-6","status":"publish","type":"page","link":"https:\/\/audienceofthefuture.live\/poster-6\/","title":{"rendered":"Modern Collectibles: Implementing Procedural and Spatial Audio in ARKit with CoreML"},"content":{"rendered":"

Poster<\/h1><\/div>

Modern Collectibles: Implementing Procedural and Spatial Audio in ARKit with CoreML<\/h3>

This study is an investigation into how immersive audio design for AR can be used to bring objects and the stories behind the \u2018Modern Collectibles\u2019 to life in the users\u2019 hands and invoke specific emotional responses as well as greater engagement in the show\u2019s subject and online offerings.<\/p><\/p>

More Details
\u2212<\/div><\/h4>

The Discovery Channel partnered with StoryFutures for the \u2018Modern Collectibles\u2019 project to produce a TV pilot with an accompanying augmented reality (AR) Apple iOS application. The experience would enable users to explore retro objects from the 1980s, using augmented reality to bring the collectible items \u2013such as a toy ATAT- to life in their own homes. Development was in collaboration with Story Things to use technology to augment the traditional space of linear television programming and execute a strategy of \u2018view and do\u2019<\/p>\n

An innovative pipeline allowed for research and development opportunities in audio design and implementation. Unity game engine and ARKit was used for an iOS build application to leverage the latest behavioural and detection capabilities in an AR scene.<\/p>\n

Computer vision was incorporated (CoreML) to obtain gestural data from users\u2019 interaction with an AR object. Controls of touch, hold and velocity of a users\u2019 hand and finger with an AR object were developed to drive sound design in real-time.<\/p>\n

Pure Data (Pd) \u2018patches\u2019 were integrated using the embeddable library \u2018libpd\u2019. Unity C# scripts combined and communicated with CoreML values and data were parsed through the \u2018libpd\u2019 layer to Pd patches containing receiving objects. Interaction, animation and scale of the AR object parametrically drove synthesis in real-time <\/p>\n

Early testing indicated that immersive audio design coupled with gestural control does have a positive impact on engagement in the context of AR and contributes to bringing \u2018modern collectibles\u2019 to life through invoking nostalgia.<\/p><\/div><\/div>

<\/div>

Rich Hemming<\/p>

<\/div>
<\/div>

Royal Holloway University of London<\/p>

<\/div>
<\/div>

Biography
\u2212<\/div><\/h4>

Rich is a Doctoral Researcher in implementing procedural and spatial audio in virtual environments within the Electronic Engineering dept. His PhD focus is on assistive technology for the blind and visually impaired creating a \u2018visual-auditory sensory substitution device\u2019 to aid spatial awareness and navigation. <\/p>\n

He was Head of a Game Audio Ba\/MA and is involved in audio design for immersive experiences, pursuing research and development opportunities in commercial and academic projects.<\/p><\/div><\/div><\/div><\/div>

Modern Collectibles: Implementing Procedural and Spatial Audio in ARKit with CoreML<\/h3>
<\/div>

Show all<\/a><\/p><\/div><\/div><\/div><\/div><\/div>

Gallery<\/h4>
<\/div>

Modern Collectibles: Implementing Procedural and Spatial Audio in ARKit with CoreML<\/p>

https:\/\/drive.google.com\/file\/d\/1szrOM3p1kLIatBYe7wiVetubN-tZRFCp\/view?usp=sharing<\/p>

Modern Collectibles: Implementing Procedural and Spatial Audio in ARKit with CoreML<\/p>