Latency experienced using the headtracker

Latency experienced using the headtracker

BauerBauer Member Posts: 8
unity 2018.3
iOS 12.2
Xcode 10.2.1
iphone7

Hi !
I am currently trying to build an audio AR experience using GPS location based things (mapbox-unity-sdk_v2.0.0), google resonance audio (ResonanceAudioForUnity_1.2.1), and the Bose AR SDK (BoseWearable-Unity_v0.13.0).

Everythings works really well on the unity editor. But when I am building the app on iOS, I experience a latency on the head tracker that is not here when I am working on the unity editor. To be more specific, I use the USB when working on unity editor and unity remote for the phone. 

The latency I experience is around 500ms for the headtracker.
Does anybody would have an idea where it would come from ? I have tried to look at many places and parameters (decreasing graphics, trying on another phone, different unity version...) but nothing is successful yet.

Any help would be so much appreciated !!
Many thanks
Valentin

Comments

  • BauerBauer Member Posts: 8
    Additional thing: I have some visuals on my app with a prefab visuals of the alto glasses. When I try the app the visuals seem to follow correctly the glasses, but the audio seem to have a small delay of 500ms regarding the location.
  • Michael@Bose[email protected] Member Posts: 66
    Hi @Bauer The USB Provider function that is offered as part of the Bose AR SDK for Unity does not reproduce 1:1 functionality. There is obviously additional latency caused by 1. data transfer plus audio transfer over wireless bluetooth and 2. the limitations of a mobile application running on a mobile OS on a mobile chipset, versus a (usually) beefy PC or Mac rig with a desktop class graphics card spoofing bluetooth via much faster USB transfer in-editor, not even via a deployed app. 

    Here is an explanatory guide that explains the end to end system latency: https://developer.bose.com/guides/bose-ar/end-end-system-latency

    One thing you can try to reduce latency is setting the data sampling rate to 20ms, but the difference between the end to end system latency at 40ms (196ms - 246ms) vs 20ms (192 - 226ms) isn't very noticeable.

    So, I don't actually recommend testing using the USB Provider but, whenever possible and certainly when testing the user experience of a Bose AR enabled app, testing via a deployed mobile application while connected to the Bose AR device via bluetooth. The USB Provider is only meant for quick spot-testing needs, where time or availability of a mobile device to deploy to is a factor.

    Also regarding your 2nd note about focusing on a visual output based on the sensor data from the Bose AR device: The tagline for Bose AR is "heads up & hands free" (audio first) for just this reason, and you will definitely feel the latency more when staring at a screen that needs to read data from the Bose AR device and make a visual change than you will with an audio-only experience where it's not essentially turning the Bose AR device into a controller of sorts.

    We think focusing on visual output from Bose AR minimizes the effectiveness of the technology and the platform, and the real value is in creating new types of interactive experiences focused on audio and gesture, where your eyes are freed up to navigate the real world, versus staring at a screen - not that there's anything wrong with that, it's just the only method of interaction we really prioritize for mobile apps at the moment.

    There is also just something about how we experience audio versus visual that leads to a much better experience when dealing with latency like this.

    One more thing on latency: 500ms is double the expected end to end system latency, so it might be something related to your admittedly older iPhone with a processor from several years ago parsing the unity build? If you test on a newer iOS device like an iPhone XR or even 8 it might yield better results.

    However, you mention it's "regarding the location" - if you are talking about the GPS location refresh rate, since there is no GPS on Bose AR devices you must be making use of your phone's GPS capabilities, so the issue might lie with either the phone, as mentioned above, or possibly even mapbox - how does mapbox perform when you are testing in-editor? Are you just spoofing location? Perhaps, if you're spoofing location in-editor when testing with USB Provider that's why the location latency might seem so much lower than real world performance on mobile? In any event - some things to explore.

    Hope this helps!

    Michael @ Bose


  • BauerBauer Member Posts: 8
    Hi @[email protected],

    thanks so much for your fast reply!

    Indeed you are right. I have just tried it with an iPhone XR and the audio latency is very nice with it. I had not thought that I would be so demanding, so thanks so much for telling me to try it with an iPhone XR or 8.

    Yet, do you know if there is a way to avoid this problem in unity and say something like "I want to prioritise the audio processing over the graphics" ? Maybe it would also yield better results on the iPhone7.

    To answer your question I was testing the GPS location in-editor with the phone (refreshing the location every 200ms).

    Thanks for the detailed explanation!
    Valentin
Sign In or Register to comment.