Head-tracking latency experienced on BoseAR Unity Advanced demo

Head-tracking latency experienced on BoseAR Unity Advanced demo

bigmousebigmouse Member Posts: 6
Playing with the original BoseAR Unity demo App - Advanced demo, I can feel there is 100 - 200 ms delay between my head rotates to a new position and the position change of the audio source.

I have also tried to minimize the latency by Edit->Project Settings->Audio->Set DSP Buffer Size to be Best latency. But latency is almost not improved.

Question:
Does Bose Frame using Bluetooth 2.1 classic or Bluetooth 4.0? (Sensors and audio)
Wondering if the delay is due to the Bluetooth audio stack?
Is there a way to improve the latency by any settings?
Will Bose Frame support Bluetooth 5.0 audio in the future without the hardware change?

Comments

  • Michael@Bose[email protected] Member Posts: 66
    Hi @bigmouse - thanks for reaching out!

    You are correct that there is at least a 100 - 200 ms delay between your head rotation and the new position change from the audio source. In fact, according to https://developer.bose.com/guides/bose-ar/end-end-system-latency on our site, you can expect end to end system latency, whether using Unity or native iOS to build your app, to be in the following ranges, depending on your preferred sampling period:

    196 - 226ms at a 20ms sampling period (50Hz)

    196 - 246ms at a 40ms sampling period (25Hz)

    When you change the buffer size to "Best Latency," as you've indicated you've done, that only affects Core Audio latency, which unfortunately only nets you single digit latency improvements.

    I would recommend inspecting the architecture diagram as well for a deeper understanding of how the stack works and what parts cause how much latency. 

    We're well aware of how important latency is and the team internally is constantly finding ways to try to improve that, so definitely stay tuned for future updates on both the firmware and SDKs, but this information is accurate as of now.

    In terms of your specific questions, all of the information we have publicly available on this topic is contained in the page I linked to above.

    Hope this helps!

  • axel3dsoundaxel3dsound Member Posts: 4
    Hi all

    There has been quite a bit of discussion about latency on iOS, suggesting that at least an iPhone XR is needed to reduce the latency. I tried Traverse on an iPhone 8 and the latency wasn't that bad. Is there a way to achieve similar results using the Unity SDK and an Android device? Have you tested some Android phones to have an idea of what minimum spec is needed to reduce the latency?

    If I have to build a simple multichannel player app using the Unity SDK, would it be just better to build for iOS directly instead of even trying to build for Android?

    Kind regards
    Axel
  • daniel@bose[email protected] Member Posts: 41
    Hello @axel3dsound,  
    What great question!  In general yes, native apps are closest to the data connection, but surprisingly that's not first place to look for optimization of the experience.   The connection across all sdks is quite fast.  

    As you may know the Unity Editor compiles cross platform code to IOS and Android native code and then the unity engine behind the scenes is running the code.   Our SDK is a plugin, which means it's running about the same code as the native SDKs.  So the latency between the Bose AR-enabled device and the smart phone (iOS or Android) is essentially the same as it would be if it was compiled as a Native app.  

    Some phones have faster processors, but the technical limitation and cap at around 20 ms is more with the Bluetooth connection.   There is a slight bit of translation between methods called between the plugin and the engine, but it's really insignificant.    

    Once the data is captured, what you do with it will also impact how it responds.   For instance if you render a super complex 3D object and try to transform that object with the data from our sensor, it may lag - not due to the sensor data, but due to limited processor or lack of memory or how you've managed an instance.   

    In IOS you have some profiling tools to look at how your app is performing.  You should be able to use those to determine where to optimize.   Likewise, if you've optimized your Unity scenes well for IOS, the Android deployment will also improve.

    Thanks for the question, and let us know what path fits best.

    @[email protected]

  • UmboltUmbolt Member Posts: 1
    Hi all,

    We are conducting some tests with Bose AR and a QC35II. We are particularly interested in developping interactive 360 audio experiences, for which latency is a real issue. 

    We conducted a very stupid test : we used the Bose headset to move around in the 360 scene, but used wired connected standard earphones to listen to the result : we got almost no latency. The result was actually quite predictable. We used the bluetooth connection to send (very light) positional data FROM the headset but we bypassed the bluetooth connection and sent (relatively heavy) audio files TO the earphones using a wire. 

    This leads us to a suggestion. If the ambition of the Bose AR program is to sustain ambitious 360 interactive experiences, isn't there a case for allowing a wired connection to send audio files from the app to the Bose headset ? 

    Cheers,

    @Umbolt
  • edited 11/20/2019 - 14:24
    JonOliveJonOlive Member Posts: 1
    Hello All

    For our purposes the bluetooth latency is essentially a deal breaker - which is a real pity. Allowing a wired connection in a build (rather than just the editor-only function at the moment) would make the Bose product line viable for use in our projects. i completely understand that the latency issue is not of Bose' making - but I think, as @Umbolt says, rather than giving the impression of trying to brush the issue under the carpet - from a developers point of view at least - it would serve Bose well to address the issue directly by giving us hard wired support for applications where the bluetooth latency is a problem which would be most of them I would imagine!
Sign In or Register to comment.