Search Issue Tracker

Won't Fix

Votes

0

Found in

2017.3.0a1

2018.3.0a1

2018.3.8f1

2019.1.0a1

2019.2.0a1

Issue ID

1137011

Regression

No

[Hololens] KeywordRecognizer initialization freezes when the microphone picks up loud sound

XR SDK

-

Steps to reproduce:
1. Download attached project
2. Switch platform to UWP and build project
3. Open Visual Studio solution switch to x86 and Remote machine
4. Build and launch app notice that it takes ~2s to initialize KeywordRecognizer
5. Relaunch the app.
6. Near the HoloLens put the speaker and play music or other loud sounds.
7. After 10s stop playing music. Notice that after stopping the music KeywordRecognizer initialization unfreezes.

Expected results: KeywordRecognizer initialization doesn't react to sound picked up by the microphone
Actual results: KeywordRecognizer initialization freezes when the microphone picks up loud sound

Reproduced with: 2019.2.0a10, 2019.1.0b9, 2018.3.10f1, 2017.4.24f1

Note:
- The issue is reproducible even when microphone capabilities are disabled in the package manifest.
- Not reproducible with build in laptops microphone
- Reproducible on .NET and IL2CPP backends

  1. Resolution Note (2019.3.X):

    This issue is a Microsoft bug that occurs within the HoloLens OS which Unity cannot fix. Please file a bug with Microsoft for this problem.

    The best we can offer is a work around which activates the Windows.Media.SpeechRecognition.SpeechRecognizer COM object on a separate thread allowing Unity's main thread to continue (Update and Render). Once COM activation finally completes, KeywordRecognizer can be instantiated without any delay, since it'll merely adds RefCount to the existing COM object.

    using System.Collections;
    using System.Collections.Generic;
    using UnityEngine;
    using UnityEngine.UI;
    using UnityEngine.Windows.Speech;

    public class testscript : MonoBehaviour {
    bool initialized = false;
    bool directInitComplete = false;

    public Text text;
    // Use this for initialization
    void Start () {
    InitializeWindowsSpeechDirectly();
    }

    // Update is called once per frame
    void Update () {
    if (!initialized && directInitComplete)
    {
    Debug.Log("Creating KeywordRecognizer!");

    float timeStart = Time.realtimeSinceStartup;
    var r = new KeywordRecognizer(new string[] { "test-keyword" }, ConfidenceLevel.Low);
    float totalTime = Time.realtimeSinceStartup - timeStart;
    Debug.Log("new KeywordRecognizer took " + totalTime + "s");

    if (text != null) text.text = "new KeywordRecognizer took " + totalTime + "s";

    r.OnPhraseRecognized += (args) =>
    {
    Debug.Log("Phrase recognized!");
    };

    Debug.Log("Starting KeywordRecognizer!");
    r.Start();

    initialized = true;

    Debug.Log("Finished initialization!");
    }
    }

    #if ENABLE_WINMD_SUPPORT
    Windows.Media.SpeechRecognition.SpeechRecognizer _recognizer;
    #endif

    private void InitializeWindowsSpeechDirectly()
    {
    #if ENABLE_WINMD_SUPPORT

    Windows.System.Threading.ThreadPool.RunAsync((workItem) =>
    {
    // WinRT APIs will throw exceptions
    try
    {
    var recogonizer = new Windows.Media.SpeechRecognition.SpeechRecognizer();

    _recognizer = recogonizer;
    directInitComplete = true;
    }
    catch { };
    });
    #else
    directInitComplete = true;
    #endif

    }
    }

Comments (1)

  1. timke

    Mar 28, 2019 18:28

    Formatting of work around code got screwed up, posting again as a comment

    using System.Collections;
    using System.Collections.Generic;
    using UnityEngine;
    using UnityEngine.UI;
    using UnityEngine.Windows.Speech;

    public class testscript : MonoBehaviour {
    bool initialized = false;
    bool directInitComplete = false;

    public Text text;
    // Use this for initialization
    void Start () {
    InitializeWindowsSpeechDirectly();
    }

    // Update is called once per frame
    void Update () {
    if (!initialized && directInitComplete)
    {
    Debug.Log("Creating KeywordRecognizer!");

    float timeStart = Time.realtimeSinceStartup;
    var r = new KeywordRecognizer(new string[] { "test-keyword" }, ConfidenceLevel.Low);
    float totalTime = Time.realtimeSinceStartup - timeStart;
    Debug.Log("new KeywordRecognizer took " + totalTime + "s");

    if (text != null) text.text = "new KeywordRecognizer took " + totalTime + "s";

    r.OnPhraseRecognized += (args) =>
    {
    Debug.Log("Phrase recognized!");
    };

    Debug.Log("Starting KeywordRecognizer!");
    r.Start();

    initialized = true;

    Debug.Log("Finished initialization!");
    }
    }

    #if ENABLE_WINMD_SUPPORT
    Windows.Media.SpeechRecognition.SpeechRecognizer _recognizer;
    #endif

    private void InitializeWindowsSpeechDirectly()
    {
    #if ENABLE_WINMD_SUPPORT

    Windows.System.Threading.ThreadPool.RunAsync((workItem) =>
    {
    // WinRT APIs will throw exceptions
    try
    {
    var recogonizer = new Windows.Media.SpeechRecognition.SpeechRecognizer();

    _recognizer = recogonizer;
    directInitComplete = true;
    }
    catch { };
    });
    #else
    directInitComplete = true;
    #endif

    }
    }

Add comment

Log in to post comment

All about bugs

View bugs we have successfully reproduced, and vote for the bugs you want to see fixed most urgently.