recognizeMicrophone causes angular (6) change detection to stop working

[jackrvaughan]

Not 100% sure why this happens, but whenever data returned from STT is used to create a new DOM element or component, change detection on those components don't seem to work.

For example:

this.sttStream.on('data', function (data) {
        here.sttResults = data.results
        here.changeDetectorRef.detectChanges()
})

• Expected behavior

We'd expect angular change detection to work normally and not have to call it explicitly.

• Actual behavior

If a UI element is bound to sttResults, it won't update unless changeDetectorRef.detectChanges() is called. It will also cause issues down the line - if you have a click function on an element that was created with STT results data, it won't fire any change detection unless you specifically tell angular there are changes (changeDetectorRef.detectChanges() or appRef.tick()).

For anyone else running into this issue, here is a post on manually triggering change detection: https://stackoverflow.com/questions/34827334/triggering-change-detection-manually-in-angular

• Steps to reproduce

Maybe we're not using the STT recognizeMicrophone() in the correct way; in that case, maybe some examples for angular usage would be helpful. We call a function to kick off the STT:

  toggleMicrophone() {
    const here = this
    // Start the microphone if it is stopped
    if (this.microphoneState === 'stopped') {
      this.microphoneState = 'started'
      this.sttStream = WatsonSpeech.SpeechToText.recognizeMicrophone({
        token: this.sttToken,
        objectMode: true,
        format: true,
        resultsBySpeaker: true
      })

      this.sttStream.on('data', function (data) {
        here.sttResults = data.results
        // here.changeDetectorRef.detectChanges()
      })

    } else {
      this.sttStream.stop()
      this.microphoneState = 'stopped'
    }
  }

We pass the data to a new component to display (which doesn't update):

<app-transcript [sttData]="sttResults"></app-transcript>

• Browser version(s)

Chrome Version 68.0.3440.106 (Official Build) (64-bit)

• SDK version

"watson-speech": "^0.34.2"

Including colleague: @kylebabs

[dpopp07]

Hi @jackrvaughan I've been looking at your problem a bit. I have no experience with angular, but I was reading up on change detection and this detail stuck out to me:

By default, Angular does not do deep object comparison to detect changes, it only takes into account properties used by the template

The results property you're pulling out of data is an array of objects. It sounds to me that, unless you are using specific nested properties of these objects in a template, change detection won't be triggered. Perhaps you are and this isn't the problem, or perhaps I'm totally misinterpreting this :)

Let me know what you think about that - feel free to explain further if you think I'm missing something.

[jackrvaughan]

Hey @dpopp07 - thanks for the reply!

Good find - but unfortunately don't think it's the issue I'm hitting. We are using the results in our component templates to display the transcript among other things.

[yLeushun]

Hi,
Faced with the same issue.. it looks like, Angular does not trigger Change Detection on streams, I took NgZone as a solution, but you may google another (e.g. https://stackoverflow.com/questions/34827334/triggering-change-detection-manually-in-angular)

Here you can find Angular5 example https://github.com/stevengregory/angular-watson-voice
Below is my code changes

import * as recognizeMicrophone from 'watson-speech/speech-to-text/recognize-microphone';
@Component(...)
export class WatsonAssistantComponent {
...
 constructor(public watsonAssistantService: WatsonAssistantService, private ngZone: NgZone) { }
// speech to text
  startStream(): void {
  this.watsonAssistantService.getSpeechToTextToken().subscribe(token => {

    this.stream = recognizeMicrophone(
      {  ...token, format: true, extractResults: true, objectMode: true }
    );


    this.ngZone.runOutsideAngular(() => {
      this.isStreaming = true;
   
      this.stream.on('data', data => {
        this.ngZone.run(() => {
          this.userInput = data.alternatives[0].transcript;
        });
      });

      this.stream.on('error', err => {
        this.ngZone.run(() => {
          this.isStreaming = false;
          console.error(err);
        });
      });

    });
  });
}

Tested on Angular5 and Angular7

[germanattanasio]

I think the workaround from @ylecleach should be enough to close this issue.
Please re open if this is not the case and thanks @ylecleach for the snippet. Feel free to write a PR with an example or an update to the README with this snippet if you want.