Make possible to select input media devices during calls #4023
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
danxuliu
added
2. developing
enhancement
feature: WebRTC 🚡
Member
|
One thing I noticed, when selecting "No device" and then selecting a camera again, the video in the call is still "off" |
Member
Author
I was not sure if the audio or video should be automatically enabled after selecting again a device or not. In the end I kept it disabled because that gives the user time to check in the preview if the device (specially the camera) is the desired one instead of immediately enabling it and sending it to the other participants. |
danxuliu
force-pushed
the
make-possible-to-select-input-media-devices-during-calls
branch
3 times, most recently
from
3756518 to
7e2ac8b
Compare
Adding listeners for MediaStreamTrack objects and dispatching events has no effect in Firefox, although it is supported in Chromium. Due to this a shim that implements events in MediaStreamTrack is added. Note that the implementation is a simplified one adjusted only to what will be needed and it does not take into account advanced parameters of event listeners like "options" or "useCapture". Signed-off-by: Daniel Calviño Sánchez <[email protected]>
According to the specification "ended" should be dispatched when a track ends for any other reason than calling "stop()" on a "MediaStreamTrack". However, dispatching it also in that case makes possible to handle all track ends in a unified way. Signed-off-by: Daniel Calviño Sánchez <[email protected]>
Signed-off-by: Daniel Calviño Sánchez <[email protected]>
Signed-off-by: Daniel Calviño Sánchez <[email protected]>
Adding listeners for MediaStream objects and dispatching events has no effect in Firefox, although it is supported in Chromium. Due to this a shim that implements events in MediaStream is added. Note that the implementation is a simplified one adjusted only to what will be needed and it does not take into account advanced parameters of event listeners like "options" or "useCapture". Signed-off-by: Daniel Calviño Sánchez <[email protected]>
Neither Firefox nor Chromium dispatch "addtrack" and "removetrack" events when tracks are added and removed to and from a MediaStream. However, this is part of the specification, so it is expected that all browsers implement that eventually. Due to that the shim checks whether the events were dispatched or not in order to do it only when needed. Signed-off-by: Daniel Calviño Sánchez <[email protected]>
Until now the audio monitors were stopped when the source stream from which the audio monitor stream was cloned was removed. Now the audio monitors are stopped as soon as all* the audio tracks in the audio monitor stream ends, which will make possible to set an audio monitor again on a previously used stream from which all the audio tracks, but not the video tracks, were removed. *As internally the audio monitor uses a MediaStreamAudioSourceNode actually only the audio track with the lower id will be used. But for simplicity, and given that only a single audio track is expected anyway, all the audio tracks in the audio monitor stream are checked. Signed-off-by: Daniel Calviño Sánchez <[email protected]>
Until now a linked stream stopped its tracks when the original tracks were stopped. Now linked tracks are added and removed too when tracks are added and removed to and from the original stream. This will make possible to keep audio monitor streams in sync with the original ones without explicit handling. Signed-off-by: Daniel Calviño Sánchez <[email protected]>
Signed-off-by: Daniel Calviño Sánchez <[email protected]>
Signed-off-by: Daniel Calviño Sánchez <[email protected]>
When an input device is changed during a call the old audio or video track is stopped, a new one is requested, and then the old track is replaced by the new one in the RTCPeerConnection. When a track is replaced in an RTCPeerConnection sometimes a renegotiation is needed. As neither the web UI nor the mobile apps support a proper renegotiation for now a forced reconnection is used. Signed-off-by: Daniel Calviño Sánchez <[email protected]>
When the selected audio or video device changes a stream from that device is requested to replace the current one. The stream is resolved asynchronously, so while that happens the user could select a different device, which in turn will request another stream. As the previous one would not have been set yet it will not be replaced by the new one, and instead several streams would end being added. Now this is enforced by preventing further stream requests while a previous one has not been completed yet. If several device changes are triggered while waiting for a previous one once that previous one is finished a new stream will be requested for the last selected device. Signed-off-by: Daniel Calviño Sánchez <[email protected]>
Cloning a track was a "silent" operation, so it was not possible to keep track of all the created MediaStreamTracks. Dispatching an event when a track is cloned will make possible to do that. Signed-off-by: Daniel Calviño Sánchez <[email protected]>
Firefox does not support more than a single device of each kind (audio or video) active at the same time. Therefore, when switching to another device all the active tracks of that kind must be stopped before requesting another track from a different device (otherwise a track from the currently active device will be returned, instead of a track from the requested device). The tracks may have been created from different places (for example, the WebRTC code and the device preview component) that do not know about each other. Due to this now MediaDevicesManager keeps track of all the active MediaStreamTrack (even those cloned from another track) and stops the incompatible ones before requesting a new track. It is expected that MediaDevicesManager clients will request a new track on their own when the audio or video input device changes to replace the stopped one. Signed-off-by: Daniel Calviño Sánchez <[email protected]>
nickvergessen
force-pushed
the
make-possible-to-select-input-media-devices-during-calls
branch
from
7e2ac8b to
f3319e5
Compare
Member
|
Rebasing after #4032 |
nickvergessen
deleted the
make-possible-to-select-input-media-devices-during-calls
branch
4 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Requires #4028
Follow up to #3913
Fixes #358
When an input device is changed during a call the old audio or video track is stopped, a new one is requested, and then the old track is replaced by the new one in the RTCPeerConnection. (Surprisingly) In most cases just replacing the track causes the new track to be used (and if the new track is null then the sent media is simply stopped), nothing else is needed. If the new track is not compatible with the old one (for example, if the video uses a different orientation) a renegotiation is needed. Proper renegotiation is not currently supported by the web UI nor the mobile applications, so in this case a reconnection is forced.
Renegotiation is also needed if the call is started without audio or video and then an audio or video input device is selected. In this case there will be no previous track to replace, and adding a new track needs a renegotiation.
The devices are changed during calls in the Preview tab, just like done before calls. The tab is expected to be moved to a settings dialog, but that will be done in another pull request.
The main trouble when changing devices during a call is that Firefox does not support more than a single device of each kind (audio or video) active at the same time. Therefore, when switching to another device all the active tracks of that kind must be stopped before requesting another track from a different device (otherwise a track from the currently active device will be returned, instead of a track from the requested device). Moreover, when an audio and video track are requested at the same time (like done when a call is started) in order to then request an audio or video track from a different device all the audio and video tracks must be stopped first (and not just the tracks from the kind being requested).
Adding some more fun,
MediaStreamTrack.stop()does not fire anendedevent (if the track is stopped for any other reason then it does), and even if it did adding a listener toMediaStreamTrackorMediaStreamdoes not work in Firefox (although it does in Chromium).Pending:
To be addressed if needed in follow up pull requests: