Fix initial camera quality with Chromium

[danxuliu]

Follow up to #5551

Due to a bug in Chromium it is not possible to increase the resolution of a track once it has been cloned.

When the video was initially requested by Talk no resolution was specified, so Chromium returned a video track with a ~640x480 resolution, no matter the maximum resolution of the camera. As the track was immediately cloned, due to the bug, the maximum resolution of the track was limited to ~640x480.

Now, if Chromium or a derived browser is used, the video is requested with a loose constraint for a high resolution, so if the camera does not have such resolution it will still return the highest resolution available without failing. If the track is then cloned then it will be possible to decrease and increase again the track resolution as desired instead of being locked to a maximum resolution of ~640x480.

Note that this only happens with real video devices; virtual video devices, like those created with v4l2loopback, do not have that issue.

You can verify all that with the following tests:

• Bug:
  • Connect an HD camera
  • Open a private window in Chromium/Edge
  • Open the browser console
  • Run the following lines in the console one by one:

stream = await navigator.mediaDevices.getUserMedia({ video: true })
console.log(stream.getTracks()[0].getSettings()) // The resolution will be 640x480 or 640x360
stream.getTracks()[0].applyConstraints({ width: { exact: 1280 } })
console.log(stream.getTracks()[0].getSettings()) // The resolution will be higher, probably 1280x960 or 1280x720
stream.getTracks()[0].applyConstraints({ width: 640 })
console.log(stream.getTracks()[0].getSettings()) // The resolution will be again 640x480 or 640x360
clonedTrack = stream.getTracks()[0].clone()
stream.getTracks()[0].applyConstraints({ width: { exact: 1280 } }) // This will fail as over constrained

• Workaround:
  • Connect an HD camera
  • Open a private window in Chromium/Edge
  • Open the browser console
  • Run the following lines in the console one by one:

stream = await navigator.mediaDevices.getUserMedia({ video: { width: 1920, height: 1200 })
console.log(stream.getTracks()[0].getSettings()) // The resolution will be higher than 640x480 or 640x360
clonedTrack = stream.getTracks()[0].clone()
stream.getTracks()[0].applyConstraints({ width: 640 })
console.log(stream.getTracks()[0].getSettings()) // The resolution will be 640x480 or 640x360
stream.getTracks()[0].applyConstraints({ width: { exact: 1280 } })
console.log(stream.getTracks()[0].getSettings()) // The resolution will be higher again, probably 1280x960 or 1280x720, even if the track was cloned

Please note that this time I could not test the pull request myself. Also please test with a Chromium derived browser, like Edge, to ensure that the user agent check works in that case too. And if you have two cameras try to switch between them during a call and check that the resolution worked as expected. Thanks :-)

How to test

• Connect an HD or Full HD camera
• Start a conversation in Chromium
• Open the browser console and check the current resolution of the video (it is assumed that there is also audio; if there is no audio use getSenders()[0] instead):
  OCA.Talk.SimpleWebRTC.webrtc.peers[0].pc.getSenders()[1].track.getSettings()
• Try to increase the resolution of the video to a higher but valid resolution:
  OCA.Talk.SimpleWebRTC.webrtc.peers[0].pc.getSenders()[1].track.applyConstraints({ width: { exact: 1280 } }).then(() => { console.log("Success") }).catch(error => { console.log(error) })

Result with this pull request

The resolution of the video will be at least 720x540, and trying to increase it will succeed.

Result without this pull request

The resolution of the video will be 640x480, and trying to increase it will fail.

[danxuliu]

/backport to stable22

[danxuliu]

/backport to stable21

[nickvergessen]

/backport to stable22.1

[PVince81]

👍 code looks fine

[nickvergessen]

Also please test with a Chromium derived browser, like Edge, to ensure that the user agent check works in that case too.

Fixed it

And if you have two cameras try to switch between them during a call and check that the resolution worked as expected.

Switching on Safari doesn't work, also before. But I can not use the back camera anymore, only the front camera. On master I can toggle to the different camera and leave+join to make it work. with this PR I only get a grey video

[backportbot-nextcloud]

The backport to stable21 failed. Please do this backport manually.