Skip to content

Conversation

@danxuliu
Copy link
Member

@danxuliu danxuliu commented Feb 21, 2022

Fixes #1815
Requires #2600 (so the first commit of this pull request is Add and remove nick provider listeners based on the peer connections)
Needed for nextcloud/spreed#6896

Once a RTCPeerConnection is established new offers can still be received for that connection. These offers will update/renegotiate the connection (for example, to add a video track to an audio only connection) and need to be handled like the offers to establish the original connection (the offer needs to be set as the remote description and a local answer needs to be created for it, set as the local description and sent to the other peer).

Pending:

  • Start a renegotiation only if the same sid as the one from the current connection is provided in the offer
    • Older external signaling server (< 0.5.0) and apps may not provide a sid parameter at all; in that case a renegotiation would be started as well, as without sid there is no way to differentiate between connections (although renegotiations would not be expected anyway if using those older versions)
  • Reset connections if an offer with a different sid is received (as done in the WebUI; this is actually a bug in the Android app)
  • Ensure that everything works as expected also for previous Talk versions both with and without HPB (it should, but... you know :-) )

How to test

  • Setup Use renegotiations to update publisher connections spreed#6896 (note that a specific pull request for the external signaling server is also needed)
  • In the browser, start a call and, in the device checker shown before actually starting the call, select an audio device and no video device
  • In the Android app, join the call
  • In the browser, open the device settings, select a video device and close the device settings
  • Enable video

Result with this pull request

In the Android app the video is now visible (note that, due to a bug in the WebUI, the video may be visible even before being actually enabled in the browser)

Result without this pull request

In the Android app the video is not visible

@danxuliu danxuliu added bug Something isn't working 2. developing Work in progress feature: call ☎️ labels Feb 21, 2022
@danxuliu danxuliu force-pushed the fix-handling-offers-in-already-established-connections branch from f952442 to f3049e6 Compare April 22, 2022 19:04
@nextcloud-android-bot
Copy link
Collaborator

Lint

TypemasterPR
Warnings156156
Errors11

SpotBugs (new)

Warning Type Number
Bad practice Warnings 9
Correctness Warnings 91
Experimental Warnings 2
Internationalization Warnings 9
Malicious code vulnerability Warnings 115
Performance Warnings 27
Security Warnings 2
Dodgy code Warnings 158
Total 413

SpotBugs (master)

Warning Type Number
Bad practice Warnings 9
Correctness Warnings 91
Experimental Warnings 2
Internationalization Warnings 9
Malicious code vulnerability Warnings 115
Performance Warnings 27
Security Warnings 2
Dodgy code Warnings 158
Total 413

@nextcloud-command nextcloud-command force-pushed the fix-handling-offers-in-already-established-connections branch from f3049e6 to 98bbae6 Compare May 29, 2022 12:28
@nextcloud-command nextcloud-command force-pushed the fix-handling-offers-in-already-established-connections branch from 98bbae6 to 9ab11fb Compare July 28, 2022 22:45
@AndyScherzinger
Copy link
Member

/rebase

@nextcloud-command nextcloud-command force-pushed the fix-handling-offers-in-already-established-connections branch from 9ab11fb to b8304f9 Compare October 11, 2022 22:23
@danxuliu danxuliu force-pushed the fix-handling-offers-in-already-established-connections branch 2 times, most recently from 7cc5ec0 to f32cd44 Compare December 4, 2022 12:22
Although the OfferNickProvider itself is still added and removed based
on the call participant the listeners for the video and screen
connections are now added and removed based on the actual peer
connections that they listen to.

Signed-off-by: Daniel Calviño Sánchez <[email protected]>
Signaling messages related to peer connections include a "sid" parameter
to be able to identify the connection. At any given time it is expected
that there will be a single peer connection for certain session and
type, and the sid is used to verify that both ends are using the same
peer connection.

Signaling messages with a different sid can be ignored, unless they are
offers; if an offer is received for an existing peer connection with a
different sid that means that the existing peer connection is stale, so
it should be removed and a new one should be created instead. This can
happen, for example, if a new offer is requested to the HPB (which
creates a new peer connection in the HPB, so the existing one can no
longer be used).

On the other hand, if an offer with the same sid as the existing peer
connection is received that means that the offer is a renegotiation, and
it should be handled by the existing peer connection.

Note that a signaling message may not contain a "sid" parameter if an
older HPB is being used (external signaling server < 0.5.0), or when the
other participant is using an older app version. In that case it is not
possible to know if a signaling message for certain session and type
actually refers to the existing connection or a different one, so it is
always handled as if it was for the existing connection (which was the
behaviour until now).

Signed-off-by: Daniel Calviño Sánchez <[email protected]>
Once a RTCPeerConnection is established new offers can still be received
for that connection. These offers will update/renegotiate the connection
(for example, to add a video track to an audio only connection) and need
to be handled like the offers to establish the original connection (the
offer needs to be set as the remote description and a local answer needs
to be created for it, set as the local description and sent to the other
peer).

In the PeerConnectionWrapper the same SdpObserver object is called when
setting both local and remote descriptions, so the answer should be
created only when a remote description was set. Before the answer was
created if there was no local description already, so this only worked
when establishing the initial connection. Once the connection is
established new answers need to replace the current local description,
so now the creation of new answers is based on the signaling state
instead.

Signed-off-by: Daniel Calviño Sánchez <[email protected]>
@danxuliu danxuliu force-pushed the fix-handling-offers-in-already-established-connections branch from 53dd98c to 1b5be43 Compare January 16, 2023 10:36
@github-actions
Copy link
Contributor

Codacy

Lint

TypemasterPR
Warnings111111
Errors00

SpotBugs

CategoryBaseNew
Correctness1212
Dodgy code174174
Internationalization55
Malicious code vulnerability33
Performance1011
Security22
Total206207

SpotBugs increased!

@github-actions
Copy link
Contributor

APK file: https://www.kaminsky.me/nc-dev/android-artifacts/1823-talk.apk

qrcode

To test this change/fix you can simply download above APK file and install and test it in parallel to your existing Nextcloud Talk app.

@mahibi mahibi self-requested a review January 16, 2023 13:52
@nickvergessen
Copy link
Member

Whats the status?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

2. developing Work in progress bug Something isn't working feature: call ☎️

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🔁 Adjustments for renegotiations

5 participants