feat(codec-selection): Use the new codec selection API (#2520)
* feat(codec-selection): Use the new codec selection API
https://github.com/w3ctag/design-reviews/issues/836. This allows the client to seamlessly switch between the codecs without having to trigger a renegotiation.
This feature is behind the flag testing.enableCodecSelectionAPI in config.js
* fix(stats): Fix local resolution stats.
The video codec for the local video sources needs to identified differently now, from the codecs field in the RTCRtpSendParameters returned by the browser. We no longer munge the remote SDP to switch to a different codec.
* feat(stats): Feed encodeTime stats for all local SSRCs to the codec selection mechanism.
* fix(codec-selection) Continue to mumge SDP for selecting H.264.
* feat(codec-selection) Make screenshare codec configurable.
If no 'screenshareCodec' is set under videoQuality or p2p settings, AV1 will be selected as default.
* squash: Address review comments
* Update modules/RTC/CodecSelection.js
Co-authored-by: Saúl Ibarra Corretgé <s@saghul.net>
* fix(codec-selection) Add codec to existing stats
---------
Co-authored-by: Saúl Ibarra Corretgé <s@saghul.net>
fix(ssrc-rewriting): Check if track exists before updating owner/sourceName.
Fixes missing audio issue when the bridge resends the full audio map even before the track for the SSRCs signaled before are created. This is possible when the RTP packets race with the thread that completed the websocket connection.
* fix: Fixes double conf requests & always send them after presence error.
* Revert "fix: Sends conference requests on retries of errors."
Seems we are sending two conference requests in the happy path.
This reverts commit 86e2fb2bed.
* fix: Sends conference requests on retries of errors.
This changes the behavior of retires when Room creation restricted error is received, we will always send the conference request to jicofo when retrying.
Also, after error for password protected rooms and providing password, a conference request was not send which can result jicofo left the room and error is received.
fix(ssrc-rewriting): Check for track owner/sourceName before calling TRACK_OWNER_SET.
When the bridge WS is re-established, jvb sends the full map of audio and video sources. Without the check, the library will end up firing TRACK_REMOVED and TRACK_ADDED for all the exiting tracks.
Also, process audio and video source maps only on JVB sessions.
fix(ssrc-rewriting): Fire track removed/added instead of owner changed.
When an existing SSRC for a remote track gets re-mapped from one source to another, fire a TRACK_REMOVED event followed by TRACK_ADDED event instead of TRACK_OWNER_CHANGED event. This should simplify the application logic for track handling.
fix: Sends conference requests on retries of errors.
This changes the behavior of retires when Room creation restricted error is received, we will always send the conference request to jicofo when retrying.
Also, after error for password protected rooms and providing password, a conference request was not send which can result jicofo left the room and error is received.
feat: Introduces connection event for custom properties. (#2493)
* feat: Introduces connection event for custom properties.
Used for shard and region values coming from the backend.
* squash: Update JitsiConnectionEvents.ts
* squash: Update JitsiConnectionEvents.ts
* fix(RTC): Remove support for plan-b SDP.
Since unified plan mode becomes the default mode, so does multi-stream mode.
* squash: Remove plan-b unit tests and fix comments
* squash: address review comments
Unify events and output single TypeScript declaration (#2407)
* fix(events): unify events to a single EventManager type, add support for single typescript declaration
* fix(lint): fix lint
* fix(events): fix incorrect instatiation
* fix(events): clean up redundant methods
* fix(events): keep EventEmitter name, alias NodeEventEmitter
* fix(events): fix loose reference
* fix(EventEmitter): remove on/off alias as redundant
* fix(RTCUtils): bring event handlers under class to use same event emitter
* fix(RTCUtils): fix lint
fix(video-quality) Enable TCC support for Firefox 115 and later.
Firefox ESR 115 stops sending REMBs to the bridge under certain network conditions making the bridge suspend all video streams sent to the Firefox eps. It is still unclear why Firefox stops sending REMBs so enabling TCC support as workaround since TCC doesn't seem to have issues with uplink BWE in the older versions.
When a self presence is received ( from mute or something else) during
leave operation, after presence unavailable is sent and before the final
presence unavaliable is received, we were interpreting it as muc joined
since the joined flag was immediately set to false.
Add SVC and Simulcast support for AV1 and VP9. (#2350)
Add SVC and Simulcast support for AV1 and VP9.
The default mode is the full SVC mode for VP9. It works as expected with the current version of JVB.
AV1 in the client is only supported when the bridge negotiates the Dependency Descriptor extension headers which are needed for reading the layer information for AV1 and H.264 video frames.
* feat: Enable H.264 simulcast support.
Multi-encoding simulcast for H.264 is supported now because of the DD header ext support added on the bridge side.
* fix(video-quality): Add default bitrates for all codecs.
Expect the videoQuality settings in the new format, deprecated configs are still supported.
videoQuality: {
AV1: {
maxBitratesVideo: {
low: 100000,
standard: 300000,
high: 1000000,
ssHigh: 1500000
},
useScalabilityModeAPI: {
enabled: true,
useSimulcast: true, //(defaults to SVC),
useKSVC: true //(defaults to L3T3_KEY)
},
},
H264: {
maxBitratesVideo: {
low: 200000,
standard: 500000,
high: 1500000,
ssHigh: 2500000
}
},
VP8: {
maxBitratesVideo: {
low: 200000,
standard: 500000,
high: 1500000,
ssHigh: 2500000
}
},
VP9: {
maxBitratesVideo: {
low: 100000,
standard: 300000,
high: 1200000,
ssHigh: 2500000
},
useScalabilityModeAPI: {
enabled: true,
useSimulcast: true, //(defaults to SVC),
useKSVC: true //(defaults to L3T3_KEY)
},
}
}
* Disable encodings when requested height for screenshare is 0.
* feat: add unit tests for scalability modes.
* ref: Move all encoding configuration calculation to TPCUtils.
* Add more unit tests for VP9 K-SVC mode.
* Make codec name in settings case insensitive.
Also read the deprecated max bitrates correctly, add unit tests to test it.
* ref(TPC): isSimulcastOn -> isSpatialScalabilityOn.
It makes more sense to call it spatial scalability than simulcast now that full SVC support is available.
* fix: Negotiate AV1 DD header exts only for AV1 and H.264.
* fix: Rename setting useL3T3Mode->useKSVC and adjust the defaults.
fix(JingleSession) Use the video type from presence.
Instead of using the videoType from source map signaled by the bridge which can be incorrect if the bridge doesn't receive the presence before it sends out the source map. It uses the default 'camera' as video type if presence is not received from the senders. Possibly fixes a bug where a screenshare source is displayed in the wrong tile after the SSRC is re-mapped.
Additional benefits are:
- ssrc -> source name will be updated on ssrc remap message from
the bridge.
- the map will be correctly cleaned when member leave (this logic was
not working well for _sourceName map)
- Looks cleaner and simpler.
TCC was disabled on older versions because of a known issue where BWE is halved on every renegotiation. The regular libwebrtc updates in Firefox seems to have fixed it. This hopefully fixes the BWE issues seen on Firefox 117.