fix(codec) Always use DD when its supported by Jicofo and browser.
The bridge is now able to use DD headers for VP8 and VP9 as well so there is no need to remove it from remote desc when the call switches from AV1 to VP8/VP9.
Disables VP9 for FF since the simulcast is disabled with FF flag by default. Introduces a few fixes so that if the FF flag is enabled simulcast works. This should also help us to be ready to enable VP9 once the FF config flag for simulcast for VP9 is enabled by default.
It doesn't get translated in the TS build, for one.
Script I used:
```python
import os
for (dirpath, dirnames, filenames) in os.walk('.'):
if '.git' in dirpath:
continue
if 'node_modules' in dirpath:
continue
if 'dist' in dirpath:
continue
if 'types' in dirpath:
continue
for filename in filenames:
path = os.path.join(dirpath, filename)
if not path.endswith('.js') and not path.endswith('.ts'):
continue
#print(path)
with open(path, 'r+') as f:
#print(f)
data = f.read()
if '__filename' in data:
p, ext = os.path.splitext(path)
txt = f"'{p[2:]}'"
print(txt)
data = data.replace('__filename', txt) # Assign the result back to data
f.seek(0)
f.write(data)
f.truncate()
```
fix(codec) Debounce the call that calc codec intersection set. (#2622)
* fix(codec) Debounce the call that calc codec intersection set.
Calculate codec intersection set only once per second even when there is a burst of joins/leaves. Also, check for current codec before chaining a codec change operation when codec selection API is used.
* squash: Address review comments
fix(video-quality) Fixes an issue where outbound resolution can be stuck at wrong resolution.
The calls to RTCRtpSender.setParameters() are all chained and the current maxHeight is set after the call to setParameters is resolved. If there is another call made to setParameters before the previous one resolves, we can end up passing the wrong maxHeight resulting in the client getting stuck at an unexpected resolution. This issue can be reproduced sometimes when the users are moving across the main and breakout rooms. TPC.setVideoCodecs() ends up pushing a wrong maxHeight for update when the previous call to setParameters hasn't resolved yet.
fix(TPC) Use videoType from 'source-add' for remote track creation. (#2596)
* fix(TPC) Use videoType from 'source-add' for remote track creation.
If 'source-add' for a remote video source is received before presence for that source, videoType will default to 'camera' and the client wouldn't be able to create the virtual participant tile for rendering the desktop track.
* squash: Include the videoType for no SSRC-rewriting case.
* ref(SDPDiffer) Convert to ES6 class.
Make it work directly with unified plan SDP that has multiple m-lines and add more unit tests.
* ref(xmpp) translate unified-plan SDP->Jingle directly.
Without having to run it through the SDPInterop.toPlanB cycle.
* ref(SDP) Always generate the MSID for signaling it to Jicofo.
* fix(SDPDiffer) Check explicitly for ssrc changes
* fix(SDP): Fix comments and cleanup.
Remove LOCAL_TRACK_SSRC_UPDATED event as the application ignores the event and no additional action needs to be taken when that event is fired.
* ref(SDP) Add a helper function for parsing the 'a=ssrc-group' line.
* squash: Address review comments
fix(quality) Do not set b:AS line in SDP for SVC codecs when codec selection API is used.
This was needed in older versions since the browser didn't apply maxBitrates from RTCRtpEncoderParameters on the encoder. In the newer versions this seems to be no longer the case. Also, when the codec selection API is used, we no longer renegotiate locally so if we switched codec from AV1->VP9-VP9, the AV1 bitrate setting in the SDP will still be effective resulting in a lower send resolution because of b/w limitation.
ref(QualityController) Add recovery mechanism and adjust the resolution check. (#2546)
* ref(QualityController) Add recovery mechanism and adjust the resolution check.
Impl a recovery mechanism for the lastN to be increased if the cpu limitation goes away and doesn't return after increasing lastN. Also, additionally improve the calculation of the expected resolution taking simulcast stream resolutions into account.
* squash: Address review comments and add more unit tests
* squash: Address review comments
feat(quality) Add a QualityController class for runtime adjustments. (#2542)
* feat(quality) Add a QualityController class for runtime adjustments.
Make run time adjustments to the client when adaptive mode is enabled.
* feat: Update lastN and receive resolution to improve quality.
* squash: Address review comments
* squash: Add more logging and address review comments.
feat(codec-selection): Use the new codec selection API (#2520)
* feat(codec-selection): Use the new codec selection API
https://github.com/w3ctag/design-reviews/issues/836. This allows the client to seamlessly switch between the codecs without having to trigger a renegotiation.
This feature is behind the flag testing.enableCodecSelectionAPI in config.js
* fix(stats): Fix local resolution stats.
The video codec for the local video sources needs to identified differently now, from the codecs field in the RTCRtpSendParameters returned by the browser. We no longer munge the remote SDP to switch to a different codec.
* feat(stats): Feed encodeTime stats for all local SSRCs to the codec selection mechanism.
* fix(codec-selection) Continue to mumge SDP for selecting H.264.
* feat(codec-selection) Make screenshare codec configurable.
If no 'screenshareCodec' is set under videoQuality or p2p settings, AV1 will be selected as default.
* squash: Address review comments
* Update modules/RTC/CodecSelection.js
Co-authored-by: Saúl Ibarra Corretgé <s@saghul.net>
* fix(codec-selection) Add codec to existing stats
---------
Co-authored-by: Saúl Ibarra Corretgé <s@saghul.net>
fix(ssrc-rewriting): Fire track removed/added instead of owner changed.
When an existing SSRC for a remote track gets re-mapped from one source to another, fire a TRACK_REMOVED event followed by TRACK_ADDED event instead of TRACK_OWNER_CHANGED event. This should simplify the application logic for track handling.
fix(SDP): Negotiate only baseline H.264 codecs for p2p.
Chrome on macOS recently started offering encoder for higher level (5.2) but decoder only for level 3.1. See https://issues.chromium.org/issues/324930413
Therefore, filter out all H.264 payload types with main and high profiles. Also, sort all H.264 payload types so that same pt is picked for both H.264 encoder and decoder. Fixes random black tile issues across different browsers when H.264 is the preferred codec for p2p.
fix(TPC): Run the DD ext header check on all browsers.
We do not want non Chromium browsers to negotiate DD ext headers when VP8/VP9 is the selected codec. Fixes poor video quality issue for Safari when Av1 is offered by Jicofo.
Set the scalabilityMode to undefined for VP8 after a VP9->VP8 switch even though the browser returns L1T2 in the RTCRtpSender.getParameters() call. Fixes an issue where the bridge stops forwarding video for some participants after vp8->vp9 switch.
If MediaStreamTrack.getSettings() doesn't return the height of the track, use the constraints passed to gUM for calculating the height. Fixes an issue for Firefox/Linux which could result in media not displayed.
fix(video-quality): Adjust encodings bitrates/scalefactor for high capture resolutions (#2436)
* fix(video-quality): Adjust encodings bitrates/scalefactor for high capture resolutions.
When the client is configured to capture camera sources at 1080p or higher, adjust the bitrates and the encodings scale factor accordingly.
* squash: Address review comments
Fix the bitrates set on the remote description and also bump up the bitrates for 1080p, 4k and screenshare.
* squash: Address some more review comments.
fix(video-quality): Remove the encoding check for webkit.
This check is not needed anymore since we now set the correct scaleResolutionDownBy value everytime the encodings are configured. Fixes a regression on Safari where it can end up sending low resolution to a p2p peer in some cases. Also, add more unit tests for p2p desktop share cases.
* fix(RTC): Remove support for plan-b SDP.
Since unified plan mode becomes the default mode, so does multi-stream mode.
* squash: Remove plan-b unit tests and fix comments
* squash: address review comments