Encoder/decoder issues were noticed with AV1 for Safari. This includes no scalability for the stream that is send, not decoded videos on the receiver side, etc. This should be disable until all issues are resolved.
Disables AV1 for FF since AV1 is not properly supported (it doesn't support SVC and it doesn't send temporal layers with simulcast).
Added a testing config flag so that we can easily enable it and test it.
It doesn't get translated in the TS build, for one.
Script I used:
```python
import os
for (dirpath, dirnames, filenames) in os.walk('.'):
if '.git' in dirpath:
continue
if 'node_modules' in dirpath:
continue
if 'dist' in dirpath:
continue
if 'types' in dirpath:
continue
for filename in filenames:
path = os.path.join(dirpath, filename)
if not path.endswith('.js') and not path.endswith('.ts'):
continue
#print(path)
with open(path, 'r+') as f:
#print(f)
data = f.read()
if '__filename' in data:
p, ext = os.path.splitext(path)
txt = f"'{p[2:]}'"
print(txt)
data = data.replace('__filename', txt) # Assign the result back to data
f.seek(0)
f.write(data)
f.truncate()
```
ref(QualityController) Add recovery mechanism and adjust the resolution check. (#2546)
* ref(QualityController) Add recovery mechanism and adjust the resolution check.
Impl a recovery mechanism for the lastN to be increased if the cpu limitation goes away and doesn't return after increasing lastN. Also, additionally improve the calculation of the expected resolution taking simulcast stream resolutions into account.
* squash: Address review comments and add more unit tests
* squash: Address review comments
feat(quality) Add a QualityController class for runtime adjustments. (#2542)
* feat(quality) Add a QualityController class for runtime adjustments.
Make run time adjustments to the client when adaptive mode is enabled.
* feat: Update lastN and receive resolution to improve quality.
* squash: Address review comments
* squash: Add more logging and address review comments.
feat(codec-selection): Use the new codec selection API (#2520)
* feat(codec-selection): Use the new codec selection API
https://github.com/w3ctag/design-reviews/issues/836. This allows the client to seamlessly switch between the codecs without having to trigger a renegotiation.
This feature is behind the flag testing.enableCodecSelectionAPI in config.js
* fix(stats): Fix local resolution stats.
The video codec for the local video sources needs to identified differently now, from the codecs field in the RTCRtpSendParameters returned by the browser. We no longer munge the remote SDP to switch to a different codec.
* feat(stats): Feed encodeTime stats for all local SSRCs to the codec selection mechanism.
* fix(codec-selection) Continue to mumge SDP for selecting H.264.
* feat(codec-selection) Make screenshare codec configurable.
If no 'screenshareCodec' is set under videoQuality or p2p settings, AV1 will be selected as default.
* squash: Address review comments
* Update modules/RTC/CodecSelection.js
Co-authored-by: Saúl Ibarra Corretgé <s@saghul.net>
* fix(codec-selection) Add codec to existing stats
---------
Co-authored-by: Saúl Ibarra Corretgé <s@saghul.net>
fix(codec-selection): put Av1 support behind a flag.
Fixes an issue with audio after p2p->jvb switch when the media active state is set to false because of a setVideoCodecs call after a remote user joins the call.
fix(codec-selection): Select VP8 as default codec.
When RTCRtpReceiver.getCapabilities() is not supported by browser or if it returns an empty list. Fixes an issue where call doesn't get established in Firefox ESR 102.
Fixes https://github.com/jitsi/jitsi-meet/issues/13839.
Add SVC and Simulcast support for AV1 and VP9. (#2350)
Add SVC and Simulcast support for AV1 and VP9.
The default mode is the full SVC mode for VP9. It works as expected with the current version of JVB.
AV1 in the client is only supported when the bridge negotiates the Dependency Descriptor extension headers which are needed for reading the layer information for AV1 and H.264 video frames.
* feat: Enable H.264 simulcast support.
Multi-encoding simulcast for H.264 is supported now because of the DD header ext support added on the bridge side.
* fix(video-quality): Add default bitrates for all codecs.
Expect the videoQuality settings in the new format, deprecated configs are still supported.
videoQuality: {
AV1: {
maxBitratesVideo: {
low: 100000,
standard: 300000,
high: 1000000,
ssHigh: 1500000
},
useScalabilityModeAPI: {
enabled: true,
useSimulcast: true, //(defaults to SVC),
useKSVC: true //(defaults to L3T3_KEY)
},
},
H264: {
maxBitratesVideo: {
low: 200000,
standard: 500000,
high: 1500000,
ssHigh: 2500000
}
},
VP8: {
maxBitratesVideo: {
low: 200000,
standard: 500000,
high: 1500000,
ssHigh: 2500000
}
},
VP9: {
maxBitratesVideo: {
low: 100000,
standard: 300000,
high: 1200000,
ssHigh: 2500000
},
useScalabilityModeAPI: {
enabled: true,
useSimulcast: true, //(defaults to SVC),
useKSVC: true //(defaults to L3T3_KEY)
},
}
}
* Disable encodings when requested height for screenshare is 0.
* feat: add unit tests for scalability modes.
* ref: Move all encoding configuration calculation to TPCUtils.
* Add more unit tests for VP9 K-SVC mode.
* Make codec name in settings case insensitive.
Also read the deprecated max bitrates correctly, add unit tests to test it.
* ref(TPC): isSimulcastOn -> isSpatialScalabilityOn.
It makes more sense to call it spatial scalability than simulcast now that full SVC support is available.
* fix: Negotiate AV1 DD header exts only for AV1 and H.264.
* fix: Rename setting useL3T3Mode->useKSVC and adjust the defaults.
fix(codec-selection): Disabled VP9 for p2p on Firefox.
When VP9 is at the top of the codec list in the remote offer, Firefox omits it in the answer but decides to use VP9 for encoding media. Therefore the remote peer gets an answer with no VP9 in the list and when it updates its local description (with no VP9) after a renegotiation, it stops rendering the VP9 video received from Firefox.
* fix(codec-selection) Use a configurable preferred codecs list to select the codecs.
Allow asymmetric codecs to be configured on the endpoints. This means that Firefox and Safari which have bugs with VP9 encode will now encode VP8 but decode VP9 coming in from Chromium endpoints.
* fix(codec-selection) Add unit tests for the codec selection logic.
* feat(codec-selection): Introduce mobileCodecPreferenceOrder setting in config.js.
* fix(codec-selection) Avoid unnecessary renegotiations.
On participant join/leave, check if the new intersection of codecs are already configured to be the top n codecs.
* Address review comments
* fix: Strip the codecs that are not in the codec list from the video m-lines.
Also, ignore remote codecs published in presence for p2p connections.
* fix: Define default codec order for mobile and desktop.
Fix an issue where p2p between mobile and desktop was broken.
* fix: Add default codecs for both p2p and jvb on mobile devices.
Munge the initial offer sent out by RN clients since RN doesn't support RTCRtpTransceiver#setCodecPreferences.
fix(codec-selection) Apply codec preferences to initial offer/answer.
This fixes an issue where p2p clients (with different codec preferences) fail to decode video because the negotiated codecs are removed from the supported codecs list after the media session is established. The codec preferences will be applied when the first offer/answer is created.
fix(codec-selection) Check if disabled is already removed from sdp.
This should avoid unnecessary renegotiations if disabledCodec setting is present in config.js and should improve time to receive media when joining big calls.
1. Checks peer's preferred codec in p2p case. Mobile and web have different preferred codecs.
2. Log an error message when the preferred codec is not offered by JVB.
3. Clean up code related to deprecated config.js settings 'preferH264' and 'disableH264'.
4. Refactor the codec selection logic so that correct codec is picked.
fix(codec-selection): Fix VP9 codec switching issue in Chrome unified-plan.
Munge only the m-line that corresponds to the source that the browser will be sending.
Do not select VP9 on Firefox.
Detect support for RTCRtpTransceiver#setCodecPreferences correctly.
Determine the preferred codec for a given endpoint based on the config.js settings and the codecs supported by the endpoint.
The preferred codec is published in presence and then used by the other endpoints in the call during join/leave to determine
if the codec needs to be changed on the fly. Different codecs can be configuered for p2p/jvb connections.
The preferredCodec/disabledCodec settings under videoQuality will have precedence over the older settins like preferH264/disableH264.