ref(QualityController) Add recovery mechanism and adjust the resolution check. (#2546)
* ref(QualityController) Add recovery mechanism and adjust the resolution check.
Impl a recovery mechanism for the lastN to be increased if the cpu limitation goes away and doesn't return after increasing lastN. Also, additionally improve the calculation of the expected resolution taking simulcast stream resolutions into account.
* squash: Address review comments and add more unit tests
* squash: Address review comments
fix(SDP): Negotiate only baseline H.264 codecs for p2p.
Chrome on macOS recently started offering encoder for higher level (5.2) but decoder only for level 3.1. See https://issues.chromium.org/issues/324930413
Therefore, filter out all H.264 payload types with main and high profiles. Also, sort all H.264 payload types so that same pt is picked for both H.264 encoder and decoder. Fixes random black tile issues across different browsers when H.264 is the preferred codec for p2p.
fix(SDP): Generate msid attr for source when browser doesn't produce one.
Chrome 122 does not generate a 'msid' attribute in the local SDP when a local source is added after the user joins muted. The SSRC doesn't get signaled to remote p2p peer when this happens. Also, when the direction of the transceiver changes from 'sendonly' to 'sendrecv', Chrome is generating a new set of SSRCs resulting the client sending out a source-remove followed by source-add. Fixes various p2p related bugs that have surfaced in p2p calls where both the endpoints are running Chromium 122.
fix(video-quality): Adjust encodings bitrates/scalefactor for high capture resolutions (#2436)
* fix(video-quality): Adjust encodings bitrates/scalefactor for high capture resolutions.
When the client is configured to capture camera sources at 1080p or higher, adjust the bitrates and the encodings scale factor accordingly.
* squash: Address review comments
Fix the bitrates set on the remote description and also bump up the bitrates for 1080p, 4k and screenshare.
* squash: Address some more review comments.
* fix(RTC): Remove support for plan-b SDP.
Since unified plan mode becomes the default mode, so does multi-stream mode.
* squash: Remove plan-b unit tests and fix comments
* squash: address review comments
Add SVC and Simulcast support for AV1 and VP9. (#2350)
Add SVC and Simulcast support for AV1 and VP9.
The default mode is the full SVC mode for VP9. It works as expected with the current version of JVB.
AV1 in the client is only supported when the bridge negotiates the Dependency Descriptor extension headers which are needed for reading the layer information for AV1 and H.264 video frames.
* feat: Enable H.264 simulcast support.
Multi-encoding simulcast for H.264 is supported now because of the DD header ext support added on the bridge side.
* fix(video-quality): Add default bitrates for all codecs.
Expect the videoQuality settings in the new format, deprecated configs are still supported.
videoQuality: {
AV1: {
maxBitratesVideo: {
low: 100000,
standard: 300000,
high: 1000000,
ssHigh: 1500000
},
useScalabilityModeAPI: {
enabled: true,
useSimulcast: true, //(defaults to SVC),
useKSVC: true //(defaults to L3T3_KEY)
},
},
H264: {
maxBitratesVideo: {
low: 200000,
standard: 500000,
high: 1500000,
ssHigh: 2500000
}
},
VP8: {
maxBitratesVideo: {
low: 200000,
standard: 500000,
high: 1500000,
ssHigh: 2500000
}
},
VP9: {
maxBitratesVideo: {
low: 100000,
standard: 300000,
high: 1200000,
ssHigh: 2500000
},
useScalabilityModeAPI: {
enabled: true,
useSimulcast: true, //(defaults to SVC),
useKSVC: true //(defaults to L3T3_KEY)
},
}
}
* Disable encodings when requested height for screenshare is 0.
* feat: add unit tests for scalability modes.
* ref: Move all encoding configuration calculation to TPCUtils.
* Add more unit tests for VP9 K-SVC mode.
* Make codec name in settings case insensitive.
Also read the deprecated max bitrates correctly, add unit tests to test it.
* ref(TPC): isSimulcastOn -> isSpatialScalabilityOn.
It makes more sense to call it spatial scalability than simulcast now that full SVC support is available.
* fix: Negotiate AV1 DD header exts only for AV1 and H.264.
* fix: Rename setting useL3T3Mode->useKSVC and adjust the defaults.
fix(SDP): Add msid to generated RTX SSRC only when present on primary SSRC.
Fixes setLocalDescription failing in the case where simulcast is disabled since the answer produced by the browser (after a local source is added) does not have the msid line.
fix(codec-selection) Pick only 1 VP9 codec for p2p negotiation.
More than one VP9 payload type is produced in the offer, more experimental ones if the local endpoint has no local video source. Munge the SDP to remove all codecs besides the one with profile-id=0 so that only one VP9 codec is negotiated with p2p peer.
* feat(signaling) Only include 'msid' in the source description.
'cname', 'label' and 'mslabel' attributes that are included in the source description of the signaling messages are ignored by Jicofo. Only 'msid' is used for source validation.
Therefore, drop those attributes and make sure 'msid' is always included and in the format '<endpoint_id>-<mediaType>-<trackIndex>-<tpcId>'.
* fix(p2p) Enable p2p for Firefox.
Set the senders attribute correctly in the outgoing jingle message. Ignore the media direction from the session, only look for the direction in the media desccription. This fixes an issue where Firefox rejects an answer because the direction is not set correctly.
1. Checks peer's preferred codec in p2p case. Mobile and web have different preferred codecs.
2. Log an error message when the preferred codec is not offered by JVB.
3. Clean up code related to deprecated config.js settings 'preferH264' and 'disableH264'.
4. Refactor the codec selection logic so that correct codec is picked.
Remove support for legacy endpoint based signaling. (#2147)
* Remove support for legacy endpoint based signaling and make source-name signaling as the only signaling mode. The legacy screensharing mode will no longer be supported. The client will still be able to process remote presence sent in the old format to support interop with very old mobile clients and jigasi that do not support source-name signaling.
* remove code related to presenter mode.
Adds a helper function getSourceIndexFromSourceName to
have track index parsing in one place.
In _processLocalSSRCsMap there was an assumption that a track
index can be a single digit only.
Use "last index of" or search from the end of the collection to
make parsing a bit more universal (does not matter as much for
Jitsi endpoint IDs, but if fixes issues if someone has
a customization there).
Reusing(removing it and adding it again) an SSRC in the
remote SDP is leading to the issue where remote video is received but
not rendered. That's why we shouldn't remove the receive only SSRCs
from the SDP. This way the sender won't send any source-remove and
source-add for the same SSRCs.
fix(TPC) Stop munging remote desc for imploding the SIM groups.
Jicofo by default now strips the simulcast SSRCs and sends only the primary SSRC and RTX to all the receiving endpoints in the call. Therefore remote SDP munging is not required on the clients anymore.
fix(SDP): Reset sources to MSID map for plan-b clients.
When source-name signaling is enabbled on plan-b clients, the sources to MSID map needs to be cleared after every transformation since browser in plan-b mode produces a new set of SSRCs and trackID when a track is removed and new track is added to the peerconnection. This fixes an issue where the track index in the source-name gets incremented after a track replace operation.
fix(multi-stream-support) Support muting of desktop track.
* fix(multi-stream-support) Support muting of desktop track.
* fix(multi-stream) Always initiate responder renegotiation even for p2p.
This is needed since the new m-line is always added to the remote description.
* fix(multi-stream) Support multi-stream only on Unfied plan endpoints.
* fix(multi-stream) Advertise source-name signaling to Jicofo when supported.
* fix(JitsiLocalTrack): Remove unnecessary reject.
* fix: add the correct source name attribute for the second video track.
* squash: Address review comments.
feat(multi-stream-support) Add the support for multiple local video streams.
* feat(multi-stream-support) Add the support for multiple local video streams.
This feature is behind 'sendMultipleVideoStreams' config.js flag and is currently supported only on clients running in Unified plan mode.
* squash: Use the track index from source name to find the matching transceiver on unmute.
feat(multi-stream-support) Handle SDP munging for multiple local/remote streams per ep. (#1868)
* feat(multi-stream-support) Handle SDP munging for multiple local/remote streams/ep.
A new SDPSimulcast class is added for handling SDP munging for local and remote descriptions to handle multiple streams for both local and remote endpoints. This new class will be used only for unified plan implentation. The sdp-simucast npm package will be used for plan-b and deprecated later when all the clients switch to unified plan.
* fix(build) fix building with TS code
Use the Babel preset for the webpack bundle since Babel 7 already
understands TS and ts-loader uses tsc instead, which we configure
differently.
* squash: Add unit test and address review comments.
Co-authored-by: Saúl Ibarra Corretgé <saghul@jitsi.org>
This PR switches to generating sctp-port based SDP.
It also adds support for parsing sctp-port, but can handle
fallback to old-style sctpmap as well.
Fixes: #1778
...to advertise track's muted state and the video type.
For now, if the source name signaling flag is enabled, both legacy and
new <SourceInfo> element will be used at the same time. This is to be
able to interoperate with legacy clients and start testing the new
format at the same time. Whenever possible <SourceInfo> will be used
as main source of truth with the fallback to legacy <audiomuted/>,
<videomuted/> and <videotype/> elements.
The issue was caused by incorrect ordering of the mixedmslabel mline.
sdp-interop is expecting that this mline was the first one but in
reality we were placing it last. This was causing the direction of a
user related m line to be sendrecv instead of sendonly. In certain
situation this issue was causing 2 MediaStreamTracks associated with a
single MediaStream/participant.
This is the first step in adding support for multiple
streams per endpoint. Each source(stream?) needs to have
an identifier. For now always generate the 0 index name
until the machinery for sending more than 1 stream is put
in place.
* send only MSID attribute
* add feature flag for source name signaling
* log a msg if source name signaling is enabled