When source-add and source-remove for the same SSRC arrive just one after another, SSRC will not be removed as it didn't make it to the remote description yet.