RR报文格式： fraction lost cumulative number of packets lost interarrival jitter extended highest sequence number received：
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
header |V=2|P| RC | PT=RR=201 | length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC of packet sender |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
report | SSRC_1 (SSRC of first source) |
block +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1 | fraction lost | cumulative number of packets lost |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| extended highest sequence number received |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| interarrival jitter |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| last SR (LSR) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| delay since last SR (DLSR) |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
report | SSRC_2 (SSRC of second source) |
block +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
2 : ... :
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| profile-specific extensions |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
丢包计算
- 接收端维护两个计数器，每收到一个RTP包都更新：
-
transmitted，接收到的RTP包的总数；
-
retransmitted，接收到重传RTP包的数量；
2.某时刻收到的有序包的数量Count = transmitted-retransmitte ，当前时刻为Count2，上一时刻为Count1;
3.接收端以一定的频率发送RTCP包（RR、REMB、NACK等）时，会统计两次发送间隔之间(fraction)的接收包信息：
//两次发送间隔之间理论上应该收到的包数量=当前接收到的最大包序号-上个时刻最大有序包序号
uint16_t exp_since_last = (received_seq_max_ - last_report_seq_max_)；//两次发送间隔之间实际接收到有序包的数量=当前时刻收到的有序包的数量-上一个时刻收到的有序包的数量
uint32_t rec_since_last = Count2 - Count1//丢包数=理论上应收的包数-实际收到的包数
int32_t missing = exp_since_last - rec_since_lastmissing即为两次发送间隔之间的丢包数量，会累加并通过RR包通知发送端
- 接收端发送的RR包中包含两个丢包，一个是fraction_lost，是两次统计间隔间的丢包率(以256为基数换算成8bit)，一个是cumulative_lost，是总的累积丢包。
丢包统计代码分析
void StreamStatisticianImpl::UpdateCounters(const RTPHeader& header,
size_t packet_length,
bool retransmitted) {
rtc::CritScope cs(&stream_lock_);
......
//接收到的RTP包的总数
receive_counters_.transmitted.AddPacket(packet_length, header);
//接收到重传RTP包的数量
if (!in_order && retransmitted) {
receive_counters_.retransmitted.AddPacket(packet_length, header);
}
......
// New max.
received_seq_max_ = header.sequenceNumber;
......
// Count only the new packets received. That is, if packets 1, 2, 3, 5, 4, 6
// are received, 4 will be ignored.
if (in_order) {
last_received_timestamp_ = header.timestamp;
last_receive_time_ntp_ = receive_time;
last_receive_time_ms_ = clock_->TimeInMilliseconds();
}
......
}
更新transmitted， retransmitted
RtcpStatistics StreamStatisticianImpl::CalculateRtcpStatistics() {
RtcpStatistics stats;
if (last_report_inorder_packets_ == 0) {
// First time we send a report.
last_report_seq_max_ = received_seq_first_ - 1;
}
//last_report_seq_max_ 上一次发送rr时的seq_max
//received_seq_max_ 当前的seq_max
//exp_since_last 期望能收到多少包
// Calculate fraction lost.
uint16_t exp_since_last = (received_seq_max_ - last_report_seq_max_);
......
// 当前实时刻收到的有序包的数量receive_counters_.transmitted.packets - receive_counters_.retransmitted.packets
// 上一次发送rr时收到包的数量 last_report_old_packets_
// 实际收到包的数量 rec_since_last
// Number of received RTP packets since last report, counts all packets but
// not re-transmissions.
uint32_t rec_since_last =
(receive_counters_.transmitted.packets -
receive_counters_.retransmitted.packets) - last_report_inorder_packets_;
// With NACK we don't know the expected retransmissions during the last
// second. We know how many "old" packets we have received. We just count
// the number of old received to estimate the loss, but it still does not
// guarantee an exact number since we run this based on time triggered by
// sending of an RTP packet. This should have a minimum effect.
// With NACK we don't count old packets as received since they are
// re-transmitted. We use RTT to decide if a packet is re-ordered or
// re-transmitted.
//计算从上一次rr到当前这段时间内，收到的重传包总数
uint32_t retransmitted_packets =
receive_counters_.retransmitted.packets - last_report_old_packets_;
//实际丢包数加上重传丢包数
rec_since_last += retransmitted_packets;
// 计算丢包数：期望收到的包总数exp_since_last - 实际收到的包总数rec_since_last
int32_t missing = 0;
if (exp_since_last > rec_since_last) {
missing = (exp_since_last - rec_since_last);
}
//丢包率 = 255 * 丢包数 / 预期收到的包总数
uint8_t local_fraction_lost = 0;
if (exp_since_last) {
// Scale 0 to 255, where 255 is 100% loss.
local_fraction_lost =
static_cast<uint8_t>(255 * missing / exp_since_last);
}
stats.fraction_lost = local_fraction_lost;
// We need a counter for cumulative loss too.
// TODO(danilchap): Ensure cumulative loss is below maximum value of 2^24.
// 累加丢包总数
cumulative_loss_ += missing;
stats.cumulative_lost = cumulative_loss_;
......
}
抖动计算
jitter定义
如果Si代表第i个包的发送时间戳，Ri代表第i个包的接收时间戳。Sj、Rj同理。
抖动(i, j) = |(Rj - Ri) - (Sj - Si)| = |(Rj - Sj) - (Ri - Si)|
WebRTC为了统一抖动，并且为了很好的降噪、降低突发抖动的影响，把上面的抖动(i, j)定义为D(i, j)，抖动J(i)定义为:
J(i) = J(i-1) + (|D(i-1, i)| - J(i - 1)) / 16
我虽然看不出J(i)和D(i)的关系，但是D(i-1, j)是唯一引起J(i)变化的因素，是需要重点关注的。
代码分析
void StreamStatisticianImpl::UpdateCounters(const RTPHeader& header,
size_t packet_length,
bool retransmitted) {
if (in_order) {
......
// If new time stamp and more than one in-order packet received, calculate
// new jitter statistics.
if (header.timestamp != last_received_timestamp_ &&
(receive_counters_.transmitted.packets -
receive_counters_.retransmitted.packets) > 1) {
//更新Jitter
UpdateJitter(header, receive_time);
}
......
}
......
}
更新jitter
void StreamStatisticianImpl::UpdateJitter(const RTPHeader& header,
NtpTime receive_time) {
//receive_time_rtp对应Rj last_receive_time_rtp对应Ri
//header.timestamp对应Sj last_received_timestamp_对应Si
//`抖动(i, j)` = `|(Rj - Ri) - (Sj - Si)|`
uint32_t receive_time_rtp =
NtpToRtp(receive_time, header.payload_type_frequency);
uint32_t last_receive_time_rtp =
NtpToRtp(last_receive_time_ntp_, header.payload_type_frequency);
int32_t time_diff_samples = (receive_time_rtp - last_receive_time_rtp) -
(header.timestamp - last_received_timestamp_);
time_diff_samples = std::abs(time_diff_samples);
// lib_jingle sometimes deliver crazy jumps in TS for the same stream.
// If this happens, don't update jitter value. Use 5 secs video frequency
// as the threshold.
if (time_diff_samples < 450000) {
// Note we calculate in Q4 to avoid using float.
J(i) = J(i-1) + (|D(i-1, i)| - J(i - 1)) / 16
int32_t jitter_diff_q4 = (time_diff_samples << 4) - jitter_q4_;
jitter_q4_ += ((jitter_diff_q4 + 8) >> 4);
}
......
}
jitter更新计算
扩展jitter计算
上面jitter计算存在的问题：每一帧的视频数据放进多个RTP包之后，这些RTP包的头部timestamp字段都是一样的（都是帧的capture time），但是实际发送时间不一样，到达时间也不同。
3) 如何正确计算抖动：
计算D(i, j)时，Si不能只使用RTP timestamp，而是应该使用该RTP实际发送到网络的时间戳。这种抖动被命名为jitter_q4_transmission_time_offset，意为考虑了transmission_time_offset的jitter。
- a. transmission_time_offset是什么?
transmission_time_offset是一段时间间隔，该时间间隔代表属于同一帧的RTP的
实际发送时间距离帧的capture time的 偏移量 。下图是对transmission_offset_time的解释：
其中，箭头代表一个RTP，发送端的竖线代表时间轴，虚线代表帧的capture time。
最开始三个RTP包在距离capture time
offset1时间之后发送到网络，因此这三个RTP包的transmission_time_offset应该是offset1。同理第四个RTP包的transmission_time_offset应该是offset2，第五个RTP包的transmission_time_offset应该是offset3。
- b. transmission_time_offset在RTP包的哪里放着?
transmission_time_offset存在于RTP的扩展头部，设置该扩展头可以参考RTPSender::SendToNetwork函数，但使用之前该扩展头之前需要注册，否则在设置transmission_time_offset扩展头会失败。
下面的代码段是WebRTC中D(i, j)的计算：
void StreamStatisticianImpl::UpdateJitter(const RTPHeader& header,
NtpTime receive_time) {
......
// Extended jitter report, RFC 5450.
// Actual network jitter, excluding the source-introduced jitter.
int32_t time_diff_samples_ext =
(receive_time_rtp - last_receive_time_rtp) -
((header.timestamp +
header.extension.transmissionTimeOffset) -
(last_received_timestamp_ +
last_received_transmission_time_offset_));
time_diff_samples_ext = std::abs(time_diff_samples_ext);
if (time_diff_samples_ext < 450000) {
int32_t jitter_diffQ4TransmissionTimeOffset =
(time_diff_samples_ext << 4) - jitter_q4_transmission_time_offset_;
jitter_q4_transmission_time_offset_ +=
((jitter_diffQ4TransmissionTimeOffset + 8) >> 4);
}
其中：
receive_time_rtp代表当前RTP的到达时间戳；last_receive_time_rtp是上一个RTP到达时记录的时间戳；header.timestamp + header.extension.transmissionTimeOffset前者是capture time，后者是对应的transmission time offset，两者相加代表该RTP实际发送到网络的时间戳；last_received_timestamp_ + last_received_transmission_time_offset_含义同上，但是代表的是上一个RTP的实际发送到网络的时间戳；
References
https://www.dazhuanlan.com/2020/01/20/5e253c28c1d67/?cf_chl_jschl_tk=a03572ddaaf7860d9c14cc0c8b7c2c112960d0c2-1602660124-0-AbYBnlAUDWglFW5OROTlf3OkRMVHkPSyiXg5Z8Hxcxw86GiwKJsWfxkvRcQVfBinIGMAQuge574_IG2twwif2YVF1D_6bLL0r6xjX84-TmAG0ZAt9ynEBpHmLX4ZBjkfUlB7IcTVowzkt0WTsgKReUrejuTpROkhxZ2glmy8ks1jLkIEyj8_EovtJwafgL0CNFjt9Q-6nfAY8YiwPjmad5kXBx1zOmX_5kMlg8u1AGxYXo_cb1NcO7_stPvjiBHbG7Iz4YJWQ1HzlKcrLut760Mq5OC1jdrgy6Lyr-LTV_1bMNHvSsEPxVqaT-cQkS4Qzg