一、远端视频数据处理
1)创建RTP接收线程
RTP接收线程是收到http承载的sdp命令中的m行,根据m行的信息创建的。
Received session description :{
"sdp" : "v=0\r\no=- 3138862463696765007 2 IN IP4 127.0.0.1\r\n
s=-\r\nt=0 0\r\n
a=group:BUNDLE audio video\r\n
a=msid-semantic: WMS stream_label\r\n
m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 102 0 8 106 105 13 110 112 113 126\r\n
c=IN IP4 0.0.0.0\r\n
a=rtcp:9 IN IP4 0.0.0.0\r\n
a=ice-ufrag:xxx\r\n
a=ice-pwd:xxxx\r\n
a=ice-options:trickle\r\n
a=fingerprint:sha-256 xxxx\r\n
a=setup:active\r\n
a=mid:audio\r\n
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\n
a=sendrecv\r\n
a=rtcp-mux\r\n
a=rtpmap:111 opus/48000/2\r\n
a=rtcp-fb:111 transport-cc\r\n
a=fmtp:111 minptime=10;useinbandfec=1\r\n
a=rtpmap:103 ISAC/16000\r\n
a=rtpmap:104 ISAC/32000\r\n
a=rtpmap:9 G722/8000\r\n
a=rtpmap:102 ILBC/8000\r\n
a=rtpmap:0 PCMU/8000\r\n
a=rtpmap:8 PCMA/8000\r\n
a=rtpmap:106 CN/32000\r\n
a=rtpmap:105 CN/16000\r\n
a=rtpmap:13 CN/8000\r\n
a=rtpmap:110 telephone-event/48000\r\n
a=rtpmap:112 telephone-event/32000\r\n
a=rtpmap:113 telephone-event/16000\r\n
a=rtpmap:126 telephone-event/8000\r\n
a=ssrc:1737881718 cname:xxxx\r\n
a=ssrc:1737881718 msid:stream_label audio_label\r\n
a=ssrc:1737881718 mslabel:stream_label\r\n
a=ssrc:1737881718 label:audio_label\r\n
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 127 124 125\r\n
c=IN IP4 0.0.0.0\r\n
a=rtcp:9 IN IP4 0.0.0.0\r\n
a=ice-ufrag:xxxx\r\n
a=ice-pwd:+xxxx\r\n
a=ice-options:trickle\r\n
a=fingerprint:sha-256 xxxx\r\n
a=setup:active\r\n
a=mid:video\r\n
a=extmap:2 urn:ietf:params:rtp-hdrext:toffset\r\n
a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\n
a=extmap:4 urn:3gpp:video-orientation\r\n
a=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\n
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay\r\n
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type\r\n
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/video-timing\r\n
a=sendrecv\r\n
a=rtcp-mux\r\n
a=rtcp-rsize\r\n
a=rtpmap:96 H264/90000\r\n
a=rtcp-fb:96 ccm fir\r\n
a=rtcp-fb:96 nack\r\n
a=rtcp-fb:96 nack pli\r\n
a=rtcp-fb:96 goog-remb\r\n
a=rtcp-fb:96 transport-cc\r\n
a=fmtp:96 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\n
a=rtpmap:97 rtx/90000\r\n
a=fmtp:97 apt=96\r\n
a=rtpmap:98 VP8/90000\r\n
a=rtcp-fb:98 ccm fir\r\n
a=rtcp-fb:98 nack\r\n
a=rtcp-fb:98 nack pli\r\n
a=rtcp-fb:98 goog-remb\r\n
a=rtcp-fb:98 transport-cc\r\n
a=rtpmap:99 rtx/90000\r\n
a=fmtp:99 apt=98\r\n
a=rtpmap:100 VP9/90000\r\n
a=rtcp-fb:100 ccm fir\r\n
a=rtcp-fb:100 nack\r\n
a=rtcp-fb:100 nack pli\r\n
a=rtcp-fb:100 goog-remb\r\n
a=rtcp-fb:100 transport-cc\r\n
a=rtpmap:101 rtx/90000\r\n
a=fmtp:101 apt=100\r\n
a=rtpmap:127 red/90000\r\n
a=rtpmap:124 rtx/90000\r\n
a=fmtp:124 apt=127\r\n
a=rtpmap:125 ulpfec/90000\r\n
a=ssrc-group:FID 489266959 3672397520\r\n
a=ssrc:489266959 cname:xxxx\r\n
a=ssrc:489266959 msid:stream_label Ultrasonic\r\n
a=ssrc:489266959 mslabel:stream_label\r\n
a=ssrc:489266959 label:Ultrasonic\r\n
a=ssrc:3672397520 cname:xxxx\r\n
a=ssrc:3672397520 msid:stream_label Ultrasonic\r\n
a=ssrc:3672397520 mslabel:stream_label\r\n
a=ssrc:3672397520 label:Ultrasonic\r\n
a=ssrc-group:FID 3677845131 3184560832\r\n
a=ssrc:3677845131 cname:xxxx\r\n
a=ssrc:3677845131 msid:stream_label camera_0\r\n
a=ssrc:3677845131 mslabel:stream_label\r\n
a=ssrc:3677845131 label:camera_0\r\n
a=ssrc:3184560832 cname:xxxx\r\n
a=ssrc:3184560832 msid:stream_label camera_0\r\n
a=ssrc:3184560832 mslabel:stream_label\r\n
a=ssrc:3184560832 label:camera_0\r\n",
"type" : "answer"
}
->RtpVideoStreamReceiver :: RtpVideoStreamReceiver
->VideoReceiveStream :: VideoReceiveStream
->Call:: CreateVideoReceiveStream
->WebRtcVideoChannel :: WebRtcVideoReceiveStream :: RecreateWebRtcVideoStream
->WebRtcVideoChannel :: WebRtcVideoReceiveStream :: WebRtcVideoReceiveStream
->WebRtcVideoChannel :: AddRecvStream
->WebRtcVideoChannel :: AddRecvStream
->BaseChannel :: AddRecvStream_w //创建接收流资源。
->BaseChannel :: UpdateRemoteStreams_w
VideoChannel :: SetRemoteContent_w //根据SDP协议,创建接收的音视频、数据报文资源
2)创建解码线程
解码器是当有rtp报文输入的时候,动态创建的。
->H264DecoderImpl::InitDecode
->VCMGenericDecoder::InitDecode //初始化解码器
->VCMCodecDataBase::CreateAndInitDecoder //根据PT查找map表,确定解码格式,创建对应解码器
->VCMCodecDataBase::GetDecoder //每来一包,判断一下视频解码器是否创建,已经创建,用原有的。没有创建,创建新解码器
->VideoReceiver::Decode
->VideoReceiver::Decode
->VideoReceiveStream::Decode //解码线程读取RTP数据
VideoReceiveStream::DecodeThreadFunction
3)创建渲染线程
二,本端视频数据处理
1)创建视频capture线程
VideoCapturer::OnFrame
WebRtcVideoCapturer::OnFrame
videocapturemodule::VideoCaptureImpl::DeliverCapturedFrame
videocapturemodule::VideoCaptureImpl::IncomingFrame
videocapturemodule::CaptureSinkFilter::ProcessCapturedFrame
videocapturemodule::CaptureInputPin::Receive
2)创建编码线程
3)创建发送线程
webrtc::RTPSenderVideo::SendVideo
webrtc::RTPSender::SendOutgoingData
webrtc::ModuleRtpRtcpImpl::SendOutgoingData
webrtc::PayloadRouter::OnEncodedImage
webrtc::internal::VideoSendStreamImpl::OnEncodedImage
webrtc::VideoStreamEncoder::OnEncodedImage
webrtc::VCMEncodedFrameCallback::OnEncodedImage
webrtc::H264EncoderImpl::Encode
webrtc::VCMGenericEncoder::Encode
webrtc::vcm::VideoSender::AddVideoFrame
webrtc::VideoStreamEncoder::EncodeVideoFrame
webrtc::VideoStreamEncoder::EncodeTask::Run
rtc::TaskQueue::Impl::RunPendingTasks
rtc::TaskQueue::Impl::ThreadState::RunThreadMain
rtc::TaskQueue::Impl::ThreadMain
rtc::PlatformThread::Run
rtc::PlatformThread::StartThread