[MMUSIC] Comments on BUNDLE -37

Discussion:

[MMUSIC] Comments on BUNDLE -37 - RTP handling

Colin Perkins

2017-04-07 17:06:29 UTC

I have some comments on BUNDLE -37 - apologies that I wasn’t able to participate fully in the mailing list discussion before the meeting.

The current text is generally reasonably clear, and is certainly precisely written, but I do think it has some problems. Most of these relate to the issue of whether RTP packets or RTP streams are being processed. I think it’s important that this draft is written in terms of how to associate RTP streams with m= lines, rather than how to route and demultiplex RTP packets, since RTP packet routing and demultiplexing is already specified by the various RTP specifications. In particular, there are some places where the draft as written conflates the two layers in ways that conflict with the RTP and RTCP specifications, that I’ve tried to correct by more cleanly separating the layers.

I’ve tried to write my suggestions in a prescriptive manner, keeping to the style of the existing text where possible. Hopefully the result is clear and understandable.

10.2. Associating RTP/RTCP Streams With Correct SDP Media Description
NOTE: The text in this section is copied from Appendix B of JSEP.
The community has not yet agreed on the text.
As described in [RFC3550], RTP packets are associated with RTP
streams [RFC7656]. Each RTP stream is identified by an SSRC value,
and each RTP packet includes an SSRC field that is used to associate
the packet with the correct RTP stream. RTCP packets also use SSRCs
to identify which RTP streams the packet relates to. However, a RTCP
packet can contain multiple SSRC fields, in the course of providing
feedback or reports on different RTP streams, and therefore can be
associated with multiple such streams.
In order to be able to process received RTP/RTCP packets correctly,
it must be possible to associate an RTP stream with the correct "m="
Holmberg, et al. Expires October 2, 2017 [Page 20]
Internet-Draft Bundled media March 2017
line, as the "m=" line and SDP attributes associated with the "m="
line contain information needed to process the packets.
As all RTP streams associated with a BUNDLE group use the same
address:port combination for sending and receiving RTP/RTCP packets,
the local address:port combination cannot be used to associate an RTP
stream with the correct "m=" line. In addition, multiple RTP streams
might be associated with the same "m=" line.
An offerer and answerer can inform each other which SSRC values they
will use for an RTP stream by using the SDP 'ssrc' attribute
[RFC5576]. However, an offerer will not know which SSRC values the
answerer will use until the offerer has received the answer providing
that information. Due to this, before the offerer has received the
answer, the offerer will not be able to associate an RTP stream with
the correct "m=" line using the SSRC value associated with the RTP
stream. In addition, the offerer and answerer may start using new
SSRC values mid-session, without informing each other using the SDP
'ssrc' attribute.
In order for an offerer and answerer to always be able to associate
an RTP stream with the correct "m=" line, the offerer and answerer
using the BUNDLE extension MUST support the mechanism defined in
section 14, where the offerer and answerer insert the identification-
tag associated with an "m=" line (provided by the remote peer) into
RTP and RTCP packets associated with a BUNDLE group.
When using this mechanism, the mapping from an SSRC to an
identification-tag is carried in RTP header extensions or RTCP SDES
packets, as specified in section 14. Since a compound RTCP packet
can contain multiple RTCP SDES packets, and each RTCP SDES packet can
contain multiple chunks, a single RTCP packet can contain several
SSRC to identification-tag mappings. The offerer and answerer
maintain tables used for routing that are updated each time an RTP/
RTCP packet contains new information that affects how packets should
be routed.

The text above looks good. In particular, it correctly focusses on associating RTP streams with m= lines, which is the correct level of abstraction.

However, some implementations of may not include this identification-
tag in their RTP and RTCP traffic when using the BUNDLE mechanism,
and instead use a payload type based mechanism for demuxing. In this

The term “demuxing” is unclear. For precision, I suggest changing “use a payload type based mechanisms for demuxing” to “use a payload type based mechanism to associate RTP streams with SDP m= lines”.

situation, each "m=" line MUST use unique payload type values, in
order for the payload type to be a reliable indicator of the relevant
"m=" line for the RTP stream. Note that when using payload type
based demuxing,

Similarly, for precision, I suggest changing “when using payload type based demuxing” to “when using the payload type to associate RTP streams with m= lines”.

an SSRC will be mapped to an “m=“ line

Suggest changing to “an RTP stream, identified by SSRC, will be mapped…” to be clear what’s being done.

by the first
packet with that SSRC, and the mapping will not be changed even if

Suggest changing to “when the first RTP packet of that RTP stream is received, and…” to be clear, since there could be RTCP packets with the same SSRC.

the same SSRC is received with a different payload type. In other

Suggest changing to “the payload type used by that RTP stream changes. In other” to be precise.

words, the SSRC cannot to "move" to a different "m=" line simply by
changing the payload type.
Holmberg, et al. Expires October 2, 2017 [Page 21]
Internet-Draft Bundled media March 2017
Applications can implement RTP stacks in many different ways. The
algorithm below details one way that demultiplexing can be
accomplished, but is not meant to be prescriptive about exactly how

To be clear about what is being demultiplexed, I suggest changing “one way that demultiplexing can be accomplished” to “one way that RTP streams can be associated with m= lines”.

an RTP stack needs to be implemented. Applications MAY use any
algorithm that achieves equivalent results to those described in the
algorithm below.
To prepare for demultiplexing RTP/RTCP packets to the correct "m="
line, the following steps MUST be followed for each BUNDLE group.

I suggest changing “…prepare for demultiplexing RTP/RTCP packets to the correct…” to “…prepare to associate RTP streams with the correct…”. RTP and RTCP packets are not demultiplexed to m= lines, they’re demultiplexed into RTP streams, and then those RTP streams are associated with m= lines. The distinction is important, in order to implement RTCP correctly.

Construct a table mapping MID to "m=" line for each "m=" line in
this BUNDLE group. Note that an "m=" line may only have one MID.
Construct a table mapping incoming SSRC to "m=" line for each "m="
line in this BUNDLE group and for each SSRC configured for
receiving in that “m=" line.

The SSRC is a property of an RTP stream, so I suggest changing “mapping incoming SSRC to” to “mapping SSRCs of incoming RTP streams to”.

Construct a table mapping outgoing SSRC to "m=line" for each "m="
line in this BUNDLE group and for each SSRC configured for sending
in that “m=" line.

Similarly, change “mapping outgoing SSRC” to “mapping the SSRC of each outgoing RTP stream”

Construct a table mapping payload type to "m=" line for each "m="
line in the BUNDLE group and for each payload type configured for
receiving in that "m=" line. If any payload type is configured
for receiving in more than one "m=" line in the BUNDLE group, do
not it include it in the table, as it cannot be used to uniquely
identify a "m=" line.
Note that for each of these tables, there can only be one mapping
for any given key (MID, SSRC, or PT). In other words, the tables
are not multimaps.
As "m=" lines are added or removed from the BUNDLE groups, or their
configurations are changed, the tables above MUST also be updated.
For each RTP packet received, the following steps MUST be followed to
route the packet to the correct "m=" section within a BUNDLE group.

The goal is not to route RTP packets to m= lines, but rather to associate the corresponding RTP streams with m= lines. This keeps the distinction between RTP and RTCP processing that has to happen irrespective of the m= line, and the application level processing that depends on the choice of m= line. I suggest changing this sentence to: “When an RTP packet is received, it MUST be delivered to the RTP stream corresponding to its SSRC. That RTP stream MUST then be associated with the correct m= line within a BUNDLE group, according to the following steps.”

Note that the phrase 'deliver a packet to the "m=" line' means to
further process the packet as would normally happen with RTP/RTCP, if
it were received on a transport associated with that "m=" line
outside of a BUNDLE group (i.e., if the "m=" line were not BUNDLEd),
including dropping an RTP packet if the packet's PT does not match
any PT in the “m=" line.

Dropping RTP packets with unknown payload type breaks RTCP. The RTP packet must be processed by the RTP layer as normal, updating the statistics that are maintained by RTCP. The payload is then discarded, because there’s no decoder associated with the payload type. I suggest removing this part of the paragraph entirely.

If the packet has a MID, and that MID is not in the table mapping
MID to “m=" line, drop the packet and stop.

Dropping the RTP packet will break the RTCP reports. The RTP packet has to be processed as normal, but the corresponding RTP stream is not decoded if it’s not associated with an “m=“ line (i.e., you drop the payload, not the RTP packet). I suggest replacing the above with something like:

If the MID associated with the RTP stream is not in the table mapping
MID to “m=“ line, then the RTP stream is not decoded and the payload
data is discarded.

If the packet has a MID, and the packet's extended sequence number
is greater than that of the last MID update, as discussed in
[RFC7941], Section 4.2.6, update the incoming SSRC mapping table
to include an entry that maps the packet's SSRC to the "m=" line
for that MID.

There are two things that need to be done here: update the MID associated with the RTP stream to match that in the newly received RTP packet, and update the mapping table. I suggest changing “update the incoming SSRC mapping table to include an entry that maps the packet’s SSRC to the “m=“ line for that MID” to “update the MID associated with the RTP stream to match the MID carried in the RTP packet, then update the mapping tables to include an entry that maps the SSRC of that RTP stream to the “m=“ line for that MID”.

If the packet's SSRC is in the incoming SSRC mapping table, check
that the packet's PT matches a PT included on the associated "m="
line. If so, route the packet to that associated "m=" line and
stop; otherwise drop the packet and stop.

Dropping the packet breaks RTCP. The RTP packet is processed as normal, but the corresponding RTP stream is not decoded if it’s not associated with an “m=“ line. I suggest replacing the above with:

If the SSRC of the RTP stream is in the incoming SSRC mapping table,
check that the payload type used by the RTP stream matches a payload
type included on the matching “m=“ line. If so, associate the RTP
stream with that “m=“ line. Otherwise, the RTP stream is not decoded
and the payload data is discarded.

If the packet's payload type is in the payload type table, update
the the incoming SSRC mapping table to include an entry that maps
the packet's SSRC to the "m=" line for that payload type. In
addition, route the packet to the associated “m=" line and stop.

RTP packets correspond to RTP streams, and those RTP streams are associated with m= lines. Suggest changing to:

If the payload type used by the RTP stream is in the payload type
table, update the incoming SSRC mapping table to include an entry
that maps the RTP stream’s SSRC to the “m=“ line for that payload
type. Associate the RTP stream with the corresponding “m=“ line.

Otherwise, drop the packet.

The packet cannot be discarded without breaking RTCP. I suggest replacing the above with:

Otherwise, mark the RTP stream as not for decoding and discard the
payload.

For each RTCP packet received (including each RTCP packet that is
part of a compound RTCP packet), the packet MUST be routed to the
“m=“ line for the RTP streams it contains information about. This
routing is type-dependent, as each kind of RTCP packet has its own
mechanism for associating it with the relevant RTP streams.

The first sentence might be clearer written “For each RTCP packet received (including each RTCP packet that is part of a compound RTCP packet), the packet is processed as usual by the RTP layer, then is passed to the “m=“ lines corresponding to the RTP streams it contains information about for further processing.”

Packets for which no appropriate "m=" line can be identified (i.e.,
for unknown RTP streams) are not relevant in the context of this
algorithm and MAY be dropped. This situation may occur with certain
multiparty RTP topologies.

These packets can’t be dropped. They setup state at the RTP layer that might become relevant when further RTP/RTCP packets are received (RTCP packets can round-robin SDES items, such as the MID, in some cases, so these can potentially be important). I suggest changing this to:

RTCP packets for which no appropriate “m=“ line can be identified
MUST be processed as usual by the RTP layer, updating the metadata
associated with the corresponding RTP streams, but are not passed
to any “m=“ line. This situation can occur with certain multiparty
RTP topologies, or when RTCP packets are sent containing a subset
of the SDES information.

Rules for handling the various types of RTCP packets are explained
below.

Perhaps change to “Rules for additional processing of the various…”, to make it clear that this doesn’t replace the usual RTCP processing.

If the packet is of type SDES, for each chunk in the packet whose

"If the RTCP packet is…”

SSRC is found in the incoming SSRC table, deliver a copy of the
packet to the "m=" line associated with that SSRC. In addition,

“a copy of the SDES packet” presumably, since otherwise it’s ambiguous if the entire compound RTCP packet is delivered or just the SDES packet.

for any SDES MID items contained in these chunks, if the MID is
found in the table mapping MID to "m=" line, update the incoming
SSRC table to include an entry that maps the chunk’s SSRC to the

“…maps the RTP stream associated with the chunk’s SSRC to…”

"m=" line associated with that MID, unless the packet is older
than the packet that most recently updated the mapping for this
SSRC, as discussed in [RFC7941], Section 4.2.6.
Note that if an SDES packet is received as part of a compound RTCP
packet, the SSRC to "m=" line mapping may not exist until the SDES
packet is handled (e.g., in the case where RTCP for a source is
received before any RTP packets). Therefore, when processing a
compound packet, any contained SDES packet MUST be handled first.

It might be worth referencing RFC 3550 section 6.1, which says:

Each individual RTCP packet in the compound packet may be processed
independently with no requirements upon the order or combination of
packets.

as justification for this.

Holmberg, et al. Expires October 2, 2017 [Page 23]
Internet-Draft Bundled media March 2017
If the packet is of type BYE, it indicates that the RTP streams

“If the RTCP packet is…”

referenced in the packet are ending. Therefore, for each SSRC
indicated in the packet that is found in the incoming SSRC table,

“…in the BYE packet…”

first deliver a copy of the packet to the “m=" line associated

“…a copy of the BYE packet…”

with that SSRC, but then remove the entry for that SSRC from the
incoming SSRC table after an appropriate delay to account for
"straggler packets", as specified in [RFC3550], Section 6.2.1.
If the packet is of type SR or RR, for each report block in the

“If the RTCP packet is…”

report whose "SSRC of source" is found in the outgoing SSRC table,
deliver a copy of the RTCP packet to the “m=" line associated with

“the RTCP packet” or “the SR or RR packet”? To avoid confusion when compound RTCP packets are used, I suggest the latter phrasing here, and in the later sections.

that SSRC. In addition, if the packet is of type SR, and the
sender SSRC for the packet is found in the incoming SSRC table,
deliver a copy of the packet to the “m=" line associated with that

“a copy of the SR packet”?

SSRC.
If the implementation supports RTCP XR and the packet is of type
XR, as defined in [RFC3611], for each report block in the report
whose "SSRC of source" is is found in the outgoing SSRC table,
deliver a copy of the RTCP packet to the “m=" line associated with

“the RTCP packet” or “the XR packet”?

that SSRC. In addition, if the sender SSRC for the packet is
found in the incoming SSRC table, deliver a copy of the packet to

“the packet” -> “the RTCP packet” or “the XR packet”?

the "m=" line associated with that SSRC.
If the packet is a feedback message of type RTPFB or PSFB, as

“If the RTCP packet is…”

defined in [RFC4585], it will contain a media source SSRC, and
this SSRC is used for routing certain subtypes of feedback
messages. However, several subtypes of PSFB messages include
target SSRC(s) in a section called Feedback Control Information
(FCI). For these messages, the target SSRC(s) are used for
routing.
If the packet is a feedback message that does not include target

“If the RTCP packet is…”

SSRCs in its FCI section, and the media source SSRC is found in
the outgoing SSRC table, deliver the packet to the “m=" line

Which packet?

associated with that SSRC. RTPFB and PSFB types that are handled
Generic NACK: [RFC4585] (PT=RTPFB, FMT=1).
Picture Loss Indication (PLI): [RFC4585] (PT=PSFB, FMT=1).
Slice Loss Indication (SLI): [RFC4585] (PT=PSFB, FMT=2).
Reference Picture Selection Indication (RPSI): [RFC4585]
(PT=PSFB, FMT=3).
Holmberg, et al. Expires October 2, 2017 [Page 24]
Internet-Draft Bundled media March 2017
If the packet is a feedback message that does include target

“If the RTCP packet is…”

SSRC(s) in its FCI section, it can either be a request or a
notification. Requests reference a RTP stream that is being sent
by the message recipient, whereas notifications are responses to
an earlier request, and therefore reference a RTP stream that is
being received by the message recipient.
If the packet is a feedback request that includes target SSRC(s),

“If the RTCP packet is…”

for each target SSRC that is found in the outgoing SSRC table,
deliver a copy of the RTCP packet to the "m=" line associated with
Full Intra Request (FIR): [RFC5104] (PT=PSFB, FMT=4).
Temporal-Spatial Trade-off Request (TSTR): [RFC5104] (PT=PSFB,
FMT=5).
H.271 Video Back Channel Message (VBCM): [RFC5104] (PT=PSFB,
FMT=7).
Layer Refresh Request (LRR): [I-D.ietf-avtext-lrr] (PT=PSFB,
FMT=TBD).
If the packet is a feedback notification that include target
SSRC(s), for each target SSRC that is found in the incoming SSRC
table, deliver a copy of the RTCP packet to the "m=" line
associated with that SSRC. PSFB types that are handled in this

“deliver a copy of the RTCP packet to the “m=“ line associated with the RTP stream with matching SSRC”

Temporal-Spatial Trade-off Notification (TSTN): [RFC5104]
(PT=PSFB, FMT=6). This message is a notification in response
to a prior TSTR.
If the packet is of type APP, the only routing information
included is the source of the packet, and therefore the packet
could be related to any existing "m=" line. Accordingly, deliver
a copy of the packet to each “m=" line.

Are APP packets exposed in the WebRTC APIs? Given that we have the data channel, it’s not clear that we want to support arbitrary application information passing via RTCP. It might be better to say that APP packets are processed in an application specific manner, and if the application doesn’t understand them, they’re not passed to any m= line?

Finally, I note that this section of the draft doesn’t mention the CSRC list anywhere, but should probably do so. As written, RTP packets with a CSRC list will be delivered to the RTP stream matching their SSRC in the usual way, then passed up to the m= line associated with that RTP stream. That may well be sufficient, but if so it’s likely useful to say that, to make the intent clear.

Colin

--
Colin Perkins
https://csperkins.org/

Christer Holmberg

2017-04-09 06:54:35 UTC

Permalink

Hi Colin,

Thanks for your input!

Would it be possible for you to create a pull request with your suggested changes?

BUNDLE can be found at: https://github.com/cdh4u/draft-sdp-bundle

Regards,

Christer

-----Original Message-----
From: mmusic [mailto:mmusic-***@ietf.org] On Behalf Of Colin Perkins
Sent: 07 April 2017 20:06
To: mmusic (E-mail) <***@ietf.org>
Subject: [MMUSIC] Comments on BUNDLE -37 - RTP handling

I have some comments on BUNDLE -37 - apologies that I wasn’t able to participate fully in the mailing list discussion before the meeting.

The current text is generally reasonably clear, and is certainly precisely written, but I do think it has some problems. Most of these relate to the issue of whether RTP packets or RTP streams are being processed. I think it’s important that this draft is written in terms of how to associate RTP streams with m= lines, rather than how to route and demultiplex RTP packets, since RTP packet routing and demultiplexing is already specified by the various RTP specifications. In particular, there are some places where the draft as written conflates the two layers in ways that conflict with the RTP and RTCP specifications, that I’ve tried to correct by more cleanly separating the layers.

I’ve tried to write my suggestions in a prescriptive manner, keeping to the style of the existing text where possible. Hopefully the result is clear and understandable.