[Community-allwww-cvs] www/2005/devdocs RDT_Feature_Level_20.txt, NONE, 1.1 RDT_Feature_Level_30.txt, NONE, 1.1
rishimathew at helixcommunity.org rishimathew at helixcommunity.orgUpdate of /cvsroot/protocol/www/2005/devdocs
In directory cvs:/tmp/cvs-serv26602/2005/devdocs
Added Files:
RDT_Feature_Level_20.txt RDT_Feature_Level_30.txt
Log Message:
Organizing Protocol project a little better
--- NEW FILE: RDT_Feature_Level_30.txt ---
Copyright (c) 2005, RealNetworks, Inc. All rights reserved.
RDT Feature level 3.0
Design Specification
1 Introduction
This paper defines a set of changes to the RDT transport for release in the
next Vodafone deliverable. The primary impetus for these changes is the desire
to ensure good performance of sender-side congestion control mechanisms. While
we do not see RDT as a transport we will focus development on in the future, our
current RTP implementation does not support ARQ based error correction and this
mechanism is highly desirable for congestion management as well as handover/cell
reselection. Thus these changes constitute highly tactical improvements with
minimal impact on existing components in the field.
2 References
[ 1 ] M. Handley, J. Padhye, S. Floyd, J. Widmer. TCP Friendly Rate Control
(TFRC): Protocol Specification. RFC 3448, rfc3448.txt, January 2003.
[ 2 ] 3GPP TS 26.234 V5.3.0 Transparent end-to-end Packet-switched
Streaming Service (PSS);Protocols and codecs(Release 5), December 2002
[ 3 ] H. Schulzrinne, S. Casner, R. Frederick, V. Jacobson. RTP: A Transport
Protocol for Real-Time Applications. RFC 1889, rfc1889.txt, January 1996.
[ 4 ] T. Friedman, R. Caceres, A. Clark. RTP Control Protocol Extended
Reports (RTCP XR). draft-ietf-avt-rtcp-report-extns-06.txt, May 2003
[ 5 ] J. Rey, D. Leon, A. Miyazaki, V. Varsa, R. Hakenberg. RTP Retransmission
Payload Format, draft-ietf-avt-rtp-retransmission-08.txt, December 2002
[ 6 ] S. Casner, V. Jacobson. Compressing IP/UDP/RTP Headers for Low-
Speed Serial Links, RFC 2508, rfc2508.txt, February 1999
[ 7 ] C. Bormann, Editor. RObust Header Compression (ROHC): Framework
and four profiles: RTP, UDP, ESP, and uncompressed, RFC 3095, rfc3095.txt,
July 2001
[ 8 ] R. Braden, Editor. Requirements for Internet Hosts -- Communication
Layers. RFC 1122, rfc1122.txt, October 1989
3 Overview
The congestion control model being implemented in the server is based upon
an equation-based model, as in [ 1 ], in which congestion is managed at the
sender through feedback provided by the receiver. The two most important inputs
to this equation are loss history and propagation delay. These variables are
sufficient to characterize the link throughput dynamics in order to maintain a
maximum sustainable throughput. These variables, however, must be made
available frequently and accurately. Further, as delivery of media data becomes
initially bounded by network capacity rather than the consumption rate
mechanisms must be put in place to prevent buffer overrun on the receiver. 3GPP
SA4 has defined a model for achieving this. Annex G in [ 2 ] defines a video
buffer verifier to prevent receiver overflow, but this model allows for certain
ambiguities to arise in underflow situations. This proposed change addresses
issues in receiver buffer monitoring as well.
While the RDT transport already provides sufficient loss characterization,
it does not currently provide a sufficient basis for computing propagation
delay. While RDT NAK/ACK packets provide some estimation, it lacks a timeline
synchronization with the sender clock as appears in receiver reports from RTP [
3 ]. This solution defines a new packet requesting this synchronization
information. The change will result in a feedback frequency roughly equivalent
to TCP.
This change also defines a new semantic requirement for the receiver. We
define a way to ensure that live media packets lost en route to the sender do
not trigger a congestion event on the end receiver. This addresses issues
observed within a large live infrastructure in which packets in live delivery
spread a cascading effect to all connected receivers.
These changes define a tactical improvement within the possibly short-term
future life of the RDT protocol. With an increased need to interoperate with
other client and server applications, convergence on the standard RTP transport
[ 3 ] is becoming more necessary as the focus of our development. This
observation should serve to frame the discussion on the changes to a short-term
window. At the point when error correction and QoS enabling extension to RTP
are standardized and available as in [ 4 ] and [ 5 ], as well as mechanisms for
header compression as in [ 6 ] and [ 7 ] become integrated in metworks, we will
consider the possibility of transitioning to RTP as the mainline delivery
mechanism.
4 Specification
The following specifications apply only if the RTSP header for both client
and server specify support for RDTFeatureLevel 3.0 or higher.
4.1 RDTTransportInfoRequestPacket
struct RDTTransportInfoRequestPacket
{
bit[1] dummy0; /* 0 */
bit[5] dummy1; /* 0 */
bit[1] request_rtt_info;
bit[1] request_buffer_info;
u_int16 packet_type; /* 0xff09 */
if (request_rtt_info == 1)
{
u_int32 request_time_ms;
}
}
The RDTTransportInfoRequestPacket serves as a request for propagation delay
information and/or receiver buffer state. This request packet MAY be issued by
either the sender or the receiver of media data, but if issued by the receiver
of data is MUST NOT set the request_buffer_info flag. The sender of the
request packets SHOULD meter their issuance so that the traffic generated does
not exceed the feedback of delayed ACK TCP as specified in [ 9 ]. The request
packet MAY be included as part of another RDT datagram (as part of an RDT data
packet, for instance), but SHOULD NOT be added if, in adding it, the datagram
exceeds the network's maximum segment size.
4.2 RDTTransportInfoResponsePacket
struct RDTBufferInfo
{
u_int16 stream_id;
u_int32 lowest_timestamp;
u_int32 highest_timestamp;
u_int32 bytes_buffered;
}
struct RDTTransportInfoResponsePacket
{
bit[1] dummy0; /* 0 */
bit[4] dummy1; /* 0 */
bit[1] has_rtt_info;
bit[1] is_delayed;
bit[1] has_buffer_info;
u_int16 packet_type; /* 0xff0a */
if (has_rtt_info == 1)
{
u_int32 request_time_ms;
if (is_delayed)
{
u_int32 response_time_ms;
}
}
if (has_buffer_info == 1)
{
u_int16 buffer_info_count;
RDTBufferInfo[buffer_info_count] buffer_info;
}
}
The RDTTransportInfoResponsePacket is issued in response to the
RDTTransportInfoRequestPacket and SHOULD contain responses for all requested
fields. Since these packets are generally delivered via an unreliable
channel, the server MUST NOT depend on the reception of any particular
response.
4.2.1 has_rtt_info response
When has_rtt_info is set in the response packet, the response SHALL include
the request_time_ms from the request_time_ms field from the request packet it is
responding to. The response_time_ms field SHOULD be filled from the most
accurate clock available to the receiver and represent the elapsed time from the
receipt of the corresponding request packet. The receiver of this packet SHOULD
respond to every request. The response SHOULD be issued as soon as is feasible.
4.2.2 has_buffer_info response
When has_buffer_info is set in the response packet, the response MUST
include RDTBufferInfo structures for each for every stream established for that
RDT session. The order of those streams MUST correspond to the stream_id's from
the RDT session. Data within each RDTBufferInfo SHOULD contain data for all
packets in the stream not yet passed to the renderer. The receiver MUST ensure
that any packets with identical timestamps equal to the lowest_timestamp field
are associated together such that there instance when some but not all packets
of equal timestamp are included in the bytes_buffered field. The bytes_buffered
field MUST include only payload data and not any transport related headers. If
there are gaps in the sequence number space in the buffered data, then the
receiver SHOULD include an RDTNakPacket in the same datagram. If no packets are
currently in the receiver's buffer, the receiver MUST include the timestamp
value of the packet most recently sent to the renderer as both the
highest_timestamp and lowest_timestamp. If no data has been sent to the
renderer, the receiver MUST set both values to zero.
Example:
If the receiver has received the following packets and has not sent any to the renderer:
Seqno: 0 Stream: 0 TS: 0 Len: 400
Seqno: 0 Stream: 1 TS: 0 Len: 100
Seqno: 1 Stream: 0 TS: 0 Len: 400
Seqno: 1 Stream: 1 TS: 100 Len: 100
Seqno: 2 Stream: 1 TS: 100 Len: 100
Seqno: 2 Stream: 0 TS: 200 Len: 400
The receiver would include the following:
stream_id 0
lowest_timestamp 0
highest_timestamp 200
bytes_buffered 1200
stream_id 1
lowest_timestamp 0
highest_timestamp 100
bytes_buffered 300
On stream 0, since the first two sequence numbers have the same timestamp, the
buffer calculation must include both of them or neither of them, so the
following would be invalid:
stream_id 0
lowest_timestamp 0
highest_timestamp 200
bytes_buffered 800
the receiver should remove both at once as in:
stream_id 0
lowest_timestamp 100
highest_timestamp 200
bytes_buffered 400
4.3 Bandwidth Overhead for TransportInfo packets
The following is a short explanation of the expected bandwidth consumption
invoked from using the TransportInfo mechanism. We use as our example an
audio/video session over GPRS at 25kbps with an average packet size of 400
bytes. We also assume that the round trip time is probed on every other data
packet and that the buffer information is probed once per second.
For the downstream (request case) we have:
25kbps / 400 bytes per sec = 8 packets/sec / 2 = 4 request packets/sec
If the request packets are embedded with data packets then there is no UDP/IP
overhead added and we get:
4 packets/sec * 7 bytes = 28Bps = 0.218kbps = 0.8% of the media rate
For the upstream case (here the packets are generally sent individually, so we
will consider UDP/IP overhead for the rtt packets with the buffer_info combined
with one of the rtt)
((28 bytes IP header + 20 bytes UDP header + 11 bytes rrt_info)
* 4 packets/sec)+ (30 bytes BufferInfo * 1 packet/sec) = 266 Bps
= 2.078 kbps = 8.31% of the media rate
The percentage overhead should generally serve as an upper bound since the
average packet size will tend to increase with bitrate.
4.4 Handling of NULL Packets
A NULL packet is defined as an RDT packet with a zero length data segment.
If the data sender is unable or chooses not to send a given data packet, it MUST
send a NULL packet with the corresponding sequence number instead. Upon receipt
of a NULL packet, the client MUST treat this segment as lost in respect to the
media render. It MUST NOT, however, treat the packet as lost in respect to
congestion. The client SHALL NOT reduce sending rate nor initiate a media
downshift upon receipt of a NULL packet. The client SHALL treat the sequence
number of received NULL packets as received in respect to reception bit field of
RDTAckPacket. In parsing the NULL packet, all fields other than stream_id and
seq_no SHALL be undefined. Any intermediate device proxying the delivery of RDT
data SHALL pass through a NULL packet as it would any other packet.
5 Revision History
0.1 06/26/03 Sean Robinson: Initial draft
0.9 07/02/03 Sean Robinson: Changed the the requirement for all requested fields from
SHOULD to MUST. Clarified usage of response_time_ms
0.91 07/09/03 Sean Robinson: Added bandwidth usage section
$Log: RDT_Feature_Level_30.txt,v $
Revision 1.1 2005/07/22 21:44:40 rishimathew
Organizing Protocol project a little better
Revision 1.1 2005/05/12 00:02:17 mfrazier
added update RDT spec
Revision 1.3 2003/07/14 18:32:21 srobinson
Fixed buffer example
Revision 1.2 2003/07/11 18:03:05 srobinson
Made inclusion of response_time_ms depndent on is_delayed flag.
Changed response inclusion requirement for MUST to SHOULD.
Revision 1.1 2003/07/10 20:48:37 srobinson
Added buffer example
Fixed incorrect bandwidth calaculation
Specified empty buffer handling
--- NEW FILE: RDT_Feature_Level_20.txt ---
===========================
RDT Data recovery mechanism
===========================
This document discusses the data recovery mechanism used within the
RealNetworks Data Transport(RDT). It covers the method used by the client
to notify the server of packets not received as well as the method the
server used to notify the client of a packet lost before transport that
is unrecoverable.
-------------------------------------------------------------------------------
Table A. RDT DATA PACKET (non-aggregated)
012 78 24 32 64 80
+++----++---------------+++-----+-------...+-----------...+--...+
||| ||SeqNo |||Rule |Timestamp |Reliable SeqNo|Data |
+++----++---------------+++-----+-------...+-----------...+--...+
^^^ ^^ ^^^
||| || |||
||| || ||ASM Rule
||| || |slow_data
||| || back_to_back_packet
||| ||
||| |RDT Sequence Number
||| is_reliable
||stream_id
|need_reliable (always true)
length_included_flag
-------------------------------------------------------------------------------
I. RDT data packets
The RDT stream consists of one or more streams each with a set of
sequence numbers. Each stream has it's own set of sequence numbers that start
at 0 and wrap at 0xff00. Thus a packet's uniqueness is determined by the
combination of stream_id and sequence number. Some streams are denoted by the
datatype as high priority or less loss tolerant through the is_reliable flag.
There is a second sequence number is global to the stream and is the sum total
of all packets with the is_reliable flag set. This second sequence number
is application specific.
If the length_included_flag is set on an RDT data packet, then
packet aggregation is occurring. Packet aggregation is inclusion of more
than one RDT packet within a UDP datagram to reduce system calls in the server.
See the chart below for aggregation rules. Aggregated packets have a slightly
different format than non-aggregated. They contain a length field allowing the
client to jump to the next RDT header. Packet aggregation can be disabled at
the server with the option --dpa.
-------------------------------------------------------------------------------
Table B. PACKET AGGREGATION VALUES IN RealSystem Server 8.01
Normal LoadState Minimum size size to aggregate 200 bytes
Maximum size to aggregate to 300 bytes
High LoadState Minimum size size to aggregate 700 bytes
Maximum size to aggregate to 1000 bytes
Extreme LoadState
(or high load state and bitrate > 100Kbps)
Minimum size size to aggregate 1000 bytes
Maximum size to aggregate to 1350 bytes
Note: The server will only aggregate if the sum of two or more
packet lies within the minimum and maximum values for the given
server condition
-------------------------------------------------------------------------------
II. Packets lost before transport
All RealNetworks datatypes are tolerant to loss. Sometimes in a live
presentation, a packet will be lost en route to the server. If unrecoverable
packet loss occurs between the encoder and server, for example, the server
will not have all the packets to send to the client. The server cannot,
however, just increment the sequence number as though the lost packet never
existed since the datatype needs to know this data is missing. The server
also cannot simply increment the sequence number as the client will issue a
NAK packet assuming the packet was lost in transport. To solve this the server
sends a data packet with the missing sequence number, but no payload data. The
client is aware of this convention and marks this packet as lost, but does not
try to recover by sending a NAK packet. Note that in TCP mode the server just
leaves a gap in the sequence number as there is no ambiguity the packet is
assumed lost before transport.
-------------------------------------------------------------------------------
Table C. RDT NAK/ACK PACKET
RDT NAK PACKET
012 78 24 40 56 72
+++----++---------------+------...+---...+------...+-------+
||| ||0xff02 |stream_id|seq_no|bit_count|map_len|
+++----++---------------+------...+---...+------...+-------+
^
|
lost_high (1 for NAK)
- First three bytes of NAK packet are always 0x40ff02
- bit_count and map_len are always 0 for NAK
- packet indicates that packet on stream stream_id with seq_no was
not received.
- lost_high means that the sequence number listed for the stream was
lost. Since the NAK is always for one packet, it is always set.
RDT ACK PACKET
|<---Repeated for each stream in session--->|
012 78 24 40 56 72 80
+++----++---------------+------...+---...+------...+-------+-----...+
||| ||0xff02 |stream id|seq_no|bit_count|map_len|bitfield|
+++----++---------------+------...+---...+------...+-------+-----...+
^
|
lost_high (almost always 0 for ACK)
- lost_high is basically never set for ACK packets. See section III.
- data portion contains one section per stream
- stream_id is the stream being ACKed
- seq_no is the highest sequence number in the bitfield.
- bit_count indicates number bits in bitfield (packets being ACKed)
- map_len is the length in bytes (rounded up) of the bitfield
- There is one bitfield for every packet ACKed. 1 indicates packet was
received. 0 indicates packet was not received.
-------------------------------------------------------------------------------
III. Packets lost in transport
Data recovery is achieved through the client's issuance of ACK/NAK
packets. As soon as a gap in sequence numbers is detected on the client,
the client issues a NAK packet for each missing sequence number. Note this
means the server will issue a NAK for packets received out of order. In
addition the client, at an interval of 1 second, will issue an ACK packet.
The ACK packet contains a bitfield for each stream in the session. 1 in the
field indicates the packet was received, 0 indicates it was not. If the server
receives either a NAK packet or a ACK with a a packet designated as missing,
it will resend the packet. Every packet marked as received is removed from
the server's resend buffer.
This document has referred ACK and NAK packets as if they were
differing packet formats, which they aren't. They can, however, be effectively
distinguished by the lost_high flag. This flag indicates that the sequence
number listed was lost. Since ACK packets are generated based on the highest
number sequence received, the highest sequence number would not be treated as
lost, just not received. The only exception would be a rare situation in
which a stream ACK exceeded the maximum bitset size and had to be sent in 2
ACK packets AND the last packet in the first ACK section happened to be lost.
The maximum bitset is 48 bytes or 384 packets. Since ACK are generated once per
second, this cannot realisticly happen for streams les than 1.5Mbps.
IV. Identifying RDT data recovery on the wire
NAK packets always begin with 0x40ff02 and are issued once per lost packet.
ACK packets always begin with 0x00ff02 and are issued about once per second.
Placeholder packets for data lost before transport are always 10 bytes long.
-------------------------------------------------------------------------------
Appendix - packet definitions and field summaries
(if expressions indicate differing packet formats based on values in static
fields)
packlen struct TNGDataPacket
{
bit[1] length_included_flag; /* Inc. length field
*/
bit[1] need_reliable_flag; /* Inc. rel. seq no */
bit[5] stream_id; /* 31 == expansion */
bit[1] is_reliable;
u_int16 seq_no; /* overload for packet type*/
if(length_included_flag == 1)
{
u_int16 rdt_packet_length; // payload length if
// length flag is set
}
bit[1] back_to_back_packet;
bit[1] slow_data;
bit[6] asm_rule_number; /* 63 is expansion */
u_int32 timestamp;
if(stream_id == 31)
{
u_int16 stream_id_expansion;
}
if(need_reliable_flag == 1)
{
u_int16 total_reliable;
}
if(asm_rule_number == 63)
{
u_int16 asm_rule_number_expansion;
}
buffer data;
}
- length_included_flag is only set when using packet aggregation (i.e.
more than one payload packet per datagram). When not set size of datagram
is used to infer size of payload
- need_reliable_flag is always set
- stream_id is datatype specific (usually 0 for audio, 1 for video)
stream_id 31 is reserved for future exapansion for sessions with more than
31 streams. It is not currently used.
- is_reliable is set for certain rules deemed less tolerant to loss
- seq_no starts at 0 and wraps at 0xff00. seq_nos are per stream.
Values higher than 0xff00 are used to designate special packet types (such
as ACK 0xff02)
- back_to_back_packet is used for timing and is set one out of ten
data packets
- slow_data is reserved for future use and is not currently set
- asm_rule_number is used by SureStream in conjunction with ASM rulebook
in file headers to allow client to subscribe to streams catering to
the client bandwidth. asm_rule_number 63 is reserved for future exapansion
for streams with more than 63 rules. It is not currently used.
- timestamp is datatype specific, but tends to increase over time.
- total_reliable is a separate sequence number kept for all packets
with is_reliable set for a given stream.
struct TNGACKPacket
{
bit[1] length_included_flag;
bit[1] lost_high;
bit[5] dummy0; /* 0 */
bit[1] dummy1; /* 0 */
u_int16 packet_type; /* 0xff02 */
if(length_included_flag == 1)
{
u_int16 length;
}
/*
* data consists of header + bitmap repeated for each stream:
* u_int16 stream_number
* u_int16 last_seq_no
* u_int16 bit_count
* u_int8 bitmap_length
* buffer bitmap
*/
buffer data;
}
- length_included_flag is always 0 for ACK
- lost_high indicates that the sequence number in the data segment is
a lost packet. These packets have no ACK component. These packets will only
have a data segment for 1 stream.
- packet_type is 0xff02 for ACK/NAK packets.