Master Dissertation
Master Dissertation
Scenario: COP as server, SOLDIER as client. Network bandwidth varied between 200kbps and 300kbps, 30ms network latency, packet loss varied between 0% and 10%.
Application adapted to SCTP: poc version 0.3.7.
The test determined the ability of some network combinations to deliver an MP3 audio stream with reasonable quality through a limited bandwidth network with packet losses. This test is completely different from all others because the definition of "reasonable" audio is highly subjective. Still, the following guidelines were followed:
Another difference from the other tests, is that we compare SCTP with UDP here. SCTP was allowed to deliver messages unordered, as well as discard messages past lifetime. Also, PR-SCTP could finally be used and compared with standard SCTP.
Prebuffering at client was 200ms, except when FEC protocol is used since it does not allow buffer dimensioning in terms of time.
Even though MP3 has fixed bitrate and is somewhat capable of recovering from losses, it is not really appropriate for transmission over lossy channels. (RTP is not lossy by itself but is normally transmitted over UDP, so the RTP+UDP combination makes a lossy channel). Some protocols were designed to adapt MP3 to lossy RTP: RFC 2250, RFC 3119 a.k.a. FEC (Forward Error Correction) — the latter proposed by poc's author.
In the charts below, X axis is the packet loss, and Y axis is the bandwitdh. The upper-left points represent the most favorable network scenarios (i.e. the easiest scenarios for a protocol to handle: more bandwidth and few losses). There is one line for every protocol bundle (RTP+UDP, RTP+SCTP, RTP+PR-SCTP) that divides the cartesian plane in two portions:
So we can say that, the bigger the area left of the line, the stronger is the protocol bundle when handling network problems.
RFC 2250 is a poor adaptation of MP3 over RTP since it is very sensible to packet losses, as shown in the bad performance of UDP. Any loss will introduce periodic glitches in audio, that tend to be unrecoverable.
Using a confirmed protocol improves a lot the loss resistance. Standard SCTP renders RFC 2250 perfectly usable in several adverse scenarios. Another interesting side effect of SCTP was recovery of failures: after the fail, the periodic glitch disappears after some seconds.
PR-SCTP performance was less tolerant than standard SCTP, and it did not work with losses above 2%. Possibly the message discard pattern of PR-SCTP was not tolerated by RFC 2250. (As mentioned before, RFC 2250 is not a particularly good way of delivering MP3 audio.)
RFC 3119 is more efficient than RFC 2250 when employed over lossy networks, and SCTP further improved packet loss resistance, as shown by the above graphic.
PR-SCTP did somewhat better than standard SCTP when losses were below 2%. Message lifetime was configured to 200ms. We believe that PR-SCTP could perform even better if application calibrates the lifetime dynamically, accordingly to network conditions.
The FEC (Forward Error Correction) algorithm creates redundant data for error correction. The 192kbps MP3 content fattens up to 260kbps, but the encoded content is resistant to data loss, because the receiver can often recreate the lost data.
FEC is roughly the same mechanism used in audio CDs, where 25% of raw data is redundancy, which allows correction of errors caused by scratches and dust. Data CDs have a second level of redundancy of 25%, which make them even more scratch-resistant.
FEC is tailored for lossy transports as UDP, and graphic shows that UDP performance was excellent. Note that audio content could be delivered even when network bandwith was narrower than 260kbps (provided that packet losses were few). This shows that FEC is really rebuilding lost data.
Standard SCTP performed badly, worse than UDP in every scenario. The main reason is that it simply cannot cope with a network that is slower than the data stream (260kbps). This is expected since SCTP is a confirmed protocol, that strives to deliver all the data.
PR-SCTP was a lot more adequate in this test; it could handle with a network slower than data flow (althrough not as well as UDP since SCTP overhead is bigger). And PR-SCTP was better than UDP in severe packet loss scenarios, since it can retransmit lost messages within the timeframe of message lifetime.
The better performance of PR-SCTP was not achieved for free. It was necessary to lower the message lifetime from 200ms to 80-100ms, when packet loss was 5% or higher. Without this, the flow stalls completely.
PR-SCTP can do much better if the transmitter calibrates the message lifetime accordingly to actual network conditions, and the minimum RTO was lowered to an value adequate to network latency (30ms in this test). The manual adjust to 80ms had in mind the network RTT of 60ms (2x30ms), with a small margin for jitter. This is enough for exactly one retransmission. In a network with losses of 10%, the possibility of one retransmission effectively reduces the actual loss to 1% (10% of 10%), which is deep within the FEC recovery capabilites.