Master Dissertation

SCTP performance test - 100Mbps network with variable loss

Scenario: COP as server, SOLDIER as client, Ethernet 100Mbps with variable packet loss. Messages of 500 bytes.

Throughput - variation of packet loss

Protocol	0%	0,1%	1%	3%	10%
TCPM	90596	78846	57876	23396	878
SCTP	60773	56834	57030	46438	852

Packet loss in %, throughput in kbps (1kbps = 1000 bits/s).

With moderate losses (1% to 3%), SCTP could surpass TCPM, possibly helped by the bidirectional data flow created by this test, which allowed the stronger SCTP SACK to perform the best.

In latency test, the lack of continuous bidirectional traffic prevented SACK from working properly.

Latency - variation of packet loss

Protocol	0%	0,1%	1%	3%	10%
TCPM	281,0	483,9	5190,0	14418,5	67097,0
SCTP	318,5	1920,0	24662,0	82718,0	671100,5

Packet loss in %, latency in µs

Among all tests, this was the worst SCTP result, due to a combination of factors:

Default initial RTO (transmission timeout) is 1000ms, too high for a local network. This parameter can be ajusted via sysctl;
Minimum RTO is 1000ms, way too high. This parameter can also be adjusted via sysctl. This value means the smallest RTO practiced by the congestion control system, even if the real-time calculations point to a smaller value. A minimum RTO of 1 second is hundreds of times the correct value for a local network;
TCP demonstrated to use a smaller minimum RTO, the tests show that it adapted well to lossy networks;
SCTP congestion control follows exactly the "classic" algorithm, which mandates, among other things, a minimum RTO of one second. Perhaps Linux TCP have incorporated some additional heuristics;
SCTP congestion control could be interpreting (wrongly) that every loss is caused by congestion. Such interpretation would lower throughput until full stall, while Linux TCP would be recognizing losses as link-inherent packet loss and adapting to it.

In the first test, that supplied the data for dissertation and for the above graphic, adjusting the minimum RTO did not suffice: traffic began fast but ended up stalling and converging to a very slow speed.

In a second test, run after dissertation concluded, adjust of minimum RTO solved the problem and SCTP showed performance close to TCPM in lossy network scenarios. This weakens the hypothesis of Linux TCP congestion control being inherently better than LK-SCTP's.

Still, some doubt remains about a possible instability of SCTP congestion control (which would change its behaviour randomly), or about some mistake committed during our first tests.

If SCTP congestion control has no bugs, still the LK-SCTP implementation should consider lowering default RTO parameters, since most users never try to do any network tuning, would simply blame LK-SCTP, or SCTP itself, when faced with low performance.