Project 04 of 39 · AI systems · Flagship
CongestIQ
Multi-agent reinforcement learning for adaptive congestion control. Several flows learn to share a bottleneck under centralised training and decentralised execution, with a cooperative reward that charges queueing delay, loss and window growth so no agent simply fills the buffer. The policies are developed against ns-3 and benchmarked against Reno, CUBIC and BBR on the same two-flow dumbbell topology.
01Overview
A simulation-based research platform asking whether a learned policy can beat the standard congestion control algorithms. Each flow is an agent that adjusts its congestion window every 100 milliseconds, choosing from five discrete actions, trained together but executed independently with no shared state at decision time.
The work happens in simulation rather than against a live kernel, for three deliberate reasons: an untrained policy can crash a real network, a simulator gives byte-identical reproducibility, and training runs in accelerated time instead of at real acknowledgement intervals.
02How it is built
- The first multi-agent evaluation matched CUBIC's numbers exactly, which was the tell rather than the result. The no-op congestion control had been configured before the transport protocol object existed, so the configuration call matched nothing and every socket quietly ran the default CUBIC. The corrected environment attaches the override after the internet stack installs and aborts on a miss.
- Two earlier bugs surfaced the same way. The environment reported throughput as the sum of in-flight byte snapshots rather than actual delivery at the sink, and the action was applied as an absolute window value instead of a multiplier. Every reward function had been failing to push past roughly 200 kbps because of them; after the fix the same setup reached 8,565 kbps.
- The reward is cooperative and charged: throughput and fairness are rewarded while queueing delay, loss and window growth are penalised, which is what stops both agents from settling into a permanently full buffer.
- The final policy came from an imitation warm start followed by PPO. On a two-flow dumbbell it holds the window near the bandwidth-delay product: 8,696 kbps, 22.80 ms delay, zero measured loss and a Jain fairness index of 1.0000.
- The baselines were re-run on the same topology rather than quoted from literature. CUBIC wins on raw throughput at 5,331 kbps against Reno's 4,611, and both pay for it with roughly 61 ms of bufferbloat on a 20 ms path. The learned policy uses 87 percent of the link at a third of CUBIC's delay, and the write-up states plainly that it sits below CUBIC on throughput.
- Fairness and latency are measured rather than asserted: Jain's index, per-flow delay and loss come out of the simulator's own flow monitor and are plotted per milestone. Each of the six milestone folders carries its code, its results and its experiment log.
- Every milestone has a written as-built narrative recording what actually happened, including the parts that were wrong, alongside a walkthrough video and the final report and defence material.