Flutter and WebRTC: reaching sub-300 ms mobile latency
What we had to check — beyond the WebRTC config — to get smooth peer-to-peer video on a phone.
WebRTC isn’t “just” a video stream. It’s a chain of layers where each can add a hundred milliseconds if you’re not careful. On VisiOne, the goal was to stay under 300 ms end to end in the documented test conditions. Here’s what mattered, in order.
Signaling can’t be a bottleneck
Our signaling used MQTT rather than a dedicated WebSocket. Good call — the camera already spoke MQTT. Bad initial reflex: sending SDP base64-encoded in the MQTT payload. The broker didn’t care, but the round-trip with application confirmation cost 40 to 60 ms. We moved SDP to a binary payload with QoS 1 and no application-level ack — and gained 30 ms of measurable time.
STUN when possible, TURN only when needed
TURN guarantees connectivity but imposes a relay. On the client’s networks, NAT was traversable in 80 % of cases. We put a STUN server first, TURN as fallback. On the 80 % STUN cases, end-to-end latency dropped by ~80 ms.
Codec: hardware H.264 or nothing
On Android, flutter_webrtc can use VP8 by default. On mid-range phones popular in Cameroon, decoding VP8 in software adds 20 % CPU and 40–60 ms of perceived latency. We forced H.264 with VP8 fallback only if the hardware decoder refused — verified via RTCRtpSender.getCapabilities.
Flutter rendering isn’t free
RTCVideoRenderer bridges native → Flutter texture. On mobile, that bridge costs one frame of latency unless you enable native SurfaceView rendering. We enabled it wherever possible, accepting the constraint: the texture can no longer be composed pixel-perfect with other Flutter widgets. In practice, we displayed the stream fullscreen or in a fixed container — that was fine.
What didn’t work
- lowering resolution below 480p: CPU load went up paradoxically because of screen upscaling;
- disabling echo cancellation “to save time”: no measurable effect;
- using a Chrome simulator for end-to-end latency measurement — its numbers were systematically 60 ms too optimistic.
The method that worked
Instrumentation from the first prototype. We embedded a timestamp in each video frame (via SEI metadata) and read it at display time. A timeline with 6 checkpoints: capture, encode, signaling out, receive, decode, render. Each optimisation was measured. Without that timeline, we would have “felt” the stream was smoother without knowing why.
The final number — < 300 ms — is not a demo performance. It’s the result of a budget held tight at every layer.