BigBlueButton – The Terabyte Week

Image: data traffic over 3 daysAn interesting piece of data when operating a video conferencing server is how much data is transferred over the course of a week or a month. When I initially calculated the potential amount of data my BigBlueButton server could potentially consume over the course of a month with 70 people in 4 sessions all showing their videos, 10 hours a day, 5 days a week, I came up with around 25 TB per month. That would have been all right since my virtual server rental contains 20 TB of traffic a month, and each additional TB is billed at a euro or two. But that was the theory, how about practice?

After two weeks of operation I have some actual numbers. As not all of the 70 people were in video calls all the time and not all people show their videos for a number of reasons, the real world number is significantly lower. Also, I spun up a second server that was configured as ‘moderator-only’ video for video conferences in which people with old and restricted devices participate that can’t handle 20 simultaneous video streams. In this setup, only the moderator sees all videos while all participants only see the moderator’s video but not those of other participants.

So in this mixed setup, data traffic over the week was 1 TB, or around 5 TB a month. That’s quite far away from my initial theoretical worst case calculation. Still, an incredible number!

The screenshot on the left shows incoming/outgoing traffic of both servers on Wednesday afternoon, i.e. about halfway through the week. The values on the left is the N:N video setup while the values on the left are from the 1:N video setup. As you can see the ratio between outgoing and incoming is 362 GB out / 52 GB in = 6.9 for N:N video. That means that each incoming video was streamed out again to an average of 6.9 participants. In practice, I noticed that even when 20 videos were steamed to a client, the client’s downlink data rate was around 3.5 Mbit/s. For 20 original videos at 300 kbit/s that should have been twice that rate. So perhaps, some compression is done on the server? If that is the case, then the real ratio would have been 14 streams outgoing for each incoming stream.

For the 1:N video setup the ratio between outgoing and incoming traffic was 57/29 = 1,9 for 1:N video. That means each incoming video was streamed to 1.9 other people. This is because the moderator sees all videos while participants only see one video. Also, CPU load on this server per participant is less than 50% of the load on the N:N video server. A significant difference!