WebSockets on the Server

Chat must run on Courselore’s one server per our requirements. If that server cannot keep up, our only option is vertical scaling: moving it to a bigger machine. So can one server hold a WebSocket for every open client?

Every open client keeps one WebSocket to the server for as long as Courselore is open in the browser. At the busiest hour, per our load estimates, that is 600 open WebSockets. Our scalability requirement asks us to meet the latency target at twice the estimated load, which is 1,200 open WebSockets.

What an open WebSocket costs

Most of the time, a WebSocket carries no messages: Marcus is reading, and nobody in Marcus’s rooms is posting or typing.

Like a kept long poll, an open WebSocket that carries nothing costs the server memory and an open connection:

• Memory. The server keeps buffers for the data the connection sends and receives, and a record of the user the connection belongs to. That comes to anything from a few KB to a few tens of KB per connection. Even at 50 KB each, 1,200 WebSockets take 60 MB.
• A file descriptor. The operating system (on the server) counts each open connection as an open file, and it gives the process a number for each one, called a file descriptor. (As an aside, Linux allows a process 1,024 open files by default. That is fewer than 1,200, so we would have to change the default setting on Courselore’s server to meet our scalability requirement.)

What about CPU usage? An open WebSocket costs almost no processor time while nothing arrives on the connection; the server has no work to do for it.

One server is enough

So 1,200 WebSockets cost the server about 60 MB of memory and almost no processor time, and the only setting to change is the limit on open files. That is manageable for one server.

A single server with the right configuration can hold far more connections than Courselore needs. In 2015, for example, the developers of Phoenix, a web framework, held 2 million WebSocket connections on one server. Courselore’s 1,200 is less than a thousandth of that. So its one server is enough, with plenty of room to grow.