Uber, Netflix, and Segment

Uber

Uber launched in San Francisco in 2010, with one kind of ride in one city. Around 2014, its backend was mostly two large services. One, called dispatch, handled the live work of a ride: drivers going online, riders requesting rides, and drivers accepting or rejecting them. The other, called api, kept the users and the trips. For a small team building one product, this was a sensible design. An engineer could follow a request through the code without crossing the network.

Then Uber added cities and products, and the number of engineers grew from tens to hundreds. More machines could handle the growing traffic, but they did not help with the releases. Unrelated changes went out together, a bug in one change could break the whole platform, and releases were often rolled back.

So Uber moved capabilities out of the two services into microservices, one at a time, while the original system kept serving rides. By September 2015, it had more than 500 services. It also built the tools it needed to run so many services: shared contracts, a way for services to find each other, and a shared way to deploy and monitor them.

By 2020, Uber had about 2,200 microservices, and it had new problems. One request could pass through a long chain of services. One feature could need changes from several teams, and teams sometimes changed another team’s service to finish their own work. These are the signs of a distributed monolith. Instead of merging the services back together, Uber grouped them into about 70 domains, such as fares and matching, and put one gateway in front of each domain. A team that needs fares calls the fares gateway and does not need to know the services inside it. Each domain is close to a coarse service in SOA. The microservices are inside it.

Netflix

In 2008, Netflix still mailed DVDs to its members. Its system ran in its own data center, with one large application and relational databases. In August 2008, a major database was corrupted, and for three days Netflix could not ship DVDs. The outage showed that if one database failed, the whole business stopped.

Netflix decided to move to the cloud, to AWS. Instead of copying the old system onto rented servers, it rebuilt almost all of its systems as hundreds of microservices. Each microservice ran on many instances, so losing one instance did not stop the service. Teams could also release on their own, instead of waiting for one coordinated release.

Netflix moved one part at a time. Most systems, including everything that members used, were in the cloud before 2015. Billing and the data about customers and employees moved last. In January 2016, about seven years after the outage, Netflix shut down the last parts of its data center that the streaming service used.

Segment

Segment collects events from its customers’ applications, such as a page view or a purchase, and forwards each event to the other tools the customer uses, such as an analytics tool. Segment calls each of these tools a destination, and it supported more than 100 of them. At first, one service delivered the events for every destination. When one destination was slow or down, Segment kept retrying its events, and those retries delayed delivery to every other destination.

To keep one destination from delaying the others, Segment gave each destination its own service. That fixed the delays. But the number of services grew with the number of destinations, and it reached more than 140. The destinations shared code, and a change to the shared code meant testing and deploying dozens of services, so different services ended up on different versions of it. Each service also needed its own scaling. Three full-time engineers spent most of their time keeping the system running. The team was small. So Segment had the costs of microservices, but it did not get their main benefit, which is that separate teams can release on their own.

In 2018, Segment built a new way to keep a slow destination from delaying the others, and merged all of the destinations back into one service. Now one engineer could deploy it in minutes. Segment also said what it gave up: a crash in one destination’s code can now take down the whole service.