53. Scaling Telecommunications Data with a Service Mesh

Julián Duque is a senior developer advocate at the Heroku, and he's interviewing Luca Maraschi, a chief architect at TELUS Digital. TELUS is a large communications company in Canada, which processes a massive amount of data produced by millions of customers all across the country. One of their challenges, for example, was to design a single datastore which provided a consistent experience across online services and offline ones, such as call centers or brick-and-mortar stores. In the end, Luca describes how they were able to break apart the existing monolith in favor many different microservices.

Luca notes that the data sources are not uniform; they can be coming from Excel files, Cassandra clusters, PostgresQL, MongoDB, and so on. He talks about the technologies they used to accomplish this feat, settling on GraphQL as a main component to the stack, as their services already act as edges in a graph. TELUS also makes use of Kuma and Kong, two relatively new projects. Luca finds new technologies to be not something to necessarily be afraid of, but rather, because they are often open source software, to make use of them as opportunities for driving the project's decision.

The conversation wraps up with a discussion on the future of distributed systems. In the end, Luca believes that more responsibility for data manipulation should be placed on the client, rather the server identifying what it believes to be the "perfect" data set. This allows for greater flexibility, and opportunities for changing backend strategies as needs evolve, without user-facing disruption.

Links from this episode

• Luca's talk at NodeConfEU 2019 titled "Scaling data from the lake to the mesh via OSS"
• Kuma (which ties microservices together) and Kong (which provides an API layer for them) are two technologies which Luca loves to use for TELUS' needs
• GraphQL helps TELUS with its goal for providing data as nodes in a graph