Platform Engineering: What is it and why the recent hype?
Hint: It's important & valuable yet nothing new under the sun
It’s hard for a day to go by nowadays with reading an article or a Tweet about Platform Engineering. Is the hype warranted? Is Platform Engineering really novel, or is it a rebranding effort for a set of activities that are decades old?
What is Platform Engineering?
Perhaps a good place to start is to try and define what Platform Engineering is. I sampled a few videos and links that I found on Twitter and through a simple Google search below.
“Platform engineering is a process organizations can use to leverage their cloud platform as efficiently as possible so that engineers can deliver value to production quickly and reliably.” Source: liatrio.com
“Platform engineering is the concept of a team that works to design, develop, implement, as well as manage technical platforms the organization relies on. The services and tools put in place by the platform engineering team allow the organization to do things such as run applications, store data, and more. “ Source: StackSpot.com
“Platform engineering is an emerging discipline focused on enhancing developer productivity by reducing the complexity and uncertainty of modern software delivery.” Source: CircleCI
The latter is the definition that resonates the most with me. Platform Engineering at it’s heart encompasses the set of activities that enhance developer productivity. This in turn means that developers are the customers of a Platform Engineering team. That is a very important role, especially in this new regime of belt tightening and not defaulting to simply hiring more engineers.
What do Platform Engineering teams do?
Great, so if we agree that increasing the productivity of engineers is the raison d'etre of a Platform Engineering then, what are the set of activities that Platform Engineering teams concern themselves with? I’ll relay ones from my own experience, with (recency) bias towards what StrongDM’s Platform Engineering team does.
Build system
I’ve come across various surveys and statistics that show that the average software developer spends 30-40% of their time writing code. The rest is waiting for builds to complete :) I jest on the latter, but it’s fair to say that accelerating the time it takes to build your code base is one sure way of reducing idle time. Platform Teams spend cycles improving the build system
Testing
Platform Teams, at least ones I have encountered, do not necessarily own writing tests. That should be part of the software development cycle. They do however, spend cycles building common test automation frameworks and test fixtures. The intent here is to facilitate the development and reliability of tests: unit, integration, performance, stress, chaos..etc. Facilitating the development of tests ensures that software development teams are able to write tests with ease and by applying consistent patterns. Reliability provides the guarantee that a failing test is a strong indication of a software problem versus some flaky test environment. You want your engineering teams to write more tests and to spend time debugging failing tests to improve the quality of your product versus chasing a flaky test environment. The former is productive, the latter utterly frustrating. Another test related activity is providing the ability to easily and efficiently create dev/test environments. These environments could be entirely hosted on a developer’s machine, in the cloud,, on a device..etc.
CI/CD
I covered two critical elements of a CI/CD system in the earlier sections: building and testing. Platform Teams own the entire CI/CD pipeline, all the way to releasing assets to production.
Tooling & dev ergonomics
Linters, debug extensions, IDE extensions, observability dashboards, refactoring hard to maintain code, updating dependencies like third party libraries and languages.
Profile and overlap with {Dev}Ops
The best Platform Engineering teams that I have encountered tend to be staffed from the same archetype of engineers as those working on the product. In fact, almost all members of these top performing Platform teams had members who are software developers on the core product. Going through the experience of building (literally) your products, experiencing the same friction that the rest of the developers encounter, provides much needed context and empathy. Your Platform This point is further exacerbated by the idiosyncratic nature of software development. The build system at your company is not the same as mine. Hence idiosyncratic knowledge is of great importance.
Platform teams are often conflated with DevOps/ Infrastructure teams. These are two separate yet overlapping concerns. The table below illustrates how I view these two teams, in terms of responsibilities and core skills.
* This isn’t entirely true. OPs Teams work closely with the software development teams too, but their main responsibility, especially for SaaS companies, is to manage and maintain the infrastructure that powers the products your customers use.
* * There’s more (a lot more) to developer happiness than what a Platform Team alone can do, but their existence and success does help with this noble effort.
Why the hype?
I’ve been exposed to what Platform Engineering teams do since my earliest days in software (circa the turn of the century 🙂) The teams doing what Platform Engineering teams to today were called “The Build Team”, “Infra”, “OPs”. To be fair some of what Platform Teams do today fell outside the above teams and was oftentimes picked up by other engineers. But the work and set of activities, ultimately got done. So why the current hype then?
I’ve been in the software industry long enough now to have witnessed tens of buzzwords and hype spanning products/ tools and processes. Big Data, SRE, DevOps, DevSecOps, DataOps, BizOs, RevOps, Os everything. I know, I jest, but you get the point. Maybe this is anther example of a new buzzword taking off?
Perhaps as an industry matures it tends to over specialize? I recall a joke being made about the over-specialization in the medical field. We now have physicians specializing in the right ear and others the left. Maybe the software industry is also headed into this (over)specialized direction? Specialization gives a sense of identity. More importantly, it allows for the creation of new categories of tools, training and skills. Those create new markets and new opportunities. We’ll see if the same applies here.
Don’t get me wrong, I believe that having a Platform Team (I do @ StrongDM) or at least solving for the problems these teams provide is a must-have. There’s huge value in solving these problems. I just don’t get why this is such a novelty now, when these problems, and how to solve them, have been around for decades.
A few things I enjoyed reading this week
Benn puts together a well thought out argument about the potential bundling of various data products into a data conglomerate.
The (sad) State of Software Venture Capital. All the more reasons to absolutely try and avoid fund-raising, especially for series B+ now.
Engineering in a Hybrid World The data behind high-functioning engineering organizations
Well said Karim! It's a hype but also the focus is well placed by companies that have these. I have seen successful implementations recently and 'm a fan. Related to platform engineering, there is the concept of "Core Services", which are application engineers building the CORE of the platform. They often come to own core domain model from which different microservices and end user projects benefit. Would be nice to hear your thoughts on that and how it related to platform engineering, if at all.