Do you want to maintain high CSAT?
"It's the product, stupid"
CSAT, or its closely related NPS, are metrics that startups tend to measure over time. In my experience, which is entirely empirical and not scientific, the change in CSAT or NPS follows a pattern illustrated in the graph below.
My graph shows the relationship between CSAT and customer count, or more generally, CSAT relative to the growth of the startup. I highlight three distinct phases. An early, and very happy phase (1), followed by a nadir (2) in the middle and finally a fork in the road (3). My focus in this post is on the fork in the road - that third phase. I’ll offer my observations on why it happens and the actions that can result in either 3a or 3b.
Earlier, I had written a post on the Support & Engineering relationship. In that post I had alluded to that early phase and dubbed it “The Halcyon days”.
That in essence is why that phase tends to witness very high, arguably the highest CSAT or NPS the startup will witness. Your earliest customers are early adopters, they are die hard fans that decided to get onboard your startup journey at the earliest stages. The support you offer at that stage also tends to be personalized, curated and oftentimes from the founders directly.
The middle phase (2) is the messiest. It tends to witness the adoption of a different caliber of customers: laggards and enterprise. These customers have different expectations relative to the early adopters of your startup’s products. Rapid investments in scaling the support organization alongside others, including Customer Success, Professional Services and Technical Documentation tends to occur in this phase. All these efforts are attempts to address the gaps in the needs of this new breed of costumes and your startup’s abilities. These scaling efforts should - all else being equal - result in CSAT numbers getting better over time, until we hit that very interesting fork in the road (3).
The fork in the road
You’ve scaled your support organization, you’ve invested in technical documentation, offer professional services and yet you start noticing that your CSAT is starting to drop. You are slowly ascending down towards the asymptote of mediocrity (3b). Why is that?
“It’s the product, stupid”
More specifically, sliding down towards that asymptote of mediocrity is almost always indicative of quality issues with your product. Your support team, your technical content can no longer stem the tide of issues that your customers are facing. I have often observed that companies continue to double down on investing in increasing the size of the support organization to try and address these CSAT issues. Those, in my opinion, are the wrong investments. The investments should have been applied at the product level, mainly in better testing and addressing critical feature gaps. Below are the two largest sources of customer frustrations during that phase.
Regressions
You continue to release software, but every so often, a new release results in some customers suffering from broken functionality. This tends to happen due to inadequate end to end testing which is further exacerbated with the widening surface area of your product, especially for B2B software. Early on you might have tested your software on a few operating systems and handful dependencies. Now, you are faced with a Cambrian explosion in combinations of operating systems, network topologies, on-premises and cloud deployments, legacy applications and more.
Additionally, your testing will have to evolve from simple validations tests to ones that cover stress and performance. The latter are required not just to ensure that your product is able to work under high enterprise load, but to characterize how your product behaves under load.
The only way to address these issues is to invest, and do so as early as you can, in augmenting your test infrastructure. It might be impossible to cover 100% of the scenarios that your product will be deployed under, but you must cover the more common ones and always verify that your soon to be released software doesn’t regress those.
Lack of enterprise features
Early on your software - SaaS or on-premises - likely works in complete isolation. Your software is on its own isolated island, not integrating with anything else within the enterprise. Large customers will start pushing you towards integrating your software with an ever growing number of other enterprise applications they posses.
Your product doesn’t integrate with a SIEM? Well now it’s a must have for security and observability. AuthN and AuthZ now need to cover a long list of identity providers, multi-factor auth and more. Your assumption that Terraform is what every organization uses, well that’s wrong. You now need to support that and Pulumi and Chef and Puppet and the cloud idiosyncratic IaC. Then, there’s always that legacy financial services company that requires NIS & Nagios. Note, that every addition of one of these points of integration to your product will also result in requiring additional end to end tests. The Cambrian explosion continues.
One of the problems I notice during this phase, is startups defaulting to try and address these asks outside direct product investments. That Nagios integration will be put together by Professional Services and Support, who (and rightly so) are trying to keep their customers happy. That though introduces a challenge: how are these out-of-band “features” tested? They usually aren’t and thus result in more escalations and more frustrations. The best way to address these gaps is by prioritizing them and making direct investments in the product
The best way to ensure that you don’t descend into that asymptote of mediocrity is by shifting investments to the R&D organization. Continuing to invest in increasing the size of your support organization alone won’t stem this decline. You will frustrate your customers, your support team and your gross margins will suffer. Apply that pressure and necessary investments on the R&D side.
Articles that caught my eye
Mitchel Hashimoto’s article Growth of AI Through a Cloud Lense offered some insights into the similarities between the early days of cloud and AI.
Admittedly, I feel the hype around AI has a significantly broader social reach than cloud, so I think the time horizons on market maturity (if one develops) are shorter. But still, I predict at least many years of "open window" early-mover opportunity.
At the very least, I would caution against fully ignoring this one.