There’s a fine line between the “reality distortion field” founders create and just being a bit delusional. During our first year and half as a company — May 2011 to December 2012 — we were closer to delusional, and this massively extended how long it took us to find product-market fit.
In particular, I so desperately wanted an idea to “work” that I just forced ideas through and never stopped to deeply question whether people really, really had this problem. Eventually though, in December 2012, we had an internal moment of crisis and actually did verify that people had the problem we were solving. At that moment, everything changed, and today Segment is growing quickly, with 6,500 customers and a team of 70 people.
I hope that by sharing our story of finding product market fit, other founders can avoid years of “wandering around lost in the woods” like we did, and emerge with product-market fit sooner.
Lost in the Woods
Y Combinator’s constant, sage advice is to avoid solving problems that nobody has. It’s the most common startup failure mode, and it was certainly our failure mode for that first year and half.
Here are the three ideas I pushed us to build, all the while convincing ourselves that these were real problems people had:
Bookxor was a tool for students to read and annotate lecture notes. In return, professors could see usage and readership analytics on their course notes. Six months went by at MIT before we realized that professors didn’t really care whether people read the notes.
ClassMetric was a tool for students to press a button to anonymously say “I’m confused!” The professor would get a graph of how confused their students were over time. We spent ~6 months at MIT and at Y Combinator in 2011 building this before realizing that (1) professors often don’t really care how good their lectures are, (2) professors can’t respond to the feedback in real-time anyways, so it only helps them improve next semester (slooooowww), and (3) the students all just went to Facebook and Gmail on their devices anyway.
Segment.io was originally an analytics tool like Google Analytics, except focused entirely on user segmentation (Segment is now a very different product). We spent ~12 months after Y Combinator in 2012 building this segmentation product before realizing that the market for analytics tools was incredibly crowded, and that we didn’t actually understand what problems our customers were trying to solve with more charts and analysis.
In each of these cases, I deluded myself and my co-founders into believing that professors really cared about students reading their lecture notes, that professors wanted deeply to improve their lectures (or that students even wanted to focus during lectures), or even that companies needed better segmentation in their analytics. In fact, all of these ideas were very poorly validated.
Good validation work would’ve looked something like this:
Hour-long interviews with professors and TAs diving into what they found most frustrating about teaching.
Digging and digging until we found a problem we could solve, or even more critically, possibly determining that there were no problems for us to solve.
As engineers who had never done this before, talking to people didn’t seem like real work. Real work was coding. But in reality, 20 hours of great interviews probably would’ve saved us an accrued 18 months of building useless stuff.
Put another way… we were making enormous errors in the “Conception” phase (“ConOps” in the diagram below), but we weren’t correcting it until after we’d done a staggering amount of engineering effort all the way at the right.
If you think about odds of a team failing to find product-market fit, it might look something like this (for the sake of illustration only):
Non-technical team: 15% odds of being able to build it, 60% odds of solving a real problem
Technical team: 90% odds of being able to build it, 10% odds of solving a real problem
If you’re a technical founding team like we were, building it is the least useful way to reduce the risk in the company. You should be focusing the vast, vast majority of your effort as a technical founding team on making sure you’re really solving a problem. Because if you are, I’m quite sure you can build the solution.
That said, a really odd thing happens when you solve your own problem: you rarely mess up in finding a real problem! (Almost tautologically.) In the diagram above, you’re unlikely to screw up the Concept or Requirements phase because you already know what the problem is and what you need to solve it.
And this is how Segment as it stands today was discovered. (Slack, Dropbox and all kinds of other companies have similar stories.)
Accidentally Solving a Real Problem
Let’s rewind to the early days of ClassMetric, at the very beginning of Y Combinator in June 2011. We were just starting to build our lecture tool (for students to say “I’m confused!”) and we figured we should have analytics on it. So we googled “analytics” and found Google Analytics, KISSmetrics and Mixpanel. Their APIs for collecting data and the graphs that each of them provided looked similar… we just couldn’t make up our minds about which one to use. So we wrote a pretty gnarly abstraction that just took an event like “Signed Up” and sent it out to all three analytics services.
Then, a few months later when we were hacking on a music project called Ethertunes, we put our lightweight little abstraction into its own little javascript library, and cleaned it up a bit.
Six months later we were deep into building Segment.io (the analytics service), and we had another idea. People kept telling us they already had Google Analytics installed, so how was Segment.io different? We really wanted to show our analytics service in direct comparison to Google Analytics. So my co-founder Ilya cleaned up our javascript library a bit more, gave it a name (“analytics.js”), added Segment.io as the fourth analytics service it could send data to, and open-sourced it on Github so that prospective customers could give it a spin. Analytics.js started getting some stars on Github, and even a couple pull requests, which was mind-blowing to us at the time.
By December 2012, after a year and half of working on made-up problems full-time (with analytics.js just a side project in fits and starts), we were starting to run out of money. The four of us hotly debated whether we should shut down our analytics service (which had no real customers), try a totally new idea, or just throw in the towel. Then my co-founder Ian came out of left field and said: “You know, I think there’s a really big business behind analytics.js. We should drop all this other stuff we’re working on and focus on that.” I remember very vividly thinking (and probably saying), “That’s a terrible idea. This is a javascript library that’s already open source, it’s about five hundred lines of code… how could you conceivably build a business around that?” But by this time Calvin and Ian had drafted a “manifesto” about the future possibilities that lay beyond analytics.js (surprisingly accurate in retrospect!)
I felt so strongly skeptical that I said something to the effect of, “Alright, before we can go ahead with this, we need some sort of proof that there’s a real problem here.” We decided to build a landing page that just explained analytics.js and the data-routing problem it solved. Then we put the page up on Hacker News. I was expecting it to be completely ignored, and that we’d move on and try something else.
But to my great surprise, analytics.js went straight to the top of Hacker News and got about a thousand stars on Github in that first day.
The skepticism and staunch disagreement between co-founders had forced an extraordinarily clarifying test. We already knew analytics.js solved a problem for ourselves, but after the HN launch, it was obvious analytics.js was solving a real problem for other people too. Rather than months and months of engineering effort, all we needed in the end was a landing page built over a couple days. That little open source library (with only 8 incomplete integrations) was our minimum viable product:
Or, another way of looking at it… we barely had a skateboard version of the product:
But it was a proof point none-the-less. So with that little win in hand, we started expanding scope to solve more of the problem we’d accidentally discovered. Over the next 6 months we added ~40 more integrations, the ability to send data from server-side languages like Node and Ruby, and built libraries to collect data in mobile apps on iOS and Android.
More recently we launched the ability to easily load all your web and mobile data into a data warehouse. It’s still very much early days at Segment, and there’s an incredibly large set of problems to solve for our customers. But once we found that initial sliver of a problem, digging in has been incredibly rewarding, fun and challenging in new ways.
My advice from being woefully “lost in the woods” is that if you’re not 100% sure you’re solving a real problem, you’re almost certainly not. Make sure you really validate the problem you’re solving. You’d be batshit crazy not to. It may be an utterly bizarre juxtaposition, but you need to balance your reality distortion field with extreme skepticism to ensure you’re not deluding yourself and moving in a useless direction. Best of luck!