Bad Ideas Can Get Surprisingly Far

Stop giving things credence for getting traction

“TEED” talk image from StableDiffusion

In general, it’s not wrong to trust things more if they have social proof or market validation. “If people I trust believe this, it’s more likely to be true” and “if people paid money for this, it’s more likely to be useful” aren’t totally wrong.

On the other hand, it’s really not hard for an absolutely terrible idea to get enough market validation and social proof to look prima facie plausible.

Here’s one example, based on a number of actual things I’ve seen:

You hear about a new technological mini-industry: BUZZWORD for APPLICATION.

You think: “does BUZZWORD actually add value in APPLICATION? I don’t really see how it would. On the other hand, lots of the BUZZWORD-for-APPLICATION startups have customers who are big APPLICATION companies, and they’ve raised a lot of money, and surely somebody is vetting the technology to make sure it’s a good idea, so maybe they know something I don’t!"

But, okay, let’s take a step back.

The entire BUZZWORD-for-APPLICATION “industry” is only a few years old, and has a total revenue, let’s say, of $500M this year.

That’s nothing. Big companies in the APPLICATION industry can easily spend that much on a probably-useless technology just for value of information in case it turns out to be awesome.

What’s more, big companies in the APPLICATION industry have correlated sources of information. There are a handful of top management consulting firms and enterprise software firms that are used by basically all Fortune 500 companies. If one of those firms recommends or partners with a BUZZWORD-for-APPLICATION startup, there’s a good chance that startup will be able to honestly claim endorsement or use by many Fortune 500 companies, without those companies having independently and individually vetted its technology.

And all of the major companies in the APPLICATION industry send their executives to the same conferences where “thought leaders” talk about the Future of the APPLICATION Industry, and the importance of integrating BUZZWORD into your operations.

And yes, executives take that sort of thing seriously. They’re not stupid; they know it’s marketing and includes a fair bit of puffery. But they’re also short of time, not familiar with the relevant technical fields, and know that they will need to adopt new technologies too quickly to personally understand them on a deep level. Additionally, the social proof of a technology being “buzzword-y” makes it more likely that other stakeholders (who read the same white papers and go to the same conferences) will also support adoption; it’s a coordination mechanism. And that doesn’t even take into account irrational biases like the fact that people feel more comfortable with ideas they’ve heard before or seen their peers agree with.

Big APPLICATION companies will allocate budgets — sometimes billion-dollar budgets over periods of several years — towards various buzzword technologies. The executives in charge of spending those budgets have strong individual incentives to spend down the whole pot, otherwise they look like failures to their own bosses. This implies that the marginal BUZZWORD-for-APPLICATION product that gets purchased need not actually be useful or cost-effective; it may only be purchased because it fits in the remit of the budget category and helps pad out the spending.

Meanwhile, sometimes there are very large governmental subsidies for BUZZWORD, or even regulatory restrictions against not using BUZZWORD.

Yes, big APPLICATION companies will often have rigorous vetting processes for the technologies they use. In manufacturing it’s called a Factory Acceptance Test or FAT, and those are taken very seriously and they’re all about meeting objective technical specifications. On the other hand, there are often special isolated magisteria for pilot projects or other experimental things that aren’t expected to be really integrated into the main production process. Just because BUZZWORD Startup says that ACME MEGACORP is their customer doesn’t mean that their product is actually being used on a daily basis by ACME MEGACORP’s core activities — and therefore, it doesn’t mean that BUZZWORD Startup’s product has gone through a rigorous evaluation process.

Even if BUZZWORD for APPLICATION is objectively a terrible idea on the merits (not cost-effective without subsidies, not better than the technologies APPLICATION is already using, literally sits on a shelf and goes unused by its customers, etc), you should expect to see at least some BUZZWORD-for-APPLICATION startups raise several rounds of VC funding, get multiple Fortune 500 clients, make the news, and even IPO.

In the long run, of course, the market should penalize stupid ideas, and as far as I can tell, it usually does in most industries. But that can take more than a decade to shake out; in many cases, only after a BUZZWORD-for-APPLICATION company has gone public and short sellers notice that customers are no longer buying.

Meanwhile, the founders of successful BUZZWORD-for-APPLICATION companies are decamillionaires.

And you can find many knowledgeable, hardworking, honest technical experts working in the BUZZWORD-for-APPLICATION industry. If you ask them “so…is this even useful?” sometimes they will have a plausible-sounding rationale. (Sometimes they won’t even really have asked themselves that question, especially if it touches on issues outside their field.)

So, as an outsider, you can easily encounter individuals who are successful and admirable in many ways, and have based their careers on the premise that BUZZWORD is useful for APPLICATION…even when that’s 100% false.

I’m not saying this situation is a scandal that needs to be changed. There will always be some misallocation of resources in any economic system. It’s sometimes the correct strategy to spend resources on ideas that have a high chance of failure, because the rewards of success are sufficiently great.

I personally expect that there are usually some intentional scammers who get into the BUZZWORD-for-APPLICATION business knowing it’s not providing real value and intending to cash in cynically, but the dynamics I’m talking about can happen even if everyone involved is a sincere believer in BUZZWORD for APPLICATION.

But common human social proof heuristics like “I talked to a guy and he seemed smart and good and he does this for a living” totally fail to function relative to the vastness of the economy. There is room for you to meet hundreds of guys over the course of a few years, all of whom seem smart and good and all of whom do something for a living that will, in the fullness of time, turn out to be of no practical value.

For a long time I failed to completely integrate this into how I saw the world.

One mistake I made was a sort of “laziness heuristic.” My intuition was like:

People are lazy; nobody likes working very hard. Nobody would bother to go to all the trouble and effort involved in starting a company or working long hours at a job if the whole premise was faulty and BUZZWORD was not in fact useful for APPLICATION. Wouldn’t people get demoralized and quit? Why put so much work into a doomed enterprise?

This is, empirically, just not how everyone works. There are a lot of high-energy, career-motivated people who are willing to do tons of work if they expect to get paid or credentialed well for it, and aren’t very concerned with questioning whether their entire field is based on faulty assumptions or whether the technology they work on makes (economic or technical) sense. In fact, maybe that kind of “tunnel vision” actually makes you better at accumulating legible career accomplishments.

Another mistake I made was a sort of implicit “duty to be humble", which goes something like:

I shouldn’t judge someone to be wrong unless I can match all their accomplishments, skills, and virtues.

It sounds silly once I say it explicitly, but we’ve all heard admonitions like “It’s easy to be a critic” or “don’t confuse your Google search with my degree”. Disagreeing with someone is often perceived as setting oneself “above” them, and it would be obviously unfair for me to consider myself somehow “superior” to people who know more about their fields than I do, or who are more generally worthy of admiration than me.

Nevertheless, the fact remains… most people, including my “superiors”, should be expected to be wrong about lots of things. The world is vast, and being right is really hard. And disagreement need not imply personal contempt or a claim of personal superiority.

So… think about it in your own life.

What ideas are you giving the benefit of the doubt because they have “traction”?

Where are you giving ideas more credit when they come from well-funded or big-corporation-associated institutions than you would if just heard them from “some dude” with a cartoon avatar?

How easy would it be for a terrible idea to produce the signals of “traction” that you’re seeing? If the answer is “pretty darn easy”, then the traction should not reassure you!