top of page

Notes from "How Innovation Works: And Why It Flourishes in Freedom"

These are my notes from "How Innovation Works: And Why It Flourishes in Freedom" by Matt Ridley. Matt is the author of one of my favorite books ever, "The Rational Optimist". In this book he discusses the ingredients of innovation, where it comes from, and what it has led to for us humans. He argues innovation happens gradually and is less of a phenomenon of individual invention, but rather a collective evolutionary process. This book is amazing and I'd highly recommend it.

"The things we rely on to make our lives prosperous are (all of them without exception) physical states far from thermodynamic equilibrium, and the world was brought, sometimes over long periods of time into these convenient configurations by energy conversion, the use of which reduced entropy in one corner of the universe, ours, and increased it by an even larger margin somewhere else. The more ordered and improbable our world becomes, the richer we become, and as a consequence them more disordered the universe becomes overall." - John Constable.

Innovation then, means finding new ways to apply energy to create improbable things and see them catch on. It means much more than invention, because the word implies developing and invention to the point where it catches on because it is sufficiently practical, affordable, reliable, and ubiquitous to be worth using.

Innovation happens when people are free to think, experiment, and speculate. It happens when people can trade with each other. It happens where people are relatively prosperous, not desperate. It is somewhat contagious. It needs investment.

The chief way innovation changes our lives is by enabling people to work for each other. The main theme of human history is that we have become steadily more specialized in what we produce and steadily more diversified in what we consume. We move away from precarious self-sufficiency to safer mutual interdependence. By concentrating on serving other people's needs for 40 hours a week, which we call a job, you can spend the other 72 hours (not counting 56 hours in bed) drawing upon the services provided to you by other people.

Perhaps the most important event in the history of human kind, I would argue, happened somewhere in northwest Europe, sometime around 1700, and was achieved by somebody or somebodies, probably French or English, but we may never know who. At the time nobody would have noticed or realized it's significance and innovation was anyway a little valued thing. There is confusion too about whose contribution among several candidates mattered most, and it was a gradual stumbling change with no eureka moment. These features are typical of innovation. The event I'm talking about is the first controlled conversion of heat to work - the key breakthrough that made the industrial revolution possible, if not inevitable, and hence led to the prosperity of the modern world and stupendous flowering of technology today.

Thorium is much more abundant than uranium, in can breed almost indefinitely by creating uranium-233, it can generate almost 100 times as much power from the same quantity of fuel, it does not give rise to fissile plutonium, and it generates less waste with a shorter half life.

Following the 3 Mile Island accident in 1979 and Chernobyl in 1986, activists and the publics demanded greater safety standards. They got them. According to one estimate, per unit of power, coal kills 2000 times as many people as nuclear, bioenergy 50 times, gas 40 times, hydro 15 times, solar 5 times (people fall off roofs installing panels), and even wind power kills nearly twice as many as nuclear. These number include the accidents at Chernobyl and Fukushima.

The stifling of innovation of nuclear power keeps older nuclear plants running longer, which increases the chance of an accident. If there were new nuclear power plants being built, the old ones would be phased out sooner, leading to greater safety.

Energy is the root of all innovation, if only because innovation is change, and change requires energy. Along came the heat to work transition of 1700 and suddenly it became possible to create ever more improbable and complex material structures from the harnessing of fossil fuels, with their huge energy yield on energy invested.

The story of penicillin reinforces the idea that even when a discovery is make by serendipitous good fortune, it takes a lot of practical work to turn it into a useful innovation.

The story of the internal combustion engine displays the usual features of an innovation. A long and deep prehistory characterized by failure, a short period marked by an improvement in affordability characterized by simultaneous patenting and rivalries, and a subsequent story of evolutionary improvement by trial and error.

Innovation in material is vital to realizing an advance that can be conceived but not built.

In America you are now more than 700 times more likely to die in a car per mile travelled than in a plane.

Desperate to get access to guano, the American congress passed an act saying that any American who found a guano island in the Pacific could claim it for the United States, which is why so many mid-Pacific atolls belong to America today.

If Harbor and Bosch had not achieved their near-impossible innovation (making plant fertilizer) the world would have plowed every possible acre, felled every forest, and drained every wetland, yet would be teetering on the brink of starvation.

Innovation in food production has spared land and forest from the plow, the cow, and the axe on a grand scale by increasing the productivity of the land we do farm. It turns out that this land-sparing is much better for biodiversity than land-sharing would have been, by which is meant growing crops at low yields in hopes that abundant wildlife lives in fields alongside crops.

Between 1960 and 2010 the acreage of land needed to produce a given quantity of food has declined by about 65%. Had this not happened, pretty well every acre of forest, wetland, and nature reserve in the world would have been cultivated or grazed, and the Amazon rainforest would have been far more severely destroyed. As it is, the area of wild lands and nature reserves is steadily increasing, while forest cover has stopped declining and in many places is now increasing, so that overall there has been a 7% increase in tree cover since 1982.

For a given yield of food, intensive agriculture not only uses less land, but produces fewer pollutants, causes less soil loss, and uses less water than organic or extensive systems.

The lesson of wheeled baggage is that you often cannot innovate before the world is ready, and that when the world is ready, the idea will be already out there waiting to be employed.

Innovation is a process of search and recombination of existing components.

The sharing economy is a form of more from less, or growth by shrinkage, economic enrichment caused by using resources more frugally.

The computer as we know it has four indispensable ingredients that distinguish it from a mere calculator:

  1. It must be digital, in particular, binary

  2. Electronic

  3. Programmable

  4. General purpose

The first machine to meet these requirement was the ENIAC which began operating in 1945 in the University of Pennsylvania.

There is a symbiotic relationship between hardware and software, as there is between cars and oil, each industry feeds the other with innovative demand and innovative supply.

Something can be inevitable in retrospect, and entirely mysterious in prospect.

The development of the search engine and social media follow the usual path of innovation:

  1. incremental

  2. gradual

  3. serendipitous

  4. inexorable

Few eureka moments or sudden breakthroughs.

Social media, far from ushering in a world of democratic utopian enlightenment, in which the world is flat, everybody is sharing, and we all see each other's point of view, in plunged us into a maze of echo chambers and filter bubbles in which we spend our time confirming our biases and railing against the opinions of others.

Left to their own devices, personalization filters serve up a kind of invisible auto-propoganda, indoctrinating us with our own ideas, amplifying our desires for things that are familiar, and leaving us oblivious to the dangers in the dark territory of the unknown.

True, a life of farming often proved to be one of drudgery and malnutrition for the poorest, but this was because the poorest were not dead. In hunter-gathering societies, those at the margins of society who were unfit because of injury or disease, simply died. Farming kept people alive long enough to raise offspring even if they were poor.

Dogs diverged from wolves about 40,000 years ago and subsequently split into two, eastern and western, branches of the dog family tree around 20,000 years ago. Chinese village dogs are distinctly different from European dogs after this date.

Innovation in the upper Paleolithic is all about demography. Dense populations inevitably spur technological change because the create the conditions in which people can specialize.

Sometime before 150,000 years ago, human beings had become reliant on a collective social brain, mediated through specialization and exchange. If you cut people off from exchange, you lower their chances of innovating.

Cooking predigests food. It gelatinizes starch, almost doubling its digestible energy. It denatures proteins, increasing the energy available from eating an egg or a steak by 40%. It's like having and external extra stomach. Cooking therefore explains why we have small teeth, small stomachs, and a gut that is a little over half as big as in other apes relative to our body weight. This small gut costs us less to run. 10% less energy is burned by people just by keeping the elementary canal alive compared with other apes. So cooking not only provides us with extra energy, it also saves us energy. Cooking released the possibility of bigger brains.

At the dawn of life there was an innovation by fortuitous recombination, and the result was a reduction in entropy through the harnessing of energy. Since that roughly designed civilization and technology too, there is a clear sense in which the invention of human innovation is just the continuation of a process that began 4 billion years ago.

Manmade technologies evolve from previous manmade technologies and are not invented from scratch. This is a key characteristic of evolutionary systems - a move to the adjacent possible step.

If innovation is a gradual evolutionary process, why is it so often described in terms of revolutions, heroic breakthroughs, and sudden enlightenment? Two answers. Human nature and the intellectual property system.

The most influential new technologies are often humble and cheap. Mere affordability often counts for more than the beguiling complexity of an organic robot.

It is the people who find ways to drive down the cost and simplify the product who make the biggest difference.

Some inventions come from unexpected events, and the ability to recognize these and use them to advantage.

Recombination is the principal source of innovation upon which natural selection draws when innovating biologically. Sex is the means by which most recombination happens. Innovation happens when ideas have sex.

Innovation involves trial and error. Tolerance of failure is critical.

Technology is absurdly predictable in retrospect, and wholly unpredictable in prospect. Thus, predictions of technological change nearly always look foolish. They either prove wildly overblown, or equally wildly underblown.

Amara's Law (named for Roy Amara) states that people tend to overestimate the impact of a technology in the short run, but to underestimate it in the long run.

For at least 1000 years innovation has disproportionately happened in cities.

A city's infrastructure scales at a sublinear rate, but the economic product of the city scales at a superliner rate. This fact is not true of companies. As they grow bigger beyond a certain point, they grow less efficient, less manageable, less innovative, less frugal, and less tolerant of eccentricity. That is why companies die all the time, but cities never do.

Innovation increasingly means using fewer resources rather than more. The bigger cities get, the more productive and efficient they become in terms of their use of energy to create improbability, just as the bodies of animals do.

Growth can take place by doing more with less.

In the 10 years since 2008, America's economy grew by 15% but its energy use fell by 2%. This is not because the American economy is generating fewer products - it's producing more. It is not because there is more recycling, though there is. It's because of economies and efficiencies created by innovation.

It will always be possible to raise living standards further by lowering the amount of a resource that is used to produce a given output. Growth is therefore indefinitely sustainable.

Knowledge is both a public good, and a temporarily private one. Knowledge is expensive to produce, but can sometimes pay for itself.

We must be aware of downplaying the development of technologies after they are first invented, a huge part of innovation, lest we credit a beaver with the Hoover Dam.

Innovations emerge by serendipity and the exchange of ideas, and are pushed, pulled, molded, transformed, and brought to life by people acting as individuals, firms, markets, and sometimes public servants.

The work that led to the invention of CRISPR gene editing was driven partly by a desire to solve practical problems in the yogurt industry. My point is that we make a mistake if we assume that science is always upstream of technology. Quite often scientific understanding comes from an attempt to explain a technical innovation.

To contribute to human welfare, and therefore catch on without subsidy, an innovation must meet two tests: it must be useful to individuals, and it must save time, energy, or money in the accomplishment of some task. Something that costs more to buy than existing device but offers no extra benefits will not thrive, no matter how ingenious.

The idea that innovation destroys jobs comes around in every generation. So far it has proved wrong. Over the past few centuries, productivity in agriculture significantly increased, but farm workers moved to cities and got jobs in manufacturing. Then productivity in manufacturing rocketed upwards, freeing huge numbers of people to work in services, yet still there was no sign of mass unemployment. Candles were replaced by electric lights, but wick trimmers found other work.

On the whole, society has decided to use the greater productivity provided by innovation to give everybody a lot more leisure.

Do you or does anyone else have absolutely everything you can even dream of desiring, which requires human work to deliver to you? No? Still short of a back rub or a peeled grape? Then there's still a task or two for humans to do.

Robots are never going to be without cost, if only because they need energy.

As Adam Smith pointed out, the purpose of production is consumption. The purpose of work is to earn enough to get the things you want. Enhanced productivity means enhanced ability to acquire the goods and services you need, and therefore enhanced demand for the work of those that supply them. It is only the high productivity, and therefore high spending power of the average modern worker that keeps restaurant chefs, and pet vets, and software experts, and personal trainers, and homeopaths in business.

Innovation frees people to do the things they truly value.

For innovation to flourish, it is vital to have an economy that encourages, or at least allows, outsiders, challengers, and disruptors to get a foothold. This means openness to competition, which historically is a surprisingly rare feature of most societies.

Because it inhibits an enzyme found only in plants, gylyphosate (Round Up), it is virtually harmless at normal doses to animals, including people. And because it decomposes rapidly, it does not persist in the environment. It is far safer than the stuff it replaced, paraquot, which was sometimes used by suicides. Glyphosate has transformed agriculture by allowing farmers to control weeds chemically rather than by the more ecologically harmful activity of plowing. It has led to the no-till revolution. This is especially true where crops genetically modified to be gylphosate resistant are grown.

Patents tend to favor inventions, not innovations, upstream discoveries of principles rather than downstream adaptations of devices to the market. This has led to the proliferation of patent thickets - vague intellectual property hedges that block the progress of people trying to move through the intellectual landscape and develop new products.

The economist William Baumol has argued that if the policy background means that the best way to get rich is by building a new device and selling it, then entrepreneurial energy will flow into innovation. But if it is simpler to profit by lobbying government, to set the rules up to favor an existing technology, then all the entrepreneurial energy will go into lobbying.

A key concept in the study of innovation is Baumol's cost disease. This is the economist William Baumol's realization that innovation in one sector can cause an increase in the cost of products or services in another sector if the latter experiences less innovation. If innovation transforms the labor of manufacturing, then that will drive up salaries throughout the economy, making services more expensive.

The main ingredient in the secret sauce that leads to innovation is freedom. Freedom to exchange, experiment, imagine, invest, and fail. Freedom from expropriation or restriction by chiefs, priests, and thieves. Freedom on the part of consumers to reward the innovations they like and reject the ones they do not.

Innovation is the child of freedom because it is a free creative attempt to satisfy freely expressed human desires. Innovative societies are free societies, where people are free to express their wishes, and seek the satisfaction of those wishes, and where creative minds are free to experiment, to find ways to supply those requests so long as they do not harm others.

Growth increasingly means getting more benefits from fewer resources.

Compliance with regulations almost always hits smaller companies harder, proportionally, than big ones, thus deterring new entrants with new ideas from entering existing markets.

Things that require five different apps in the west can be done on one app in China.

What explains this speed and breadth of innovation fury? In a word, work. Chinese entrepreneurs are dedicated to the 9-9-6 week: 9am to 9pm 6 days a week. That is what Americans were like too when they changed the world.

Willingness to put in the hours, to experiment and play, to try new things, to take risks, these characteristics for some reason are found in young newly prosperous societies and no longer in old tired ones.

Innovation happens when ideas can meet and mate, when experiment is encouraged, when people and goods can move freely, when money can flow rapidly towards fresh concepts, and when those who invest can be sure their rewards will not be stolen.

Innovation is the child of freedom and the parent of prosperity. It is, on balance, a very good thing. We abandon it at our peril. The peculiar fact that one species above all others has somehow got into the habit of rearranging the atoms and electrons of the world in such a way as to create new and thermodynamically improbable structures and ideas that are of practical use to the wellbeing of that species never ceases to amaze me. That many members of the species show little curiosity about how this rearranging comes about and why it matters, puzzles me. That many people think more about how to constrain rather than to encourage it, worries me. That there is no practical limit to the ways in which the species could rearrange the atoms and electrons of the world into improbable structures in the centuries and millennia that lie ahead, excites me. The future is thrilling, and it is the improbability drive of innovation that will take us there.


bottom of page