Will the statisticians inheirit the earth?

Most good ideas are readily apparent long before their time. As a result, visionaries usually fail, and, once the idea's time has come, there is a long record of the idea failing in the world, which discredits the idea. In other words, the people who succeed with an idea are often not particularly gifted, but merely too young or too uneducated to be familiar with the idea's long history of failure.

The secret behind Google's explosive growth has been the confluence of the scientific method, large databases and massive computational power. The Googlers quickly recognized that, in the "information age", all information has value, and they leveraged their market dominance by using the statistically significant samples generated by that dominance to improve their algorithms. Now they have reached the point in the growth curve where their own success has attracted competition, but they imagine that their hegemony will continue through sheer momentum.

But the word is out that large databases are cheap to build and easy to mine. I can have a multi-terrabyte database in my living room that encompasses almost all known data relating to several months of my industry, built for a net outlay of less than twenty thousand dollars. Is my neighborhood filled with multi-terrabyte databases, quietly waiting to take over the world? Or does Google have a monopoly on intelligence and ambition?

Nothing lasts forever, and capitalism is very good at allocating resources to address conditions of scarcity. Currently, an extreme scarcity of intelligence in human species, with the minute genetic aberrations that foster tiny differences in intellectual capacity creating massive differences in social remuneration, has put smart people in the cross-hairs of the economic system. And once intelligence has been automated, most of the people who are strutting around defending the current social order will be thrust back into the seething masses, and the idea that a business has an adantage because it has "smart" employees will be as quaint as the idea that a restaurant has an advantage because it has a toaster.

But one should not underestimate the massive stupification that has taken place in this country over the last few years. Google has hired smart people, but they have also succeeded in making the entire country stupider. Google's engines have a bias twoards bloggers who write in simple, declarative prose with clear titles and plenty of unnecessary links. A post filed under "Sarbanes Oxley Sucks", with frivolous links to various sites, discussing the degree to which Sarbanes-Oxley undermines traditional center-left issues, will get five times the traffic as an identical -- linkless -- post entitled "Let's put the donkey before the Ox". The net result is that simple thoughts are rewarded, and the circumlocutory convolutions that might require a pleasant exertion of the mind languish in obscurity.