On getting it wrong

On getting it wrong

Shell International has a tradition of informing itself of technological change which may affect its operations decades in the future. Danny Hillis came to Shell Centre in around 1985, to present to the Shell Group about the Connection Machine. Fujitsu and Cray also gave us their foresight on the future of supercomputation. Virtually all of these talks started with slides that mocked past predictions - Thomas Watson expecting the world to need five computers, or the head of the British post office first damning telephones and then, a few years later, suggested that every woman in England would soon be a telephone operator. The slides that followed could, however, be themselves used to damn prediction.

The URL www.top500.org surveys the current supercomputing capability. In 1993, the Connection Machine and its peers made up 7% of the top 500 machines. Machines like the Cray made up 27%, whilst the rest were dedicated processors which broke codes or predicted the weather as a sole task. Today, these specialists are down to 7%, the parallel generalists are extinct and almost the entire capability consists of networks of conventional workstations. These are called 'edge of network' solutions. As an example, the volunteer SETI@home project deploys around 13 teraflops (million million floating point operations per second.) It is safe to say that absolutely nobody foresaw this.

It is equally safe to say that predictions about the utility of such networks was, however, spot on. We use them for almost exactly the kinds of application that we had expected: seismic, weather, CAD, protein folding, entertainment. This is interesting: scenarios often get technology utterly wrong, but get social systems and people more or less right. Is it that we know what determines us better than we understand that which drives technology, or is it that we drive towards human-related goals, and not towards the technical means of attaining those goals? I suggest that the latter force is the predominant factor.

If this is so, then business rents may derive from identifying bottlenecks on the path to generally desired outcomes, which we can foresee with some confidence. Of itself, this is not a very interesting insight, for we can indeed be confident that people will continue to want to heat their homes with cheap, clean technologies. However, this does not get us very far. However, it may be possible to identify technical or regulatory bottlenecks with more confidence than we can pick technological winners.

For example, it turns out that the limiting factor in the 'edge of network' solution to supercomputing is synchronisation and intercommunication. The co-ordination of SETI takes a considerable amount of the entire bandwidth of the University of California, which co-ordinates it. The solution to this is, it seems, identical to the problems of co-ordination in a knowledge network: to have synchronizing 'domain-spanning' entities - people, or in this case, software - which note connections which particularly need to interact, or which could pool their resource, and which the facilitate the communications needed to undertake this. If one looks at what major organisations are doing with outsourcing, one sees that this is precisely the model that is emerging.