Forget the matrix, welcome to the grid

Moore¿s Law, formulated by Gordon Moore of Intel, states that data density on silicon chips will double every 18 months. This has held remarkably true for more than 20 years and is expected to do so for at least another decade. This exponential increase in chip complexity has a direct relationship to computing power, which also doubles every 18 months. There are similar things happening in computer storage and network speed. A few years ago, 200MB hard disk drives seemingly offered enormous storage, and yet today 40GB seems to be the norm. Storage space doubles about every 12 months.
Wide Area Network (WAN) speeds in 1985 were around 56kb/s between major academic institutions; now they run at 40GB/s ¿ an improvement of six orders of magnitude in 17 years. Network speeds double about every nine months. The mathematical amongst you will see that communication rate improvement increases by one order of magnitude every three years, storage every four years, while computing power takes five years to do the same thing. Networks are becoming so fast and readily available, via the Internet for instance, that they are now effectively free.
If you are running Windows 2000, take a look at the task manager. This shows you how much CPU time and memory you are currently using. It may not surprise you to know that running Outlook, Word and Excel at the same time is not really taxing your high-speed CPU. In fact these tasks only use between 3 per cent and 17 per cent of your CPU ¿ the mysterious ¿system idle process¿ uses the rest. Your computer is twiddling its virtual thumbs up to 97 per cent of the time even when you are actively using it. If you look at your hard disk drive, you may find that more than 50 per cent of its space is unused. As network speeds increase it will become possible to exploit this spare storage and free CPU time using pooled computational systems. When WANs give us the speed, it won¿t matter where the computers reside: they can be anywhere in the same building, the same city or eventually anywhere on the planet. Imagine the possibilities.
A methodology is needed for making all of these machines act as a cohesive whole. The goal is to create the framework and software tools to make vast numbers of computers, of every type and operating system, act as if they are a huge distributed supercomputer. There has already been considerable success, and a number of free and commercially available systems have been developed.
Grid technology
Distributed computing networks are called grid technology, using the electrical distribution network grid as an analogy. Just as you can plug into the electricity grid anywhere in the country, so you will be able to connect to a world-wide computer network and access limitless storage and processing power.
At least that¿s the theory. In reality there are numerous issues, such as security and compatibility between operating systems. It has been said that the political problems are far more difficult to solve than the technical ones. You may well ask what anyone would do with this huge computing horsepower. The US military is unlikely to run nuclear detonation simulations on desktop computers sitting in Beijing.
An oft-quoted example is the CERN European Laboratory for Particle Physics in Switzerland. When the new supercollider goes on line in the next few years it is expected to generate 100MB/s of data, 24 hours a day, 365 days per year. A single experiment may generate a petabyte (10,000 gigabytes) of data, which is roughly 20 million four-drawer filing cabinets of text.
Storage may catch up, so that storing petabytes of data is feasible, but processing it is another thing. The world¿s fastest supercomputer would take hours just to open the file, let alone process it. What is required is computing power faster by at least an order of magnitude. But are we prepared to wait another five years for it to come along? Scientists propose that the CERN data should be made available to any researcher in the world who needs access to it, but that the data should reside in just one place. There¿s no point making local copies when the superfast network makes remote access faster than having it on your own storage system! Processing this data is a task that could easily be distributed to the grid of computers proposed.
The future
No one is quite sure when this will happen or even the uses for such a system. Some interesting examples include digitising almost all of Egypt¿s antiquities. Imagine digitising a single pyramid down to 1mm resolution, and you are thinking terabytes.
The problem isn¿t simply obtaining data ¿ it¿s knowing what is worth storing. In 16th century Holland, lock-keepers used to record the freezing of the canals in log books. In 18th century India very detailed weather information was kept along with crop records. Both of these data sets have proved immensely useful in climate change modelling today; but this is not why they were created. How can we know now what future generations would like us to store?
The best known example for distributed computing is Seti@Home. This application is a small program downloaded by millions of web-connected users that processes data from the Arecibo radio telescope to detect extra-terrestrials. This project uses up to 4 million home computers when they are idle, and has performed 1,219,000 years of equivalent data processing. This phenomenally successful application has spawned many imitators, mainly for non-profit research, such as AIDS treatments, though some are looking to achieve a commercial return.
Already grid-based systems are up and running for astronomy, earthquake research, genetics and nuclear physics. Just imagine the horsepower of thousands, or tens of thousands, of computers in parallel.
Simon Bunn -- e.nz magazine/engine

cohesive - zusammenhängend
considerable - erheblich, nennenswert
crop - Ernte
currently - gegenwärtig, im Moment
decade - Jahrzehnt
density - Dichte
digitise, to - digitalisieren
exploit, to - ausnutzen
extra-terrestrial - Außerirdischer
feasible - machbar, möglich
grid - Netz
idle - leerlaufend
increase - Steigerung
issue - Problem
let alone - geschweige denn
lock-keeper - Schleusenwärter
obtain, to - erhalten
order of magnitude - Zehnerpotenz
pool, to - zusammenschließen
quote, to - zitieren
remote - Fern-
return - Gewinn
sophisticated - hochentwickelt
spawn, to - hervorbringen
supercollider - Super-Teilchenbeschleuniger
tax, to - strapazieren, belasten
twiddle your thumbs, to - Däumchen drehen
vast - riesig