Grid Computing and its Advantages

Grid computing has been around for over 12 years now and its advantages are many. Grid computing can be defined in many ways but for these discussions let's simply call it a way to execute compute jobs (e.g. perl scripts, database queries, etc.) across a distributed set of resources instead of one central resource. In the past most computing was done in silos or large SMP like boxes. Even today you'll still see companies perform calculations on large SMP boxes (e.g. E10K's, HP Superdomes). But this model can be quite expensive and doesn't scale well.

Along comes grid computing (top five strategic technologies for 2008) and now we have the ability to distribute jobs to many smaller server components using load sharing software that distributes the load evenly based on resource availability and policies. Now instead of having one heavily burdened server the load can be spread evenly across many smaller computers. The distributed nature of grid computing is transparent to the user. When a user submits a job they don't have to think about which machine their job is going to get executed on. The "grid software" will perform the necessary calculations and decide where to send the job based on policies.

Many research institutions are using some sort of grid computing to address complex computational challenges. This post talks about how you can volunteer your workstation to be part of a grid that attempts to solve some of the world's biggest challenges.

Some advantages are quite obvious.
1) No need to buy large six figure SMP servers for applications that can be split up and farmed out to smaller commodity type servers. Results can then be concatenated and analyzed upon job(s) completion.

2) Much more efficient use of idle resources. Jobs can be farmed out to idle servers or even idle desktops. Many of these resources sit idle especially during off business hours. Policies can be in place that allow jobs to only go to servers that are lightly loaded or have the appropriate amount of memory/cpu characteristics for the particular application.

3) Grid environments are much more modular and don't have single points of failure. If one of the servers/desktops within the grid fail there are plenty of other resources able to pick the load. Jobs can automatically restart if a failure occurs. With the advancement of blade servers I've seen server farms that may have a few hundred blades in them. If one of them fails you simply pop in a new blade and have your imaging software slap another OS image on it. No need to rush either because nobody should even notice that a node is down as long as there are other nodes available to take jobs. Blades allow you to cram a lot of computing power in a very small space.

4) Policies can be managed by the grid software. Some of the most popular grid enabling software include Platform LSF, Sun Grid Engine, Data Synapse, PBS, Condor, UnivaUD, among others. Each do a good job of monitoring resources and managing job submissions based upon internal policy engines. The software is really the brains behind the grid. A client will reside on each server which send information back to the master telling it what type of availability or resources it has to complete incoming jobs.

5) This model scales very well. Need more compute resources? Just plug them in by installing grid client on additional desktops or servers. They can be removed just as easily on the fly. This modular environment really scales well.

6) Upgrading can be done on the fly without scheduling downtime. Since there are so many resources some can be taken offline while leaving enough for work to continue. This way upgrades can be cascaded as to not effect ongoing projects.

7) Jobs can be executed in parallel speeding performance. Grid environments are extremely well suited to run jobs that can be split into smaller chunks and run concurrently on many nodes. Using things like MPI will allow message passing to occur among compute resources.

Some disadvatages:
1) For memory hungry applications that can't take advantage of MPI you may be forced to run on a large SMP.

2) You may need to have a fast interconnect between compute resources (gigabit ethernet at a minimum). Infiband for MPI intense applications

3) Some applications may need to be tweaked to take full advantage of the new model.

4) Licensing across many servers may make it prohibitive for some apps. Vendors are starting to be more flexible with environment like this.

5) Grid environments include many smaller servers across various administrative domains. Good tools for managing change and keeping configurations in sync with each other can be challenging in large environments. Tools exist to manage such challenges include systemimager, cfengine, Opsware, Bladelogic, pdsh, cssh, among others.

6) Political challenges associated with sharing resources (especially across different admin domains). Many groups are reluctant with sharing resources even if it benefits everyone involved. The benefits for all groups need to be clearly articulated and policies developed that keeps everyone happy. (easier said than done...)

Areas that already are taking good advantage of grid computing include bioinformatics, cheminformatics, oil & drilling, and financial applications.

With the advantages listed above you'll start to see much larger adoption of Grids which should benefit everyone involved. I believe the biggest barrier right now is education.

_____________________

Vassilios
Co-Founder
OuterVillage.com
http://outervillage.com

If you enjoyed this posting please subscribe to our RSS feed or submit it to your favorite social networks.