Friday, November 27, 2009

Scalability, Consistency, Correctness, and Availability, Choose any Three

Above four properties are tightly locked with each other giving rise to interesting tradeoffs, and given a system, you may chose any 3, but achieving all four at once seems almost impossible.

Having said that, Correctness---fact that a system behaves according to its specification---is a property any system have to have, and systems that do have only other three are not very interesting. But it is worth noting that consistency and correctness are two different things, and an inconsistent system may be correct, given that its specification does not make any claims on consistency.

Tradeoff between scalability and consistency is well discussed under the topic of loose consistency. Amazon CTO, Werner Vogels has shed much light on the topic, and he argued that to create scalable systems, we have to let go of strong consistency guarantees like ACID (transactions). His post Eventually Consistent, revisted discusses this topic in detail. Those systems are Scalable, Correct, and Available, but only have loose consistency. Amazon's Dynomo is an example of such a system.

We can build Correct, Consistent, and Available systems using group communication, but those systems do not scale. Typically, the limit is considered to be 8-10 nodes in a group in group communication.

Finally, we can build a Scalable, Consistent, and Correct system by stopping all new requests whenever a change happen in a node untilthat change is disseminated in to all other nodes in the distributed system---even if there are thousands of nodes. Unfortunately, if updates are frequent, the system will reject messages most of the time, hence has poor availability.

Incorrect systems are bad, but neither of last three are bad by itself, and each of those are useful on some usecases. Hence they are useful tools in architect's tool box.

My Shutterbug @ Kandy

Following are couple of shots, which came out good---at least IMHO.

View from Lankathilaka Temple 2

Nelum (Lotus)

Fisherman Wharf Sea Lions

A major attraction in San Fransisco is Fisherman Wharf, specially Piper 39 and its Sea lions. Following are few shots of those noisy and smelly, but cute and lovable animals.

Sea Lions at Fisherman Warf, SF

Sea Lions at Fisherman Warf, SF

Actually, an interesting point is that those guys were never placed there by humans, rather one day, they decided to invade boat decks in Piper 39, and stayed there ever since (See http://www.cbsnews.com/stories/2009/11/24/eveningnews/main5764873.shtml). Crowd loved them, and soon they were a major attraction, which allows those snaky invaders to stay there unmolested

Tuesday, November 24, 2009

Prof. Geoffrey Fox named among "People to Watch" in HPCwire

IU digital scientist and professor Geoffrey Fox named among "People to Watch", see IU news article for details.