Friday, November 27, 2009

Scalability, Consistency, Correctness, and Availability, Choose any Three

Above four properties are tightly locked with each other giving rise to interesting tradeoffs, and given a system, you may chose any 3, but achieving all four at once seems almost impossible.

Having said that, Correctness---fact that a system behaves according to its specification---is a property any system have to have, and systems that do have only other three are not very interesting. But it is worth noting that consistency and correctness are two different things, and an inconsistent system may be correct, given that its specification does not make any claims on consistency.

Tradeoff between scalability and consistency is well discussed under the topic of loose consistency. Amazon CTO, Werner Vogels has shed much light on the topic, and he argued that to create scalable systems, we have to let go of strong consistency guarantees like ACID (transactions). His post Eventually Consistent, revisted discusses this topic in detail. Those systems are Scalable, Correct, and Available, but only have loose consistency. Amazon's Dynomo is an example of such a system.

We can build Correct, Consistent, and Available systems using group communication, but those systems do not scale. Typically, the limit is considered to be 8-10 nodes in a group in group communication.

Finally, we can build a Scalable, Consistent, and Correct system by stopping all new requests whenever a change happen in a node untilthat change is disseminated in to all other nodes in the distributed system---even if there are thousands of nodes. Unfortunately, if updates are frequent, the system will reject messages most of the time, hence has poor availability.

Incorrect systems are bad, but neither of last three are bad by itself, and each of those are useful on some usecases. Hence they are useful tools in architect's tool box.

No comments: