TL;DR
- Distributed transactions are one of the hardest problems in computer science.
- NoSQL was a response to scalability limitation and a very high cost of traditional RDBMS.
- CAP theorem says in case of network partitions, among consistency (correctness) and availability, one has to be comprised in favor of the other.
- The first generation of NoSQL DBMS chose availability and they were eventually consistent.
- In theory, they will reconcile conflicts in a finite time after a network partitioning by probabilistically voting on what the data is supposed to be.
- Most real-world eventually consistent systems use a simplistic last-write-wins based on the local system time.
- Although NoSQL systems are available and eventually consistent, they do NOT guarantee correctness.
- Therefore, databases need external consistency or ACID.
- Legacy databases have a single centralized machine and serialize writes to a single disk in a deterministic order. Scalability is only vertical.
- In primary/follower replicated databases, followers asynchronously replicate the state of the primary.
- Google’s Spanner:
- Multi-shard transactions are done by a two-phase prepare/commit algorithm.
- Shard failover is automated via Paxos.
- Physical atomic clock hardware synchronizes the system time on all shards within very small error bounds.
- Google has the resources to build and maintain atomic clock hardware and bounded-latency networks.
- Calvin is a logical clock oracle that does not rely on any single physical machine and can be widely distributed.
- The order of transactions is determined by preprocessing an externally consistent order to all incoming transactions.
- FaunaDB is a Relation NoSQL system implementing Calvin model.
- In practice, the availability of systems like Spanner and FaunaDB in the cloud is the same as the availability of AP systems.
- Bounded consistency is better than eventual consistency.