Blog
Data consistency in distributed architecture
Server-side applications typically use a single primary database. When an application or microservice operates solely on this database, maintaining transactional consistency is relatively straightforward. The problem arises when the application also needs to send a message to a queue or save data in another microservice. In such a scenario, the external service falls outside the control of the local transaction.
This is a significant challenge in distributed systems, which is why best practices have been developed to preserve data integrity.
Example scenario:
We want to save data to a table and send a message to a queue indicating that the operation was successful. This is a very common business use case. Instead of a queue, it could be any other external system — the underlying mechanism remains the same.
How do we solve this problem to avoid data inconsistency? One of the most popular approaches is the Transactional Outbox pattern.
Example implementation in Java Spring:
- We create a business table alongside an additional table, e.g., queue_message.
- We create a method annotated with @Transactional.
- Within a single transaction, we save the business data and insert a record into the queue_message table.
- We set up a periodic task, for instance using @Scheduled, which reads records from the queue_message table and publishes them to the queue.
Thanks to this solution, we can be certain that if the data save fails, the message will not be sent to the queue. This prevents data inconsistency and reduces the need for subsequent error handling. In practice, this translates to savings in both time and resources — both financial and human.
Data consistency under concurrent access
A major challenge in web applications is managing concurrent data modifications in the database. When multiple users save independent data to the same table, it rarely causes an issue. However, system architects must anticipate potential conflicts.
Consider a typical scenario:
We have a record in the meetings table. Two users read the exact same data at the same time. Next, they both attempt to update the name of this very same entry. What happens? In most standard setups, the last write wins, completely overwriting the previous update.
To prevent this loss of data integrity, concurrency control and data locking mechanisms are applied.
Optimistic Lock
The optimistic approach assumes that conflicts occur rarely. Data is not locked during the read phase; instead, conflicts are detected only during the write operation.
In Spring, this is typically implemented by adding a field annotated with @Version to the entity. When a write is attempted, the framework checks the record’s version number. If another user has modified the record in the meantime, a conflict exception is thrown.
Example:
@Version
private Long version;
Pessimistic Lock
The pessimistic approach assumes a high likelihood of conflicts from the very start of the operation. The record is locked during the read phase, preventing other users from modifying it concurrently.
Example in Spring:
@Lock(LockModeType.PESSIMISTIC_WRITE)
@Query(“select a from Data a where a.id = :id”)
Optional findByIdForUpdate(@Param(“id”) Long id);
In this scenario, subsequent users must wait until the lock is released. While this solution significantly increases data safety, it can degrade system performance under a high volume of concurrent operations.
Contact us: contact