Cluster Settings¶
This document outlines the various configuration items to keep in mind when planning a LeoFS system's cluster, and this documentation leads you to be able to configure its cluster when planning and launching it correctly.
Prior Knowledge¶
LeoFS adopts eventual consistency of the consistency model; it takes priority over AP (Availability and Partition tolerance) over C (consistency) which depends on CAP theorem.
To keep the consistency of objects eventually, LeoFS delivers the replication and recovery feature to automatically fix consistency of objects. You can configure the consistency level of a LeoFS system, and it is affected by the configuration.
How to Keep RING's Consistency¶
Case 1: Both LeoManager nodes are unavailable¶
If both LeoManager nodes are unavailable, LeoStorage and LeoGateway nodes don't update the RING to keep its consistency into the LeoFS system.
Case 2: One LeoManager node is unavailable¶
If a LeoManager node is unavailable, LeoFS can update the RING, and synchronize it with the LeoFS' system eventually. After restarting another LeoManager node, LeoManager automatically synchronizes the RING between the manager nodes.
Consistency Level¶
Configure the consistency level of a LeoFS system at LeoManager's configuration file - leo_manager_0.conf. You need to carefully configure the consistency level because it is not able to change some items after starting the system.
There are four configuration items at leo_manager_0.conf, items of which have a great impact on data availability and storage performance.
| Item | Abbr | Modifiable | Default | Description |
|---|---|---|---|---|
consistency.num_of_replicas |
n | No | 1 | A number of replicas |
consistency.write |
w | Yes | 1 | A number of replicas needed for a successful WRITE operation |
consistency.read |
r | Yes | 1 | A number of replicas needed for a successful READ operation |
consistency.delete |
d | Yes | 1 | A number of replicas needed for a successful DELETE operation |
consistency.rack_aware_replicas |
No | 0 | A number of rack-aware replicas |
Data Availability of Consistency Level¶
This document delivers the relationship of data availability and configuration level as below:
| Data Availability | Configuration Level | Description |
|---|---|---|
| Extremely Low | n=2, r=1 w=1, d=1 |
Data can not be acquired even if two nodes goes down (for personal use) |
| Low | n=3, r=1 w=1, d=1 |
Low data consistency |
| Middle(1) | n=3, r=1 w=2, d=2 |
Typical settings |
| Middle(2) | n=3, r=2 w=2, d=2 |
High data consistency than Middle(1) |
| High | n=3, r=2 w=3, d=3 |
Data can not be input and removed even if one node goes down |
| Extremely High | n=3, r=3 w=3, d=3 |
Data can not be acquired even if one node goes down (can not be recommended) |
How To Change Consistency Level¶
You can change consistency.write, consistency.read and consistency.delete of the consistency level that you use the leofs-adm update-consistency-level command, but you cannot update num_of_replicas and rack_aware_replicas.
1 2 | ## Changes the consistency level to [w:2, d:2, r:1] $ leofs-adm update-consistency-level 2 2 1 |