Introduction to storage pools, VDCs, and replication groups

This topic provides conceptual information about storage pools, virtual data centers (VDCs), and replication groups and the following topics describe the operations that are required to configure them:

The storage that is associated with a VDC must be assigned to a storage pool and the storage pool must be assigned to one or more replication groups to allow the creation of buckets and objects.

A storage pool can be associated with more than one replication group. A best practice is to have a single storage pool for a site. However, you can have as many storage pools as required, with a minimum of four nodes (and 16 disks) in each pool.

You must create more than one storage pool at a site for the following reasons:

  • The storage pool is used for Cold Archive. The erasure coding scheme that is used for cold archive uses 10+2 coding rather than the default ECS 12+4 scheme.
  • A tenant requires the data to be stored on separate physical media.
CAUTION: You are not allowed to perform any management operations on storage pools, replication groups, or namespaces until all the VDCs have been upgraded. Do not perform extend, IP change, network separation, or disk replacement during upgrade. Do not perform geo operations like PSO and add new site while performing upgrade. Also, the upgrade may have enabled new bucket-level features. It is advisable not to use new bucket-level features until all VDCs in a geo-federation have been upgraded.
  • When the storage pool reaches 90% of its total capacity, it does not accept write requests and it becomes a read-only system. A storage pool must have a minimum of four nodes and must have three or more nodes with more than 10% free capacity in order to allow writes. This reserved space is required to ensure that ECS does not run out of space while persisting system metadata. If this criteria is not met, the write will fail. The ability of a storage pool to accept writes does not affect the ability of other pools to accept writes. For example, if you have a load balancer that detects a failed write, the load balancer can redirect the write to another VDC.
  • The maximum number of VDC per ECS federation and/or RG is eight.
  • Node down on a single site VDC (e.g. VDC1) would block adding a new second VDC (e.g. VDC2).

The replication group is used by ECS for replicating data to other sites so that the data is protected and can be accessed from other, active sites. When you create a bucket, you specify the replication group that it is in. ECS ensures that the bucket and the objects in the bucket are replicated to all the sites in the replication group.

ECS can be configured to use more than one replication scheme, depending on the requirements to access and protect the data. The following figure shows a replication group (RG 1) that spans all three sites. RG 1 takes advantage of the XOR storage efficiency that is provided by ECS when using three or more sites. In the figure, the replication group that spans two sites (RG 2), contains full copies of the object data chunks and does not use XOR'ing to improve storage efficiency.

Figure 1. Replication group spanning three sites and replication group spanning two sites

Replication group configuration example

The physical storage that the replication group uses at each site is determined by the storage pool that is in the replication group. The storage pool aggregates the disk storage of each of the minimum of four nodes to ensure that it can handle the placement of erasure coding fragments. A node cannot exist in more than one storage pool. The storage pool can span racks, but it is always within a site.