Artica turns several appliances into one distributed, replicated storage system with GlusterFS. The disks of the appliances are pooled into volumes that every node can mount like a local directory, survive the loss of a disk or of a whole node, and grow by simply adding disks. Everything is driven from the Web console: no command line is required.
¶ The concept of the feature
GlusterFS is a scale-out network file system. It has no central metadata server: every appliance (a node) runs the glusterd management daemon, the nodes are linked together into a storage pool, and a file name is turned into a location by a hash computed on the client side. Data is stored as ordinary files on ordinary XFS file systems, the bricks.
In Artica, the articarest daemon is the single owner of GlusterFS on each node:
- it installs, starts, monitors and restarts
glusterd(watchdog, systemd and monit integration); - it creates and modifies the pool, the volumes, the bricks and the volume options, always reading the real state back from GlusterFS (GlusterFS is the source of truth, nothing is copied into a database);
- it mounts the volumes with persistent systemd mount units that are re-created and re-mounted automatically after a reboot or a crash;
- it opens the GlusterFS ports in the Artica firewall, only for the nodes of the pool and the client networks you allow;
- it measures the space, the bricks and the self-heal backlog every 5 minutes and sends notifications;
- it manages a certificate authority of the pool to encrypt the GlusterFS traffic with TLS;
- its configuration (
/var/lib/glusterd) is included in the Artica snapshots.
Artica ships its own maintained build of GlusterFS (the Artica fork, version 11.3 and later), compiled for Debian 12 and Debian 13 with the libraries it needs: installing GlusterFS does not require any Debian package mirror.
¶ Functional advantages
- High availability of the data: with a replicated, arbiter or dispersed volume, the files stay readable and writable when a disk or a node is lost. When it comes back, the self-heal copies what it missed.
- One namespace for several appliances: every node mounts the same volume and sees the same files, which makes it easy to share configuration files, templates, quarantines, logs or backups between appliances.
- Grow without downtime: add bricks to a volume while it is in use, then rebalance the existing files on the new bricks.
- No dedicated hardware: the storage runs on the disks of the appliances you already have; no SAN, no NAS, no metadata server.
- Wizards instead of commands: disk preparation (XFS), volume creation, mounting, brick addition, brick replacement and removal are guided; each wizard shows the equivalent
glustercommand before acting. - Secure by default on the network: the GlusterFS ports are only reachable from the pool nodes and the client networks you declare, and the traffic can be encrypted with TLS using a certificate authority managed by Artica.
- Supervised: space used, bricks offline, self-heal stalled and split-brain files raise notifications (e-mail, SIEM) and are drawn on 24 h / 7 days / 31 days charts.
- Resilient to reboots: mounts are persistent; a brick or a mount that dies is restarted automatically.
¶ Concepts
| Term | Meaning |
|---|---|
| Node | An Artica appliance running GlusterFS. |
| Storage pool | The set of nodes that trust each other (GlusterFS calls it the trusted storage pool). A volume can only use bricks of nodes of the pool. You link a node to the pool from any node that is already a member. |
| Brick | A directory on a node that stores the data of a volume, written host:/path, for example 192.168.90.99:/media/gluster1/shared-data/brick. A brick should live on a dedicated disk formatted in XFS, never on the system disk. |
| Volume | The logical file system made of one or more bricks. This is what clients mount. |
| Replica / redundancy set | A group of consecutive bricks that hold the copies (or the erasure-coded fragments) of the same files. The number of bricks of a volume must be a multiple of the size of this set. |
| Self-heal | The background process that copies to a brick the changes it missed while it was offline. Only replicated, arbiter and dispersed volumes have something to heal. |
| Split-brain | A file whose copies were modified differently on two bricks while they could not see each other. GlusterFS cannot choose which copy is right; Artica raises a critical notification. |
| Rebalance | Moves existing files so that every brick holds its share after bricks were added to a distributed volume. |
| Mount / client | A volume attached to a directory of a machine through the native GlusterFS client (FUSE). A client in the statistics is a machine outside the pool connected to the bricks. |
| Volfile server / backup servers | The node a client asks for the layout of the volume when it mounts it. Backup servers are asked when the main one does not answer; once mounted, the client talks to all bricks directly. |
¶ Volume types
| Type | How the data is stored | Usable space | Survives |
|---|---|---|---|
| Distribute | Each file is stored on one brick only. | Sum of all bricks | Nothing: losing a brick hides its files. |
| Replica (N copies) | Each file is written on N bricks of different nodes. | Total / N | The loss of N-1 bricks of a set. Use replica 3: replica 2 is exposed to split-brain. |
| Arbiter | 2 data copies plus a third brick that only stores the metadata and breaks ties. | Total of one data brick per set | The loss of one brick, without split-brain, for little extra disk space. |
| Disperse (erasure coding) | Each file is cut into fragments with parity spread over the bricks of a set. | (bricks - redundancy) / bricks | The loss of redundancy bricks of a set. |
A volume with several sets (for example 6 bricks in replica 3) is distributed-replicated: files are spread over the sets, and replicated inside each set.
¶ Use cases
- Shared storage for a cluster of Artica appliances: several proxies, reverse-proxies or mail gateways use the same directory for shared files (error pages, certificates exported by a script, whitelists, quarantines, reports).
- Replicated file storage on two or three sites: a replica 3 or arbiter volume keeps the data available when one site or one appliance is down.
- Backup target: a large distributed or dispersed volume collects the backups of several servers; capacity grows by adding disks.
- Archive and log retention: big, rarely modified files are a perfect fit for a dispersed volume, which costs less disk than replication.
- Storage for external Linux servers: a Linux server outside the pool mounts the volume with the GlusterFS client once its network is allowed, and optionally with a TLS client certificate issued by Artica.
- Single appliance with several disks: even on one node, a distributed volume joins several disks into one large directory that can later be extended to other nodes.
¶ Administration guide
The administration of GlusterFS is described in the following pages, in the order of a first deployment:
- Installation and activation
Requirements, installation, activation, firewall and client networks, backup, stopping and disabling GlusterFS on an Artica appliance. - Storage pool
Link Artica appliances into a GlusterFS storage pool, check their connection state and detach a node. - Volumes
Create a GlusterFS volume with the wizard, list volumes, change volume options and rebalance a volume after adding bricks. - Disks and bricks
Prepare XFS disks, add, remove and replace the bricks of a GlusterFS volume, and follow the self-heal. - Mounting volumes
Mount a GlusterFS volume on an Artica appliance with persistent mounts, backup volfile servers and mount options. - Encryption (TLS)
Encrypt GlusterFS management and data traffic with a pool certificate authority managed by Artica, and issue certificates for external clients. - Monitoring, alerts and troubleshooting
Usage charts, space, brick and self-heal alerts of GlusterFS volumes, and troubleshooting.
¶ Limitations
- Not available from the console: geo-replication (asynchronous copy to a remote site), volume snapshots, directory quotas, bit-rot detection, tiering.
- No NFS or SMB gateway: volumes are mounted with the native GlusterFS client only. To share a volume with Windows clients, mount it on an appliance and share the mount point with the file sharing service.
- IPv4 only for brick hosts and mount servers.
- Many small files and random I/O (databases, virtual machine disks, mail spools with millions of files) are slow on a network file system: GlusterFS fits large files and moderate metadata activity best.
- Latency matters: every write goes to all the copies synchronously; a replicated volume between distant sites is as slow as its slowest link.
- A distributed volume has no redundancy: losing a brick makes its files unavailable, and a brick cannot be replaced (add a new brick, then remove the old one with migration).
- Replica 2 is exposed to split-brain: prefer replica 3 or arbiter.
- Encrypting management requires a maintenance window (local volumes stopped).
- The firewall rules are applied only when the Artica firewall is enabled.
- All the nodes of a pool should run the same GlusterFS version.
- The measurements are taken every 5 minutes and kept 31 days.