Multi mode

If you want to run more than one instance of one daemon type on a system, you have to use multi mode.


Table of Contents (Page)

  1. General Information about Multi mode
  2. BeeGFS v7 and newer
  3. BeeGFS v6 and older
  4. Example Configuration
 


General Information about Multi mode


In BeeGFS, multi mode, is a way to run multiple instances of one type of service on one server. This can be useful if for example more than one BeeGFS file system or more than one metadata target is used on a single server. Please take into account that, when using multi mode within a single file system, BeeGFS will treat them as individual machines. So, when creating for example mirror buddy groups with the --automatic option, one group might have both services on a single server. In that case, a manual configuration for Buddy Groups is necessary.

For multi mode, you will need to create a separate configuration file for the other daemon instance, using different network ports, a different storage directory, a different log file and so on. If the second daemon instance on a machine should become part of the same file system instance (i.e. it registers at the same management daemon as the first daemon instance on this machine), then you would also need to set a different NodeID manually for the second daemon instance. (See "Configuration & Tuning" section on this page for information on how to manually set the NodeID.)



BeeGFS v7 and newer


If your system uses systemd, you can use the BeeGFS systemd service template to start multiple instances of one service:
 systemctl start beegfs-<service>@<name>.service 

where <service> is the name of the service to be started (e.g. meta, storage, ...) and <name> is a user defined string, representing the instance about to be started.
The single mode config file in /etc/beegfs/beegfs-<service>.conf will not be used when starting in multi mode. Instead, each instance has its own configuration file, stored in /etc/beegfs/<name>.d/beegfs-<service>.conf .

In order to start multiple beegfs-client services on a single server, the procedure is different. In that case, it is only necessary to add more lines to the file /etc/beegfs/beegfs-mounts.conf

Edit the single mode config file in /etc/beegfs/beegfs-mount.conf and adjust the 1st. mount point and /etc/beegfs/beegfs-client-<name>.conf. Additionally please add a 2nd. mount point for the 2nd. client service and /etc/beegfs/beegfs-client-<name>.conf and save the file. Where <name> is a user-defined string, representing the instance about to be started.

The client mount file consists of two space-separated values. The first value is the directory where you want to mount the file system, the second value is the client configuration file for this mount point. You will typically have a line like this in the /etc/beegfs/beegfs-mounts.conf file:
 /mnt/beegfs /etc/beegfs/beegfs-client.conf 


It is also possible to specify multiple mount/config entries in this file (one mount/config pair per line) if you need to mount different BeeGFS instances on the same client. Optionally, you can also inform mount options for each mount/config entry. In the example below, the first line mounts a writable BeeGFS FS on /mnt/scratch and a read-only BeeGFS FS on /mnt/software.

 /mnt/scratch /etc/beegfs/beegfs-client-scratch.conf beegfs rw
 /mnt/software /etc/beegfs/beegfs-client-software.conf beegfs ro 


Start the service with:
 systemctl start beegfs-client.service 



If your system doesn't have systemd, it uses the old init script as explained below in the v6 section.


BeeGFS v6 and older


To start BeeGFS in multi mode, you have to set
MULTI_MODE="YES"

in /etc/default/beegfs-<service> , where <service> is the name of the beegfs service to be started.
The single mode config file in /etc/beegfs/beegfs-<service>.conf will not be used when starting in multi mode. Instead, each instance has its own configuration file, stored in /etc/beegfs/<name>.d/beegfs-<service>.conf . Here <name> is a user defined string, representing the corresponding instance.

To start an instance, use the init script with the instance identifier as an argument:
 /etc/init.d/beegfs-<service> start <name> 


Example Configuration


In this example we are going to create a setup with the metadata and the storage daemons running in multi-mode, in a single BeeGFS namespace. This is an adapted version of the Commands Walk-Through from the installation instructions that only describes the adaptions for the beegfs-meta and beegfs-storage services.

$ ssh root@node01
$ /opt/beegfs/sbin/beegfs-setup-mgmtd -p /data/beegfs/beegfs_mgmtd

$ ssh root@node02
$ mkdir /etc/beegfs/inst1.d
$ mkdir /etc/beegfs/inst2.d
$ cp /etc/beegfs/beegfs-meta.conf /etc/beegfs/inst1.d/
$ cp /etc/beegfs/beegfs-meta.conf /etc/beegfs/inst2.d/
$ vi /etc/beegfs/inst1.d/beegfs-meta.conf # Adapt "connMetaPort*", "logStdFile", etc.
$ vi /etc/beegfs/inst2.d/beegfs-meta.conf # Adapt "connMetaPort*", "logStdFile", etc.
$ /opt/beegfs/sbin/beegfs-setup-meta -c /etc/beegfs/inst1.d/beegfs-meta.conf -p /data/beegfs/beegfs_meta1 -s 21 -S meta1-inst1 -m node01
$ /opt/beegfs/sbin/beegfs-setup-meta -c /etc/beegfs/inst2.d/beegfs-meta.conf -p /data/beegfs/beegfs_meta2 -s 22 -S meta1-inst2 -m node01

$ ssh root@node03
$ mkdir /etc/beegfs/inst1.d
$ mkdir /etc/beegfs/inst2.d
$ cp /etc/beegfs/beegfs-storage.conf /etc/beegfs/inst1.d/
$ cp /etc/beegfs/beegfs-storage.conf /etc/beegfs/inst2.d/
$ vi /etc/beegfs/inst1.d/beegfs-storage.conf # Adapt "connStoragePort*", "logStdFile", etc.
$ vi /etc/beegfs/inst2.d/beegfs-storage.conf # Adapt "connStoragePort*", "logStdFile", etc.
$ /opt/beegfs/sbin/beegfs-setup-storage -c /etc/beegfs/inst1.d/beegfs-storage.conf -p /mnt/myraid11/beegfs_storage -s 31 -S stor1-inst1 -i 311 -m node01
# To add a second storage target on this same machine:
$ /opt/beegfs/sbin/beegfs-setup-storage -c /etc/beegfs/inst1.d/beegfs-storage.conf -p /mnt/myraid12/beegfs_storage -s 31 -S stor1-inst1 -i 312
$ /opt/beegfs/sbin/beegfs-setup-storage -c /etc/beegfs/inst2.d/beegfs-storage.conf -p /mnt/myraid21/beegfs_storage -s 32 -S stor1-inst2 -i 321 -m node01
$ /opt/beegfs/sbin/beegfs-setup-storage -c /etc/beegfs/inst2.d/beegfs-storage.conf -p /mnt/myraid22/beegfs_storage -s 32 -S stor1-inst2 -i 322

$ ssh root@node1 systemctl start beegfs-mgmtd
$ ssh root@node2 systemctl start beegfs-meta@inst1
$ ssh root@node2 systemctl start beegfs-meta@inst2
$ ssh root@node3 systemctl start beegfs-storage@inst1
$ ssh root@node3 systemctl start beegfs-storage@inst2






Back to table of contents
Valid XHTML :: Valid CSS: :: Powered by WikkaWiki