Multi mode

If you want to run more than one instance of one daemon type on a system, you have to use multi mode.


Table of Contents (Page)

  1. General Information about Multi mode
  2. BeeGFS v7 and newer
  3. BeeGFS v6 and older
  4. Example Configuration
 


General Information about Multi mode


In BeeGFS, multi mode, is a way to run multiple instances of one type of service on one server. This can be useful if for example more than one BeeGFS file system or more than one metadata target is used on a single server. Please take into account that, when using multi mode within a single file system, BeeGFS will treat them as individual machines. So, when creating for example mirror buddy groups with the --automatic option, one group might have both services on a single server. In that case, a manual configuration for Buddy Groups is necessary.

For multi mode, you will need to create a separate configuration file for the other daemon instance, using different network ports, a different storage directory, a different log file and so on. If the second daemon instance on a machine should become part of the same file system instance (i.e. it registers at the same management daemon as the first daemon instance on this machine), then you would also need to set a different NodeID manually for the second daemon instance. (See "Configuration & Tuning" section on this page for information on how to manually set the NodeID.)



BeeGFS v7 and newer


If your system uses systemd, you can use the BeeGFS systemd service template to start multiple instances of one service:
 systemctl start beegfs-<service>@<name>.service 

where <daemon> is the name of the service to be started (e.g. meta, storage, ...) and <name> is a user defined string, representing the instance about to be started.
The single mode config file in /etc/beegfs/beegfs-<service>.conf will not be used when starting in multi mode. Instead, each instance has its own configuration file, stored in /etc/beegfs/<name>.d/beegfs-<service>.conf .

If your system doesn't have systemd, it uses the old init script as explained below in the v6 section.


BeeGFS v6 and older


To start BeeGFS in multi mode, you have to set
MULTI_MODE="YES"

in /etc/default/beegfs-<service> , where <service> is the name of the beegfs service to be started.
The single mode config file in /etc/beegfs/beegfs-<service>.conf will not be used when starting in multi mode. Instead, each instance has its own configuration file, stored in /etc/beegfs/<name>.d/beegfs-<service>.conf . Here <name> is a user defined string, representing the corresponding instance.

To start an instance, use the init script with the instance identifier as an argument:
 /etc/init.d/beegfs-<service> start <name> 


Example Configuration


In this example we are going to create a setup with the metadata and the storage daemons running in multi-mode, in a single BeeGFS namespace. This is an adapted version of the Commands Walk-Through from the installation instructions that only describes the adaptions for the beegfs-meta and beegfs-storage services.

$ ssh root@node01
$ /opt/beegfs/sbin/beegfs-setup-mgmtd -p /data/beegfs/beegfs_mgmtd

$ ssh root@node02
$ mkdir /etc/beegfs/inst1.d
$ mkdir /etc/beegfs/inst2.d
$ cp /etc/beegfs/beegfs-meta.conf /etc/beegfs/inst1.d/
$ cp /etc/beegfs/beegfs-meta.conf /etc/beegfs/inst2.d/
$ vi /etc/beegfs/inst1.d/beegfs-meta.conf # Adapt "connMetaPort*", "logStdFile", etc.
$ vi /etc/beegfs/inst2.d/beegfs-meta.conf # Adapt "connMetaPort*", "logStdFile", etc.
$ /opt/beegfs/sbin/beegfs-setup-meta -c /etc/beegfs/inst1.d/beegfs-meta.conf -p /data/beegfs/beegfs_meta1 -s 21 -S meta1-inst1 -m node01
$ /opt/beegfs/sbin/beegfs-setup-meta -c /etc/beegfs/inst2.d/beegfs-meta.conf -p /data/beegfs/beegfs_meta2 -s 22 -S meta1-inst2 -m node01

$ ssh root@node03
$ mkdir /etc/beegfs/inst1.d
$ mkdir /etc/beegfs/inst2.d
$ cp /etc/beegfs/beegfs-storage.conf /etc/beegfs/inst1.d/
$ cp /etc/beegfs/beegfs-storage.conf /etc/beegfs/inst2.d/
$ vi /etc/beegfs/inst1.d/beegfs-storage.conf # Adapt "connStoragePort*", "logStdFile", etc.
$ vi /etc/beegfs/inst2.d/beegfs-storage.conf # Adapt "connStoragePort*", "logStdFile", etc.
$ /opt/beegfs/sbin/beegfs-setup-storage -c /etc/beegfs/inst1.d/beegfs-storage.conf -p /mnt/myraid11/beegfs_storage -s 31 -S stor1-inst1 -i 311 -m node01
# To add a second storage target on this same machine:
$ /opt/beegfs/sbin/beegfs-setup-storage -c /etc/beegfs/inst1.d/beegfs-storage.conf -p /mnt/myraid12/beegfs_storage -s 31 -S stor1-inst1 -i 312
$ /opt/beegfs/sbin/beegfs-setup-storage -c /etc/beegfs/inst2.d/beegfs-storage.conf -p /mnt/myraid21/beegfs_storage -s 32 -S stor1-inst2 -i 321 -m node01
$ /opt/beegfs/sbin/beegfs-setup-storage -c /etc/beegfs/inst2.d/beegfs-storage.conf -p /mnt/myraid22/beegfs_storage -s 32 -S stor1-inst2 -i 322

$ ssh root@node1 systemctl start beegfs-mgmtd
$ ssh root@node2 systemctl start beegfs-meta@inst1
$ ssh root@node2 systemctl start beegfs-meta@inst2
$ ssh root@node3 systemctl start beegfs-storage@inst1
$ ssh root@node3 systemctl start beegfs-storage@inst2






Back to table of contents
Valid XHTML :: Valid CSS: :: Powered by WikkaWiki