FhGFS Upgrade Notes: Version 2009.08 to 2011.04 =============================================== This document contains the relevant changes for users who are upgrading from the 2009.08 release series to the 2011.04 release. 2009.08 and 2011.04 release clients and servers cannot be mixed in the same file system. The storage format has changed with the 2011.04 release. New servers cannot read the former 2009.08 storage format. There is a separate upgrade tool to convert existing data to the new storage format. See here for upgrade instructions: http://wiki.fhgfs.com/wikka.php?wakka=ServerUpgrade201104 2009.08 release config files are not compatible with the 2011.04 release. However, the values that need to be set in the config files are still the same. - All underscores ("_") in names were converted to dashes ("-"), e.g. fhgfs_meta.conf is now fhgfs-meta.conf - sysMgmtdHost needs to be set in all server and client configs (except fhgfs-mgmtd.conf). - Client mount path is set via /etc/fhgfs/fhgfs-mounts.conf. - store...Directory needs to be set in all server configs. The new release allows specifying multiple directories (=> storage targets) for storage servers as a comma-separated list here, e.g.: storeStorageDirectory=/mnt/raid1,/mnt/raid2 (fhgfs-online-cfg and the Admon GUI have been enhanced to show information about these storage targets.) - The new metadata server allows storing metadata as extended attributes instead of normal file contents, which significantly reduces disk seeks and access latency if ext3/4 inodes have at least 512 bytes size ("mkfs.ext4 -I 512"). Extended attributes have to be enabled or disabled during the first start of a metadata server ("storeUseExtendedAttribs" setting in fhgfs-meta.conf) and cannot be changed afterwards. - There are some new options to avoid typical problems in practice like "storeAllowFirstRunInit=false" to forbid daemon startup if the RAID storage is not mounted properly. "sysAllowNewServers=false" in fhgfs-mgmtd.conf can be used to deny registration of new servers (e.g. to make sure a new server is not registered accidentally in a production environment). "sysMountSanityCheckMS" can be used to let the client refuse mounting the file system if not all servers are reachable. - The FhGFS OpenTk communication library in userspace comes prebuilt with and without native Infiniband support, so there is no longer a separate build step required. Just use $ /usr/sbin/fhgfs-opentk-lib-update-ib on the corresponding machines to automatically enable/disable native IB usage. (This is also automatically called after the installation of the fhgfs-opentk-lib package.) - The build process of the client kernel module is now integrated into the client init script and happens automatically on: $ /etc/init.d/fhgfs-client start (This is intended to simplify kernel updates.) Native Infiniband support in the client kernel module is now enabled by setting the corresponding build arguments in the client autobuild config file. To enable native IB support, set the "buildArgs" parameter in /etc/fhgfs/fhgfs-client-autobuild.conf on the clients. (See comments in the autobuild config file for details.) After changing the "buildArgs" parameter, use $ /etc/init.d/fhgfs-client rebuild to force a rebuild of the client kernel module. - The Admon GUI is now contained in the admon package and can be downloaded by accessing the admon daemon with a webbrowser (http://:8000) - The fhgfs-online-cfg tool is no longer part of the client package and was moved to the new fhgfs-utils package. It is now also available under a new shorter name (/usr/bin/fhgfs-ctl).