= BeeGFS/FhGFS Changelog (2014.01 Release Series) = == Changes in 2014.01-r22 (release date: 2017-09-28) == * General Changes: * client: Add support for mlnx_ofed 4.1 == Changes in 2014.01-r21 (release date: 2017-03-14) == * general: Fix detection for the presence of systemd on recent distributions. * storage: Fix problems with new security heuristic in recent RDMA drivers that could lead to RDMA failures and loss of RDMA connectivity if errors occur during write on server with enabled quota. [Thanks to Megware and Corvid for reporting.] == Changes in 2014.01-r20 (release date: 2016-11-02) == * General Changes: * meta/storage: Added support for RDMA with latest OFED drivers (e.g. mlnx_ofed 3.3, linux-4.6, RedHat 7.2 with kernel 3.10.0-327.22.2.el7). [Thanks to Mellanox and University of Mannheim for reporting.] == Changes in 2014.01-r19 (release date: 2016-02-12) == * Improvements: * all: Support for ZFS in systemd scripts. * General Changes: * storage: Added config option to enable/disable zfs quota support. * client: Added support for RHEL 7.2 kernel. [Thanks to University of Hamburg for reporting.] * Fixes: * fsck: Fixed case that could lead to long metadata service stalls in online mode. [Thanks to T-Systems SfR for reporting.] * mgmt: Remove duplicate IDs from quota list. == Changes in 2014.01-r18 (release date: 2015-08-11) == * General Changes: * client: Unified codebased for tuneFileCacheType=paged with 2015.03 release series. * Fixes: * client: Add extra check during cleanup of refused RDMA connections, which could lead to problems with some Infiniband driver versions. [Thanks to University of Hamburg for reporting.] == Changes in 2014.01-r17 (release date: 2015-07-01) == * Improvements: * ctl: Added new option to include system/nologin users to mode "--getquota". * General Changes: * all: Added support for systemd startup. * client: Updated to be compatible with linux-2.6.18 up to linux-4.0. * Fixes: * fsck: Fixed sparse file check leading to (uncritical) false positive reports of files having wrong attributes. == Changes in 2014.01-r16 (release date: 2015-06-17) == * Fixes: * fsck: Some chunk files were not updated to the actual owner when running "fhgfs-fsck --enablequota" to initialize quota tracking. == Changes in 2014.01-r15 (release date: 2015-05-15) == New: * all: Added Debian 8 packages. * Improvements: * admon: Error detection improved during installation (e.g. if there is no internet connection on nodes or if ssh is not working without password). General Changes: * all: Removed Debian 6 Packages. * Fixes: * admon: Fixed installation on systems where dash is the default shell. * storage: Fixed potential race condition during startup phase of fhgfs-fsck. * client: Fixed rare case of wrong ID comparison in NFS export. * mgmtd: Fixed wrong return value in quota processing method, leading to unnecessary disconnects. * ctl: In mode "--storagebench", the "--wait" option was enabled by default. == Changes in 2014.01-r14 (release date: 2015-03-23) == General Changes: * all: Removed SLES10 packages. * client: Updated to be compatible with linux-2.6.18 up to linux-3.19. (Removed compatibility with linux-2.6.{16,17}.) Fixes: * client: Fixed cleanup of mutexes after free space retrieval. [Thanks to University of Oslo and Megware for reporting.] * client: Disable usage of vfs atomic_open. [Thanks to University of California Irvine for reporting.] * client: Fixed compabitility with linux-3.19. [Thanks to University of Oslo for reporting.] == Changes in 2014.01-r13 (release date: 2015-03-04) == New: * client: Cache free space information from servers to improve performance of statfs() syscall. (Cache expires after 10s by default, can be disabled via tuneStatFsCacheSecs=0 client option.) [Thanks to Universities of Kaiserslautern and Oslo for reporting.] * general: Native packages for SLES12 General Changes: * client: Updated to be compatible with linux-2.6.16 up to linux-3.19. * client: Updated to be compatible with Oracle UEK3 kernel. Fixes: * mgmtd/storage: Fixed wrong return value in quota query messages, which could lead to an unnecessary disconnect/reconnect. [Thanks to Universitiy of Kaiserslautern for reporting.] * storage: Fixed detection of correct corresponding block device if multiple mountpoints exist for the same path. * client: Fixed problem with cleanup of global POSIX locks when a process holding a global lock was terminated by a signal. == Changes in 2014.01-r12 (release date: 2015-01-15) == General Changes: * fsck: New option to ignore free disk space recommendation for database file. [Thanks to Scalable Informatics for reporting.] Fixes: * storage: Worker threads could be left with wrong user ID, resulting in access denied error messages, when quota is enabled. [Thanks to Basefarm for reporting.] == Changes in 2014.01-r11 (release date: 2014-12-19) == Fixes: * admon: Fixed updates of clientstats view. [Thanks to Texas A&M University for reporting.] * fsck: Fixed problem where not all chunks with wrong permissions were updated correctly. [Thanks to BNP Paribas for reporting.] == Changes in 2014.01-r10 (release date: 2014-12-03) == New Features: * general: Added support for quota enforcement. * utils: New BeeGFS C API to create file with user-defined chunksize and number of stripe targets. See /usr/include/beegfs ("#include beegfs/beegfs.h") * utils: Added new fhgfs-ondemand-v2 with improved parallel startup/shutdown via pdsh and improved cleanup after failure of the first on-demand node through new local (per-node) cleanup option. See /opt/fhgfs/sbin/fhgfs-ondemand-v2. * utils: New bash-completion for fhgfs-ctl command. (Install package "bash-completion" on RHEL to enable completion.) * utils: Added support for BeeGFS Hadoop connector. Connector is available for download in customer login area. General Changes: * client: Updated to be compatible with linux-2.6.16 up to linux-3.18. * fsck: Automatically switch to read-only mode if notification events were dropped in online mode. * fsck: Increased buffer sizes for better handling of situations with high system load. * fsck: Set "delete inode" as default repair action for orphaned inodes. * utils: Added new parameter "--wait" to "fhgfs-ctl --storagebench --status" to wait for benchmark completion before returning results. * utils: Added usage percent columns for "fhgfs-df". Fixes: * client: Various stability fixes for page cache (still marked as experimental). * fsck: Fixed handling of file unlink response message (uncritical). [Thanks to Deutsches Elektronen-Synchrotron (DESY) for reporting.] * client: Fixed problem with ftell() return code for append files in some cases. [Thanks to University of California, Irvine for reporting.] * client: Fixed problem of long entryIDs being printed corrupted in log files. * client: Fixed problem where file attributes were not updated correctly after being accessed with "gedit". [Thanks to TU Denmark for reporting.] * meta: Fixed update of parent dir information on directory move. (Only relevant for kernel nfs re-export.) == Changes in 2014.01-r9 (release date: 2014-08-19) == New Features: * client: Added new (experimental) caching option tuneFileCacheType=hybrid for paged reads and buffered writes. This allows to use the system page cache with its large dynamic client-side cache size for read-caching of frequently read data and fast buffers for writing to allow high throughput. * general: Added new config file option connTcpOnlyFilterFile to define IP address ranges to which no RDMA connection should be established. This is useful e.g. for environments where all hosts support RDMA, but some hosts cannot connect via RDMA to some other hosts. Improvements: * client: Several improvements for general page cache performance and stability. (Still experimental.) * client: Improved parallelism when multiple readers on same host are accessing the same file. * client: Support multi-page operations for memory-mapped files in buffered mode for higher throughput. Fixes: * client: Fix for recent Linux kernels that support atomic-open to correctly handle entries that were moved to a different metadata server. (Regression from r6.) * client: Several fixes for kernel NFS re-export to avoid stale file handles. (Requires all servers to be at r9 or higher to be effective.) * storage: Fixed problem if "fhgfs-ctl --storagebench --cleanup" was used before running a benchmark. * admon: Various minor internal fixes, e.g. improved verifcation of ssh connectivity. == Changes in 2014.01-r8 (release date: 2014-06-16) == New: * general: Native packages for RHEL7 General Changes: * fsck: Also check for orphaned mirror chunks. Fixes: * fsck: Fixed problem where only TCP-connections were established on an RDMA-capable system. * client: Fixed problem where incorrect file open information was sent to metadata servers, resulting in wrong file size being reported for open files. (Regression from r6). * meta: Fixed incorrect handling of unlink message in rare cases. * ctl: Fixed incorrect sum of quota usage in some cases in mode "--getquota". == Changes in 2014.01-r7 == * internal build, not released == Changes in 2014.01-r6 (release date: 2014-05-25) == New Features: * general: Added support for per-user statistics. Similar to the existing per-client statistics ("fhgfs-ctl --clientstats"), there now is a new mode ("fhgfs-ctl --userstats") to query live statistics for individual users. (Requires r6 or higher metadata and storage servers.) See "fhgfs-ctl --userstats --help". * general: Added new fhgfs-setup scripts to setup management/metadata/storage servers and clients without the need to manually edit the config files. (Location: /opt/fhgfs/sbin) Improvements: * client: Improved performance of file lookup/create/open through new intent-based message. (Only effective for r6 or higher metadata servers.) Enhancements: * ctl: "fhgfs-ctl --serverstats" now has a new option "--perserver" to show individual statistics for all servers. This can help to identify lagging servers or storage targets. * ctl: "fhgfs-ctl --clientstats" now has a new option "--names" to show hostnames instead of IP addresses. * ctl: "fhgfs-ctl --getquota" now also accepts user and group names instead of only numeric user and group IDs, when information for a single user or group is queried. * ctl: "fhgfs-ctl --clientstats" (and "--userstats") now have a new option "--filter" to show statistics for a single client (or a single user) only. General Changes: * client: Drop connections to fallback server interfaces (and try to reconnect via primary route) based on timeout instead of previous method, which was based on number of requests. This will avoid too frequent reconnection attempts on high load and helps speeding up reconnects to primary interfaces on low load. See new option "connFallbackExpirationSecs" in config file. (The old option connNonPrimaryExpriration is now ignored.) * ctl: Added "fhgfs-ctl --serverstats" as an alias for existing mode "--iostat" to query server statistics. This new name fits better with modes "--clientstats" and "--userstats". * ctl: "fhgfs-ctl --getentryinfo" by default now shows the storage servers, to which the stripe targets of a file are mapped. * servers: Start listening for connections on RDMA sockets before listening on TCP sockets. This is to make it more unlikely that a client initially connects via TCP to an RDMA-enabled server (e.g. after server restart). * client: Improved handling of application interrupts (e.g. ctrl+c) in the middle of server communication (i.e. while the client is waiting for a response). We now give the ongoing remote operation a few seconds to complete before handling the interrupt signal. Fixes: * client: Fixed various kernel NFS re-export related problems that could have lead to stale file handles or permission denied errors. * client: Fixed write results verification in rotated mirror mode, which resulted in false server cache loss error messages. [Thanks to Department of Physics of FU Berlin for reporting.] * client: Fixed problem with cleanup of global POSIX locks when a process holding a global lock was terminated by a signal. * storage: Fixed potential problems with unlink and truncation of mirror chunk files. == Changes in 2014.01-r5 (release date: 2014-04-13) == New Features: * client: This release contains a new fhgfs-client-compat package. It does not conflict with the normal fhgfs-client package of other major release series (e.g. fhgfs-client-2012.10) and thus allows mounting of FhGFS instances of two different major release series together on the same machine. Improvements: * fsck: Several performance improvements to reduce overall runtime. * general: Updated/added dependencies on other services (e.g. Infiniband) in init.d scripts. * general: Reduced amount of log messages under normal circumstances (e.g. disabled several repeating informative connection-related log messages, which were not indicating error conditions). * utils: Show connectable target IP and transfer protocol in fhgfs-check-servers script. * admon: Improved clientstats view, e.g. filter less important fields by default. Fixes: * client: Re-add additional local locks even if global POSIX locks are enabled, so that the Linux kernel does correct unlock calls on process exit. * storage: Fixed problems with access of files in old 2012.10 storage format after enabling quota support. [Thanks to University of Oslo for reporting.] == Changes in 2014.01-r4 == * internal build, not released == Changes in 2014.01-r3 == * internal build, not released == Changes in 2014.01-r2 (release date: 2014-03-10) == Fixes: * fsck: Fixed net message deserialization problem in mode "--enablequota" [Thanks to Heidelberg Institute for Theoretical Studies (HITS) for reporting.] == Changes in 2014.01-r1 (release date: 2014-03-10) == This is a new major release. The primary focus for this release was on performance improvements on the storage server side and connection authentication as a new feature. If you are upgrading from a 2012.10 release, make sure to read the compatibility section below. New Features: * New Data Layout: Storage servers now use a new data layout with user- and time-based grouping of chunk files. The new layout significantly reduces disk seeks by improving cache efficiency of the underlying local server file system and helps to avoid aging-related performance reduction. * Connection-based Authentication: Clients and servers now support connection authentication based on a pre-shared secret. See option "connAuthFile" in config files for more information. Enhancements: * New Machine Architectures: Clients now support system architectures that have strict memory alignment requirements. * Admon: Client operation statistics (as known from "fhgfs-ctl --clientstats") are now also available in the Admon GUI. * Append Files: Clients can now buffer files opened in append mode and can now implicitly take global locks to avoid conflicts with concurrent append writers on other hosts. This can significantly reduce the amount of network messages and thus increases throughput for files opened in append mode. See client config option "tuneUseGlobalAppendLocks" to enable buffering and global locking. General Changes: * Sparse Files: stat() operation now returns actual number of disk blocks used as reported by underlying local server file system for sparse files. * Client Directory Listing: readdir() operation now returns actual inode numbers instead of dummy values. * Client File Attributes: New optional file attribute cache that can be used e.g. to improve kernel NFS reexport performance. * Client Packaging: The client now only consists of a single package, instead of the former splitup into a client and a separate client-opentk package for the Infiniband layer. * Data Migration: "fhgfs-ctl --migrate" now correctly handles (ignores) special files, e.g. FIFOs, when migrating data away from a storage target. === Compatibility Notes === * Supported distributions: RHEL 5/6 (and Fedora), SLES 10/11 (and OpenSuse), Debian 6/7 (and Ubuntu) * Supported Linux kernel versions (client): 2.6.16 - 3.14 * Servers and clients with different minor versions of the 2014.01 release series are compatible and can be used together in the same file system instance. * Servers and clients from different major release series are not compatible and cannot be used together in the same file system instance. * If you want to ugrade from a 2012.10 release, see here for upgrade notes: http://www.fhgfs.com/release/fhgfs_2014.01/UpgradeNotes.txt