= FhGFS Changelog = == Changes in 2011.04-r23 == === Client === [r2] new: support for vanilla kernels up to version 2.6.39 (min version: 2.6.16) [r2] fixed: problem in init.d script could have caused a shutdown delay on some systems [r3] fixed: compatibility with kernel >=2.6.38 rcu mode [r3] improved: init script status message [r4] new: support for vanilla kernels up to version 3.0 (min version: 2.6.16) [r5] improved: reduced communication on stat() syscalls [r5] improved: new build option "FHGFS_INTENT" to reduce communication for various file operations. Not enabled by default, because it requires at least version r5 metadata servers. (To enable this feature, add "FHGFS_INTENT=1" to "buildArgs" in /etc/fhgfs/fhgfs-client-autobuild.conf) [r5] new: support for "ulimit -f" limits [r5] fixed: detection of TCP connection failure and NIC failover in some cases [r5] new: automatic detection of fhgfs-client-autobuild.conf modifications to force module rebuild [r6] fixed: problem that could lead to shown file sizes being too small in certain cases ("fhgfs-ctl mode=refreshentryinfo/refreshallentries" can be used to update incorrect file sizes) [r6] improved: compatibility with enabled SELinux [r7] new: show warning on init script start if SELinux is enabled, because it can lead to ambiguous errors and kernel log messages [r8] fixed: update of root inode attributes [r8] fixed: correct returned error code for failed page writes in certain cases [r9] improved: async flush of caches in buffered mode [r9] fixed: buffer flush when application is interrupted/terminated [r9] improved: minor changes in init.d scripts [r10] fixed: problem with inode reference counters of new async buffer flush mechanism from r9 [r11] new: support for RHEL 6.2 kernel [r13] new: added connPortShift to mount options [r13] improved: better read cache efficiency for some workloads that do backward seeks [r15] new: heuristic check for OFED_INCLUDE_PATH correctness in init.d script [r15] new: support for Linux vanilla kernels up to version 3.2, compiles with Linux 3.3-rc6 (min version: 2.6.16) [r15] changed: license is now GPL v2 [r16] improved: timestamps updates now more compatible with rsync [r16] fixed: problem with cleanup when mount sanity check fails (regression from r15) [r17] new: compiles with Linux 3.4 (min version: 2.6.16) [r17] improved: idle disconnect intervals to avoid ambiguous disconnect messages in client log [r17] fixed: rare async buffer flush problem where flusher component did not cancel flushing in case of an unrecoverable error condition [r20] new: retry RDMA connection when rejected because of stale cm_id [r20] improved: init script "stop/restart" now don't return an error when module unloading fails because of active fhgfs mounts that are not contained in fhgfs-mounts.conf [r20] new: fhgfs IO is now added to /proc//io, so that process IO to/from fhgfs now appears in tools like iotop or atop [r20] fixed: problem with cleanup of RDMA cm_id when process signal is pending [r20] new: added warning about SDP (= OFED Sockets Direct Protocol) to config file (note: not related to fhgfs native RDMA with ibverbs) [r22] new: support for Linux 3.5 (min version: 2.6.16) and SLES11 SP3 kernel [r23] new: build system updated to support RHEL 5.9 Infiniband drivers === Metadata Server === [r2] improved: better cleanup of files that were located on a removed storage node/target [13] new: added new target chooser "randomrobin" (see fhgfs-meta.conf for description) [r20] improved: increased default RDMA backlog size to 128 [r20] improved: doubled default number of communication slave threads [r21] fixed: possible races when different move operations are applied to same file concurrently [r22] improved: continue startup if NUMA check fails (regression from r20) === Storage Server === [r3] improved: direct IO sizes independent of tuneFileRead/WriteSize settings [r13] improved: free space info now reported based on user ID (makes a difference for file systems that reserve some space for the superuser) [r20] improved: increased default RDMA backlog size to 128 [r22] improved: continue startup if NUMA check fails (regression from r20) [r22] fixed: connInterfacesFile option was not applied correctly (regression from r20) === Management Server === [r20] fixed: rare potential problem with node list retrieval due to invalid memory access [r22] fixed: connInterfacesFile option was not applied correctly (regression from r20) === Admon (Installation & Monitoring) === [r3] fixed: GUI right mouse click on Windows [r5] fixed: memory management issues [r5] fixed: GUI main window resizing problem [r6] fixed: potential problem with retrieval of Linux distribution etc. from configured nodes during Admon-based installation. [r10] fixed: minor script parsing error during GUI-based installation with custom userspace Infiniband include path [r12] fixed: GUI repackaged with updated version string [r13] improved: better performance with X11 forwarding [r13] improved: fhgfs services status check now based on standard init.d scripts [r13] improved: faster ssh connectivity check [r13] improved: more detailed error messages during installation, if Infiniband is not configured properly [r14] improved: added GUI parameter to override version check (ignoreVersionTest=true) [r14] fixed: GUI version check fixed [r15] improved: new layout for several GUI elements (e.g. to work better with small screen resolutions) [r15] improved: several internal installation checks improved [r19] improved: minor GUI improvements (file browser, login dialog) [r19] fixed: GUI compatibility with Java 1.7 [r19] fixed: GUI download via HTTP from Admon daemon with recent Firefox versions [r20] fixed: removed leading double slashes in stripe pattern dialog [r21] improved: minor performance improvements for database access [r21] improved: added sorter to file browser table [r21] fixed: potential repaint problem in "known problems" window [r22] fixed: average graphs could not be disabled in storage server throughput window === Online Configuration Tool === [r2] improved: client stats can now be queried from multiple servers [r3] improved: more enhancements for client stats (sum, unit, ...) [r4] improved: more enhancements for client stats (per-second rw throughput) [r4] fixed: problem with client stats output on large clusters [r5] new: experimental mode to find all files that are located on a certain server [r5] new: experimental mode to migrate file chunks to other servers [r5] new: mode for reverse lookup (chunkID/entryID to path) (requires version r5 or higher metadata servers) [r7] fixed: query of client stats from server release versions that do not match fhgfs-ctl release version [r15] fixed: problem with mode=find on large directory trees [r15] new: mount= can now be given as argument instead of an explicit config file (cfgFile=), which is more convenient on systems with more than one fhgfs mountpoint on a client) [r18] fixed: problem with mode clientstats (regression from r17) [r22] fixed: potential problem with internal cleanup of mode=migrate === File System Consistency Check === [r4] new: parallel online file system check and repair tool (fhgfs-fsck) (now included in fhgfs-utils package) [r5] fixed: memory management issues [r5] improved: CPU usage [r5] new: option to forward-check only certain subdirectories [r5] note: version r5 fsck requires at least version r5 storage and metadata servers [r15] improved: added human readable error log [r15] new: added read-only option [r15] fixed: backward check could have run infinitely in some cases [r15] fixed: the same error could have been reported more than once [r20] fixed: rare potential problem when directories are removed while check is running [r21] fixed: possible endless loop on inode64 systems [r21] fixed: deletion of dentries without inodes under special circumstances === Misc === [r4] fixed: status output of some init scripts [r5] fixed: lsb tags of debian init scripts [r5] improved: packages are signed now (based on new gpg key, which is now also used to sign package repositories) [r6] improved: package dependencies on specific versions [r7] fixed: problem with autostart of fhgfs services during system boot being disabled after rpm update (required manual "chkconfig --add" after an update). This fix will only be effective for updates from r7 to higher versions. [r13] improved: added more checks to fhgfs-opentk-lib-update-ib [r17] improved: checking and output of fhgfs-opentk-lib-update-ib [r17] new: added config file (/etc/fhgfs/fhgfs-libopentk.conf) for fhgfs-opentk-lib-update-ib to allow enforcing of enabled IB support (useful e.g. if fhgfs update is installed in a chroot envirnonment) [r17] improved: internal server-side locking code improved to better prevent long wait periods of certain threads/clients under very high load [r17] improved: increased default max number of connections to the same server (connMaxInternodeNum) [r19] improved: added new OFED init script name "rdma" to fhgfs init script startup dependencies [r20] new: added connInterfacesList config option to fhgfs_mgmt/meta/storage. (Similar to the old connInterfacesFile, a comma-separated list of allowed interfaces can now be specified directly in the config file or as command line argument.) [r22] improved: fhgfs-opentk-lib debian package no longer depends on ibverbs and rdma lib packages. == Changes in 2011.04-r1 == This is a major update with new features and improvements in all areas. Some of the key features are: * Storage servers support multiple storage targets per server * Metadata servers can store metadata as extended attributes * Detailed per-client statistics * Simplified installation/updates via software repositories (yum, apt) * Support for distributed POSIX locking * Enhanced graphical administration interface Full List of Changes: === General === * new: option to use global POSIX file range and entry locks (can be activated via new client option tuneUseGlobalFileLocks) * new: servers can be configured to refuse startup with uninitialized storage directory (e.g. if local file system not mounted correctly) * improved: added new flow control algorithm to Infiniband connections (helps to keep throughput high even if file chunk sizes are larger than network buffers) * improved: new layout of config files (separated options and documentation to give a better overview of available settings) * improved: FhGFS OpenTk userspace communication library comes pre-build with and without native Infiniband support (update via /usr/sbin/fhgfs-opentk-lib-update-ib) === Client === * new: support for kernels up to vanilla version 2.6.38 (min version: 2.6.16) * new: client module build process is now integrated into client init script (i.e. no more separate "make" required); set build arguments via /etc/fhgfs/fhgfs-client-autobuild.conf * new: directory entry type hints (to allow more efficient exectuion of e.g. "ls" and "find") * new: client-side idle disconnect of unused connections (to avoid ambiguous network log warnings from former server-side idle disconnects) * new: optional mount sanity check to refuse mounting in case of detected problems * new: support for special files (e.g. FIFOs) * improved: adaptive read-ahead (detection of random access) * improved: more info in "/proc/fs/fhgfs" (e.g. connected IPs) * improved: redesigned automatic rebuild of client kernel modules in case of kernel version changes (does not require DKMS) === Management Server === * new: free space pools for storage and metadata servers to keep server usage balanced (configure in fhgfs-mgmtd.conf, query with fhgfs-ctl tool) * new: config file option to reject registration of new servers === Metadata Server === * new: option to store metadata as extended attributes (enabled by default in config file); reduces disk seeks if ext3/4 inode size is at least 512 bytes * new: optional round-robin storage targets chooser (currently most useful to have deterministic/reproducible striping during initial system tuning) * improved: extended storage format saves all file attributes, including size and mtime (less communication with storage servers required) * improved: redesign of internal request handling structure to improve behaviour in high concurrency situations * improved: memory management === Storage Server === * new: support for multiple storage targets per server * new: Direct IO flag (O_DIRECT) from client apps is now passed through to storage servers * improved: added some new tuning options to config file (e.g. read/write IO sizes) === Admon (Installation & Monitoring) === * new: can now also show metadata server statistics * new: can now retrieve server log files * new: Java GUI can be downloaded by pointing a webbrowser to the admon daemon (typically http://:8000) * improved: redesigned GUI based on faster drawing library and new internal http server * improved: installation based on new yum/zypper/apt repositories === Online Configuration Tool === * new: mode for detailed per-client operation statistics * new: reachability (liveness) check for servers and clients * new: wrapper scripts ("/usr/bin/fhgfs-check-servers" & co) to make usage more convenient * new: can read multiple paths from stdin for some modes (e.g. mode "getentryinfo") * new: mount path detection (no more relative path arguments required) * improved: help layout * improved: shorter command name "/usr/bin/fhgfs-ctl" === File System Consistency Check === * new: parallel online file system check and repair tool (fhgfs-fsck) (not included in initial release) === Compatibility === * note: This release series introduces a new storage format, which is incompatible with previous versions. We strongly recommend to backup your data before upgrading. We will also provide a tool to convert existing data to the new version.