Merge pull request #663 from trapexit/random

only return 1 branch for rand/eprand policies
2024-04-21 12:31:43 +00:00 · 2019-09-07 17:27:56 -04:00 · 2019-09-07 17:25:06 -04:00 · 2019-09-07 15:14:10 -04:00 · 2019-09-07 14:10:36 -04:00 · 2019-09-03 21:43:47 -04:00
32 changed files with 1678 additions and 775 deletions
@@ -150,7 +150,7 @@ install-base: build/mergerfs

 install-mount.mergerfs: install-base
 	$(MKDIR) -p "$(INSTALLBINDIR)"
-	$(LN) -s "mergerfs" "$(INSTALLBINDIR)/mount.mergerfs"
+	$(LN) -fs "mergerfs" "$(INSTALLBINDIR)/mount.mergerfs"

 install-man: $(MANPAGE)
 	$(MKDIR) -p "$(INSTALLMAN1DIR)"
@@ -1,6 +1,6 @@
 % mergerfs(1) mergerfs user manual
 % Antonio SJ Musumeci <trapexit@spawn.link>
-% 2019-05-18
+% 2019-09-03

 # NAME

@@ -59,42 +59,63 @@ A         +      B        =       C
                                  +-- file6
 ```

-mergerfs does **not** support the copy-on-write (CoW) behavior found in **aufs** and **overlayfs**. You can **not** mount a read-only filesystem and write to it. However, mergerfs will ignore read-only drives when creating new files so you can mix read-write and read-only drives.
+mergerfs does **not** support the copy-on-write (CoW) behavior found in **aufs** and **overlayfs**. You can **not** mount a read-only filesystem and write to it. However, mergerfs will ignore read-only drives when creating new files so you can mix read-write and read-only drives. It also does **not** split data across drives. It is not RAID0 / striping. It is simply a union.


 # OPTIONS

 ### mount options

-* **allow_other**: a libfuse option which allows users besides the one which ran mergerfs to see the filesystem. This is required for most use-cases.
-* **direct_io**: causes FUSE to bypass caching which can increase write speeds at the detriment of reads. Note that not enabling `direct_io` will cause double caching of files and therefore less memory for caching generally (enable **dropcacheonclose** to help with this problem). However, `mmap` does not work when `direct_io` is enabled.
-* **minfreespace=value**: the minimum space value used for creation policies. Understands 'K', 'M', and 'G' to represent kilobyte, megabyte, and gigabyte respectively. (default: 4G)
-* **moveonenospc=true|false**: when enabled (set to **true**) if a **write** fails with **ENOSPC** or **EDQUOT** a scan of all drives will be done looking for the drive with the most free space which is at least the size of the file plus the amount which failed to write. An attempt to move the file to that drive will occur (keeping all metadata possible) and if successful the original is unlinked and the write retried. (default: false)
-* **use_ino**: causes mergerfs to supply file/directory inodes rather than libfuse. While not a default it is recommended it be enabled so that linked files share the same inode value.
-* **dropcacheonclose=true|false**: when a file is requested to be closed call `posix_fadvise` on it first to instruct the kernel that we no longer need the data and it can drop its cache. Recommended when **direct_io** is not enabled to limit double caching. (default: false)
-* **symlinkify=true|false**: when enabled (set to **true**) and a file is not writable and its mtime or ctime is older than **symlinkify_timeout** files will be reported as symlinks to the original files. Please read more below before using. (default: false)
-* **symlinkify_timeout=value**: time to wait, in seconds, to activate the **symlinkify** behavior. (default: 3600)
-* **nullrw=true|false**: turns reads and writes into no-ops. The request will succeed but do nothing. Useful for benchmarking mergerfs. (default: false)
-* **ignorepponrename=true|false**: ignore path preserving on rename. Typically rename and link act differently depending on the policy of `create` (read below). Enabling this will cause rename and link to always use the non-path preserving behavior. This means files, when renamed or linked, will stay on the same drive. (default: false)
-* **security_capability=true|false**: If false return ENOATTR when xattr security.capability is queried. (default: true)
+* **allow_other**: A libfuse option which allows users besides the one which ran mergerfs to see the filesystem. This is required for most use-cases.
+* **minfreespace=SIZE**: The minimum space value used for creation policies. Understands 'K', 'M', and 'G' to represent kilobyte, megabyte, and gigabyte respectively. (default: 4G)
+* **moveonenospc=BOOL**: When enabled if a **write** fails with **ENOSPC** or **EDQUOT** a scan of all drives will be done looking for the drive with the most free space which is at least the size of the file plus the amount which failed to write. An attempt to move the file to that drive will occur (keeping all metadata possible) and if successful the original is unlinked and the write retried. (default: false)
+* **use_ino**: Causes mergerfs to supply file/directory inodes rather than libfuse. While not a default it is recommended it be enabled so that linked files share the same inode value.
+* **dropcacheonclose=BOOL**: When a file is requested to be closed call `posix_fadvise` on it first to instruct the kernel that we no longer need the data and it can drop its cache. Recommended when **cache.files=partial|full|auto-full** to limit double caching. (default: false)
+* **symlinkify=BOOL**: When enabled and a file is not writable and its mtime or ctime is older than **symlinkify_timeout** files will be reported as symlinks to the original files. Please read more below before using. (default: false)
+* **symlinkify_timeout=INT**: Time to wait, in seconds, to activate the **symlinkify** behavior. (default: 3600)
+* **nullrw=BOOL**: Turns reads and writes into no-ops. The request will succeed but do nothing. Useful for benchmarking mergerfs. (default: false)
+* **ignorepponrename=BOOL**: Ignore path preserving on rename. Typically rename and link act differently depending on the policy of `create` (read below). Enabling this will cause rename and link to always use the non-path preserving behavior. This means files, when renamed or linked, will stay on the same drive. (default: false)
+* **security_capability=BOOL**: If false return ENOATTR when xattr security.capability is queried. (default: true)
 * **xattr=passthrough|noattr|nosys**: Runtime control of xattrs. Default is to passthrough xattr requests. 'noattr' will short circuit as if nothing exists. 'nosys' will respond with ENOSYS as if xattrs are not supported or disabled. (default: passthrough)
-* **link_cow=true|false**: When enabled if a regular file is opened which has a link count > 1 it will copy the file to a temporary file and rename over the original. Breaking the link and providing a basic copy-on-write function similar to cow-shell. (default: false)
+* **link_cow=BOOL**: When enabled if a regular file is opened which has a link count > 1 it will copy the file to a temporary file and rename over the original. Breaking the link and providing a basic copy-on-write function similar to cow-shell. (default: false)
 * **statfs=base|full**: Controls how statfs works. 'base' means it will always use all branches in statfs calculations. 'full' is in effect path preserving and only includes drives where the path exists. (default: base)
 * **statfs_ignore=none|ro|nc**: 'ro' will cause statfs calculations to ignore available space for branches mounted or tagged as 'read-only' or 'no create'. 'nc' will ignore available space for branches tagged as 'no create'. (default: none)
-* **posix_acl=true|false:** enable POSIX ACL support (if supported by kernel and underlying filesystem). (default: false)
-* **threads=num**: number of threads to use in multithreaded mode. When set to zero (the default) it will attempt to discover and use the number of logical cores. If the lookup fails it will fall back to using 4. If the thread count is set negative it will look up the number of cores then divide by the absolute value. ie. threads=-2 on an 8 core machine will result in 8 / 2 = 4 threads. There will always be at least 1 thread. NOTE: higher number of threads increases parallelism but usually decreases throughput. (default: number of cores) *NOTE2:* the option is unavailable when built with system libfuse.
-* **fsname=name**: sets the name of the filesystem as seen in **mount**, **df**, etc. Defaults to a list of the source paths concatenated together with the longest common prefix removed.
-* **func.&lt;func&gt;=&lt;policy&gt;**: sets the specific FUSE function's policy. See below for the list of value types. Example: **func.getattr=newest**
-* **category.&lt;category&gt;=&lt;policy&gt;**: Sets policy of all FUSE functions in the provided category. Example: **category.create=mfs**
-* **cache.open=&lt;int&gt;**: 'open' policy cache timeout in seconds. (default: 0)
-* **cache.statfs=&lt;int&gt;**: 'statfs' cache timeout in seconds. (default: 0)
-* **cache.attr=&lt;int&gt;**: file attribute cache timeout in seconds. (default: 1)
-* **cache.entry=&lt;int&gt;**: file name lookup cache timeout in seconds. (default: 1)
-* **cache.negative_entry=&lt;int&gt;**: negative file name lookup cache timeout in seconds. (default: 0)
+* **posix_acl=BOOL**: Enable POSIX ACL support (if supported by kernel and underlying filesystem). (default: false)
+* **async_read=BOOL**: Perform reads asynchronously. If disabled or unavailable the kernel will ensure there is at most one pending read request per file handle and will attempt to order requests by offset. (default: true)
+* **fuse_msg_size=INT**: Set the max number of pages per FUSE message. Only available on Linux >= 4.20 and ignored otherwise. (min: 1; max: 256; default: 256)
+* **threads=INT**: Number of threads to use in multithreaded mode. When set to zero it will attempt to discover and use the number of logical cores. If the lookup fails it will fall back to using 4. If the thread count is set negative it will look up the number of cores then divide by the absolute value. ie. threads=-2 on an 8 core machine will result in 8 / 2 = 4 threads. There will always be at least 1 thread. NOTE: higher number of threads increases parallelism but usually decreases throughput. (default: 0)
+* **fsname=STR**: Sets the name of the filesystem as seen in **mount**, **df**, etc. Defaults to a list of the source paths concatenated together with the longest common prefix removed.
+* **func.FUNC=POLICY**: Sets the specific FUSE function's policy. See below for the list of value types. Example: **func.getattr=newest**
+* **category.CATEGORY=POLICY**: Sets policy of all FUSE functions in the provided category. See POLICIES section for defaults. Example: **category.create=mfs**
+* **cache.open=INT**: 'open' policy cache timeout in seconds. (default: 0)
+* **cache.statfs=INT**: 'statfs' cache timeout in seconds. (default: 0)
+* **cache.attr=INT**: File attribute cache timeout in seconds. (default: 1)
+* **cache.entry=INT**: File name lookup cache timeout in seconds. (default: 1)
+* **cache.negative_entry=INT**: Negative file name lookup cache timeout in seconds. (default: 0)
+* **cache.files=libfuse|off|partial|full|auto-full**: File page caching mode (default: libfuse)
+* **cache.symlinks=BOOL**: Cache symlinks (if supported by kernel) (default: false)
+* **cache.readdir=BOOL**: Cache readdir (if supported by kernel) (default: false)
+* **direct_io**: deprecated - Bypass page cache. Use `cache.files=off` instead. (default: false)
+* **kernel_cache**: deprecated - Do not invalidate data cache on file open. Use `cache.files=full` instead. (default: false)
+* **auto_cache**: deprecated - Invalidate data cache if file mtime or size change. Use `cache.files=auto-full` instead. (default: false)
+* **async_read**: deprecated - Perform reads asynchronously. Use `async_read=true` instead.
+* **sync_read**: deprecated - Perform reads synchronously. Use `async_read=false` instead.
+

 **NOTE:** Options are evaluated in the order listed so if the options are **func.rmdir=rand,category.action=ff** the **action** category setting will override the **rmdir** setting.


+#### Value Types
+
+* BOOL = 'true' | 'false'
+* INT = [0,MAX_INT]
+* SIZE = 'NNM'; NN = INT, M = 'K' | 'M' | 'G' | 'T'
+* STR = string
+* FUNC = FUSE function
+* CATEGORY = FUSE function category
+* POLICY = mergerfs function policy
+
+
 ### branches

 The 'branches' (formerly 'srcmounts') argument is a colon (':') delimited list of paths to be pooled together. It does not matter if the paths are on the same or different drives nor does it matter the filesystem. Used and available space will not be duplicated for paths on the same device and any features which aren't supported by the underlying filesystem (such as file attributes or extended attributes) will return the appropriate errors.
@@ -121,6 +142,15 @@ To have the pool mounted at boot or otherwise accessable from related tools use
 **NOTE:** for mounting via **fstab** to work you must have **mount.fuse** installed. For Ubuntu/Debian it is included in the **fuse** package.


+### fuse_msg_size
+
+FUSE applications communicate with the kernel over a special character device: `/dev/fuse`. A large portion of the overhead associated with FUSE is the cost of going back and forth from user space and kernel space over that device. Generally speaking the fewer trips needed the better the performance will be. Reducing the number of trips can be done a number of ways. Kernel level caching and increasing message sizes being two significant ones. When it comes to reads and writes if the message size is doubled the number of trips are appoximately halved.
+
+In Linux 4.20 a new feature was added allowing the negotiation of the max message size. Since the size is in multiples of [pages](https://en.wikipedia.org/wiki/Page_(computer_memory)) the feature is called `max_pages`. There is a maximum `max_pages` value of 256 (1MiB) and minimum of 1 (4KiB). The default used by Linux >=4.20, and hardcoded value used before 4.20, is 32 (128KiB). In mergerfs its referred to as `fuse_msg_size` to make it clear what it impacts and provide some abstraction.
+
+Since there should be no downsides to increasing `fuse_msg_size` / `max_pages`, outside a minor bump in RAM usage due to larger message buffers, mergerfs defaults the value to 256. On kernels before 4.20 the value has no effect. The reason the value is configurable is to enable experimentation and benchmarking. See the `nullrw` section for benchmarking examples.
+
+
 ### symlinkify

 Due to the levels of indirection introduced by mergerfs and the underlying technology FUSE there can be varying levels of performance degredation. This feature will turn non-directories which are not writable into symlinks to the original file found by the `readlink` policy after the mtime and ctime are older than the timeout.
@@ -138,22 +168,22 @@ By enabling `nullrw` mergerfs will work as it always does **except** that all re

 Example:
 ```
-$ dd if=/dev/zero of=/path/to/mergerfs/mount/benchmark ibs=1M obs=512 count=1024 conv=fdatasync
+$ dd if=/dev/zero of=/path/to/mergerfs/mount/benchmark ibs=1M obs=512 count=1024 iflag=dsync,nocache oflag=dsync,nocache conv=fdatasync status=progress
 1024+0 records in
 2097152+0 records out
 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 15.4067 s, 69.7 MB/s

-$ dd if=/dev/zero of=/path/to/mergerfs/mount/benchmark ibs=1M obs=1M count=1024 conv=fdatasync
+$ dd if=/dev/zero of=/path/to/mergerfs/mount/benchmark ibs=1M obs=1M count=1024 iflag=dsync,nocache oflag=dsync,nocache conv=fdatasync status=progress
 1024+0 records in
 1024+0 records out
 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.219585 s, 4.9 GB/s

-$ dd if=/path/to/mergerfs/mount/benchmark of=/dev/null bs=512 count=102400 conv=fdatasync
+$ dd if=/path/to/mergerfs/mount/benchmark of=/dev/null bs=512 count=102400 iflag=dsync,nocache oflag=dsync,nocache conv=fdatasync status=progress
 102400+0 records in
 102400+0 records out
 52428800 bytes (52 MB, 50 MiB) copied, 0.757991 s, 69.2 MB/s

-$ dd if=/path/to/mergerfs/mount/benchmark of=/dev/null bs=1M count=1024 conv=fdatasync
+$ dd if=/path/to/mergerfs/mount/benchmark of=/dev/null bs=1M count=1024 iflag=dsync,nocache oflag=dsync,nocache conv=fdatasync status=progress
 1024+0 records in
 1024+0 records out
 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.18405 s, 5.8 GB/s
@@ -186,7 +216,7 @@ When using policies which are based on a branch's available space the base path
 | action   | chmod, chown, link, removexattr, rename, rmdir, setxattr, truncate, unlink, utimens |
 | create   | create, mkdir, mknod, symlink                                                       |
 | search   | access, getattr, getxattr, ioctl (directories), listxattr, open, readlink                         |
-| N/A      | fchmod, fchown, futimens, ftruncate, fallocate, fgetattr, fsync, ioctl (files), read, readdir, release, statfs, write |
+| N/A      | fchmod, fchown, futimens, ftruncate, fallocate, fgetattr, fsync, ioctl (files), read, readdir, release, statfs, write, copy_file_range |

 In cases where something may be searched (to confirm a directory exists across all source mounts) **getattr** will be used.

@@ -223,14 +253,14 @@ If all branches are filtered an error will be returned. Typically **EROFS** or *
 | eplfs (existing path, least free space) | Of all the branches on which the relative path exists choose the drive with the least free space. |
 | eplus (existing path, least used space) | Of all the branches on which the relative path exists choose the drive with the least used space. |
 | epmfs (existing path, most free space) | Of all the branches on which the relative path exists choose the drive with the most free space. |
-| eprand (existing path, random) | Calls **epall** and then randomizes. |
+| eprand (existing path, random) | Calls **epall** and then randomizes. Returns 1. |
 | erofs | Exclusively return **-1** with **errno** set to **EROFS** (read-only filesystem). |
 | ff (first found) | Search category: same as **epff**. Action category: same as **epff**. Create category: Given the order of the drives, as defined at mount time or configured at runtime, act on the first one found. |
 | lfs (least free space) | Search category: same as **eplfs**. Action category: same as **eplfs**. Create category: Pick the drive with the least available free space. |
 | lus (least used space) | Search category: same as **eplus**. Action category: same as **eplus**. Create category: Pick the drive with the least used space. |
 | mfs (most free space) | Search category: same as **epmfs**. Action category: same as **epmfs**. Create category: Pick the drive with the most available free space. |
 | newest | Pick the file / directory with the largest mtime. |
-| rand (random) | Calls **all** and then randomizes. |
+| rand (random) | Calls **all** and then randomizes. Returns 1. |


 #### Defaults ####
@@ -303,12 +333,12 @@ The above behavior will help minimize the likelihood of EXDEV being returned but

 #### readdir ####

-[readdir](http://linux.die.net/man/3/readdir) is different from all other filesystem functions. While it could have it's own set of policies to tweak its behavior at this time it provides a simple union of files and directories found. Remember that any action or information queried about these files and directories come from the respective function. For instance: an **ls** is a **readdir** and for each file/directory returned **getattr** is called. Meaning the policy of **getattr** is responsible for choosing the file/directory which is the source of the metadata you see in an **ls**.
+[readdir](http://linux.die.net/man/3/readdir) is different from all other filesystem functions. While it could have its own set of policies to tweak its behavior at this time it provides a simple union of files and directories found. Remember that any action or information queried about these files and directories come from the respective function. For instance: an **ls** is a **readdir** and for each file/directory returned **getattr** is called. Meaning the policy of **getattr** is responsible for choosing the file/directory which is the source of the metadata you see in an **ls**.


 #### statfs / statvfs ####

-[statvfs](http://linux.die.net/man/2/statvfs) normalizes the source drives based on the fragment size and sums the number of adjusted blocks and inodes. This means you will see the combined space of all sources. Total, used, and free. The sources however are dedupped based on the drive so multiple sources on the same drive will not result in double counting it's space. Filesystems mounted further down the tree of the branch will not be included when checking the mount's stats.
+[statvfs](http://linux.die.net/man/2/statvfs) normalizes the source drives based on the fragment size and sums the number of adjusted blocks and inodes. This means you will see the combined space of all sources. Total, used, and free. The sources however are dedupped based on the drive so multiple sources on the same drive will not result in double counting its space. Filesystems mounted further down the tree of the branch will not be included when checking the mount's stats.

 The options `statfs` and `statfs_ignore` can be used to modify `statfs` behavior.

@@ -344,7 +374,7 @@ $ su -

 #### Generically

-Have git, g++, make, python, automake, libtool installed.
+Have git, g++, make, python installed.

 ```
 $ cd mergerfs
@@ -379,7 +409,12 @@ Any changes made at runtime are **not** persisted. If you wish for values to per

 ##### Keys #####

-Use `xattr -l /mountpoint/.mergerfs` to see all supported keys. Some are informational and therefore read-only.
+Use `xattr -l /mountpoint/.mergerfs` to see all supported keys. Some are informational and therefore read-only. `setxattr` will return EINVAL on read-only keys.
+
+
+##### Values #####
+
+Same as the command line.


 ###### user.mergerfs.branches ######
@@ -402,32 +437,11 @@ Used to query or modify the list of branches. When modifying there are several s
 The `=NC`, `=RO`, `=RW` syntax works just as on the command line.


-###### minfreespace ######
-
-Input: interger with an optional multiplier suffix. **K**, **M**, or **G**.
-
-Output: value in bytes
-
-
-###### moveonenospc ######
-
-Input: **true** and **false**
-
-Ouput: **true** or **false**
-
-
-###### categories / funcs ######
-
-Input: short policy string as described elsewhere in this document
-
-Output: the policy string except for categories where its funcs have multiple types. In that case it will be a comma separated list
-
-
 ##### Example #####

 ```
 [trapexit:/mnt/mergerfs] $ xattr -l .mergerfs
-user.mergerfs.branches: /mnt/a:/mnt/b
+user.mergerfs.branches: /mnt/a=RW:/mnt/b=RW
 user.mergerfs.minfreespace: 4294967295
 user.mergerfs.moveonenospc: false
 ...
@@ -457,10 +471,10 @@ newest

 While they won't show up when using [listxattr](http://linux.die.net/man/2/listxattr) **mergerfs** offers a number of special xattrs to query information about the files served. To access the values you will need to issue a [getxattr](http://linux.die.net/man/2/getxattr) for one of the following:

-* **user.mergerfs.basepath:** the base mount point for the file given the current getattr policy
-* **user.mergerfs.relpath:** the relative path of the file from the perspective of the mount point
-* **user.mergerfs.fullpath:** the full path of the original file given the getattr policy
-* **user.mergerfs.allpaths:** a NUL ('\0') separated list of full paths to all files found
+* **user.mergerfs.basepath**: the base mount point for the file given the current getattr policy
+* **user.mergerfs.relpath**: the relative path of the file from the perspective of the mount point
+* **user.mergerfs.fullpath**: the full path of the original file given the getattr policy
+* **user.mergerfs.allpaths**: a NUL ('\0') separated list of full paths to all files found

 ```
 [trapexit:/mnt/mergerfs] $ ls
@@ -477,6 +491,20 @@ A B C
 ```


+# UPGRADE
+
+mergerfs can be upgraded live by mounting on top of the previous version. Simply install the new version of mergerfs and follow the instructions below.
+
+Add `nonempty` to your mergerfs option list and call mergerfs again or if using `/etc/fstab` call for it to mount again. Existing open files and such will continue to work fine though they won't see runtime changes since any such change would be the new mount. If you plan on changing settings with the new mount you should / could apply those before mounting the new version.
+
+```
+$ sudo mount /mnt/mergerfs
+$ mount | grep mergerfs
+media on /mnt/mergerfs type fuse.mergerfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other)
+media on /mnt/mergerfs type fuse.mergerfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other)
+```
+
+
 # TOOLING

 * https://github.com/trapexit/mergerfs-tools
@@ -485,6 +513,7 @@ A B C
  * mergerfs.dedup: Will help identify and optionally remove duplicate files
  * mergerfs.dup: Ensure there are at least N copies of a file across the pool
  * mergerfs.balance: Rebalance files across drives by moving them from the most filled to the least filled
+  * mergerfs.consolidate: move files within a single mergerfs directory to the drive with most free space
  * mergerfs.mktrash: Creates FreeDesktop.org Trash specification compatible directories on a mergerfs mount
 * https://github.com/trapexit/scorch
  * scorch: A tool to help discover silent corruption of files and keep track of files
@@ -496,11 +525,27 @@ A B C

 #### page caching

-The kernel performs caching of data pages on all files not opened with `O_DIRECT`. Due to mergerfs using FUSE and therefore being a userland process the kernel can double cache the content being read through mergerfs. Once from the underlying filesystem and once for mergerfs. Using `direct_io` and/or `dropcacheonclose` help minimize the double caching. `direct_io` will instruct the kernel to bypass the page cache for files opened through mergerfs. `dropcacheonclose` will cause mergerfs to instruct the kernel to flush a file's page cache for which it had opened when closed. If most data is read once its probably best to enable both (read above for details and limitations).
+https://en.wikipedia.org/wiki/Page_cache

-If a cache is desired for mergerfs do not enable `direct_io` and instead possibly use `auto_cache` or `kernel_cache`. By default FUSE will invalidate cached pages when a file is opened. By using `auto_cache` it will instead use `getattr` to check if a file has changed when the file is opened and if so will flush the cache. `ac_attr_timeout` is the timeout for keeping said cache. Alternatively `kernel_cache` will keep the cache across opens unless invalidated through other means. You should only uses these if you do not plan to write/modify the same files through mergerfs and the underlying filesystem at the same time. It could lead to corruption. Then again doing so without caching can also cause issues.
+tl;dr:
+* cache.files=off: Disables page caching. Underlying files cached, mergerfs files are not.
+* cache.files=partial: Enables page caching. Underlying files cached, mergerfs files cached while open.
+* cache.files=full: Enables page caching. Underlying files cached, mergerfs files cached across opens.
+* cache.files=auto-full: Enables page caching. Underlying files cached, mergerfs files cached across opens if mtime and size are unchanged since previous open.
+* cache.files=libfuse: follow traditional libfuse `direct_io`, `kernel_cache`, and `auto_cache` arguments.

-It's a difficult balance between memory usage, cache bloat & duplication, and performance. Ideally mergerfs would be able to disable caching for the files it reads/writes but allow page caching for itself. That would limit the FUSE overhead. However, there isn't good way to achieve this.
+
+FUSE, which mergerfs uses, offers a number of page caching modes. mergerfs tries to simplify their use via the `cache.files` option. It can and should replace usage of `direct_io`, `kernel_cache`, and `auto_cache`.
+
+Due to mergerfs using FUSE and therefore being a userland process proxying existing filesystems the kernel will double cache the content being read and written through mergerfs. Once from the underlying filesystem and once from mergerfs (it sees them as two separate entities). Using `cache.files=off` will keep the double caching from happening by disabling caching of mergerfs but this has the side effect that *all* read and write calls will be passed to mergerfs which may be slower than enabling caching, you lose shared `mmap` support which can affect apps such as rtorrent, and no read-ahead will take place. The kernel will still cache the underlying filesystem data but that only helps so much given mergerfs will still process all requests.
+
+If you do enable file page caching, `cache.files=partial|full|auto-full`, you should also enable `dropcacheonclose` which will cause mergerfs to instruct the kernel to flush the underlying file's page cache when the file is closed. This behavior is the same as the rsync fadvise / drop cache patch and Feh's nocache project.
+
+If most files are read once through and closed (like media) it is best to enable `dropcacheonclose` regardless of caching mode in order to minimize buffer bloat.
+
+It is difficult to balance memory usage, cache bloat & duplication, and performance. Ideally mergerfs would be able to disable caching for the files it reads/writes but allow page caching for itself. That would limit the FUSE overhead. However, there isn't a good way to achieve this. It would need to open all files with O_DIRECT which places limitations on the what underlying filesystems would be supported and complicates the code.
+
+kernel documenation: https://www.kernel.org/doc/Documentation/filesystems/fuse-io.txt


 #### entry & attribute caching
@@ -524,9 +569,19 @@ Of the syscalls used by mergerfs in policies the `statfs` / `statvfs` call is pe
 Example: If the create policy is `mfs` and the timeout is 60 then for that 60 seconds the same drive will be returned as the target for creates because the available space won't be updated for that time.


+#### symlink caching
+
+As of version 4.20 Linux supports symlink caching. Significant performance increases can be had in workloads which use a lot of symlinks. Setting `cache.symlinks=true` will result in requesting symlink caching from the kernel only if supported. As a result its safe to enable it on systems prior to 4.20. That said it is disabled by default for now. You can see if caching is enabled by querying the xattr `user.mergerfs.cache.symlinks` but given it must be requested at startup you can not change it at runtime.
+
+
+#### readdir caching
+
+As of version 4.20 Linux supports readdir caching. This can have a significant impact on directory traversal. Especially when combined with entry (`cache.entry`) and attribute (`cache.attr`) caching. Setting `cache.readdir=true` will result in requesting readdir caching from the kernel on each `opendir`. If the kernel doesn't support readdir caching setting the option to `true` has no effect. This option is configuarable at runtime via xattr `user.mergerfs.cache.readdir`.
+
+
 #### writeback caching

-writeback caching is a technique for improving write speeds by batching writes at a faster device and then bulk writing to the slower device. With FUSE the kernel will wait for a number of writes to be made and then send it to the filesystem as one request. mergerfs currently uses a slightly modified and vendored libfuse 2.9.7 which does not support writeback caching. However, a prototype port to libfuse 3.x has been made and the writeback cache appears to work as expected (though performance improvements greatly depend on the way the client app writes data). Once the port is complete and thoroughly tested writeback caching will be available.
+writeback caching is a technique for improving write speeds by batching writes at a faster device and then bulk writing to the slower device. With FUSE the kernel will wait for a number of writes to be made and then send it to the filesystem as one request. mergerfs currently uses a  modified and vendored libfuse 2.9.7 which does not support writeback caching. Adding said feature should not be difficult but benchmarking needs to be done to see if what effect it will have.


 #### tiered caching
@@ -607,7 +662,7 @@ done
 * Run mergerfs as `root` (with **allow_other**) unless you're merging paths which are owned by the same user otherwise strange permission issues may arise.
 * https://github.com/trapexit/backup-and-recovery-howtos : A set of guides / howtos on creating a data storage system, backing it up, maintaining it, and recovering from failure.
 * If you don't see some directories and files you expect in a merged point or policies seem to skip drives be sure the user has permission to all the underlying directories. Use `mergerfs.fsck` to audit the drive for out of sync permissions.
-* Do **not** use `direct_io` if you expect applications (such as rtorrent) to [mmap](http://linux.die.net/man/2/mmap) files. It is not currently supported in FUSE w/ `direct_io` enabled. Enabling `dropcacheonclose` is recommended when `direct_io` is disabled.
+* Do **not** use `cache.files=off` or `direct_io` if you expect applications (such as rtorrent) to [mmap](http://linux.die.net/man/2/mmap) files. Shared mmap is not currently supported in FUSE w/ `direct_io` enabled. Enabling `dropcacheonclose` is recommended when `cache.files=partial|full|auto-full` or `direct_io=false`.
 * Since POSIX functions give only a singular error or success its difficult to determine the proper behavior when applying the function to multiple targets. **mergerfs** will return an error only if all attempts of an action fail. Any success will lead to a success returned. This means however that some odd situations may arise.
 * [Kodi](http://kodi.tv), [Plex](http://plex.tv), [Subsonic](http://subsonic.org), etc. can use directory [mtime](http://linux.die.net/man/2/stat) to more efficiently determine whether to scan for new content rather than simply performing a full scan. If using the default **getattr** policy of **ff** its possible those programs will miss an update on account of it returning the first directory found's **stat** info and its a later directory on another mount which had the **mtime** recently updated. To fix this you will want to set **func.getattr=newest**. Remember though that this is just **stat**. If the file is later **open**'ed or **unlink**'ed and the policy is different for those then a completely different file or directory could be acted on.
 * Some policies mixed with some functions may result in strange behaviors. Not that some of these behaviors and race conditions couldn't happen outside **mergerfs** but that they are far more likely to occur on account of the attempt to merge together multiple sources of data which could be out of sync due to the different policies.
@@ -646,11 +701,7 @@ If you want to move files to one drive just copy them there and use mergerfs.ded

 #### cached memory appears greater than it should be

-Use the `direct_io` option as described above. Due to what mergerfs is doing there ends up being two caches of a file under normal usage. One from the underlying filesystem and one from mergerfs. Enabling `direct_io` removes the mergerfs cache. This saves on memory but means the kernel needs to communicate with mergerfs more often and can therefore result in slower speeds.
-
-Since enabling `direct_io` disables `mmap` this is not an ideal situation however write speeds should be increased.
-
-If `direct_io` is disabled it is probably a good idea to enable `dropcacheonclose` to minimize double caching.
+Use `cache.files=off` or `direct_io=true`. See the section on page caching.


 #### NFS clients returning ESTALE / Stale file handle
@@ -667,7 +718,7 @@ Try enabling the `use_ino` option. Some have reported that it fixes the issue.

 #### rtorrent fails with ENODEV (No such device)

-Be sure to turn off `direct_io`. rtorrent and some other applications use [mmap](http://linux.die.net/man/2/mmap) to read and write to files and offer no failback to traditional methods. FUSE does not currently support mmap while using `direct_io`. There may be a performance penalty on writes with `direct_io` off as well as the problem of double caching but it's the only way to get such applications to work. If the performance loss is too high for other apps you can mount mergerfs twice. Once with `direct_io` enabled and one without it. Be sure to set `dropcacheonclose=true` if not using `direct_io`.
+Be sure to set `cache.files=partial|full|auto-full` or turn off `direct_io`. rtorrent and some other applications use [mmap](http://linux.die.net/man/2/mmap) to read and write to files and offer no failback to traditional methods. FUSE does not currently support mmap while using `direct_io`. There may be a performance penalty on writes with `direct_io` off as well as the problem of double caching but it's the only way to get such applications to work. If the performance loss is too high for other apps you can mount mergerfs twice. Once with `direct_io` enabled and one without it. Be sure to set `dropcacheonclose=true` if not using `direct_io`.


 #### rtorrent fails with files >= 4GiB
@@ -702,7 +753,7 @@ Ideally the offending software would be fixed and it is recommended that if you

 Workaround: Copy the file/directory and then remove the original rather than move.

-This isn't an issue with Samba but some SMB clients. GVFS-fuse v1.20.3 and prior (found in Ubuntu 14.04 among others) failed to handle certain error codes correctly. Particularly **STATUS_NOT_SAME_DEVICE** which comes from the **EXDEV** which is returned by **rename** when the call is crossing mount points. When a program gets an **EXDEV** it needs to explicitly take an alternate action to accomplish it's goal. In the case of **mv** or similar it tries **rename** and on **EXDEV** falls back to a manual copying of data between the two locations and unlinking the source. In these older versions of GVFS-fuse if it received **EXDEV** it would translate that into **EIO**. This would cause **mv** or most any application attempting to move files around on that SMB share to fail with a IO error.
+This isn't an issue with Samba but some SMB clients. GVFS-fuse v1.20.3 and prior (found in Ubuntu 14.04 among others) failed to handle certain error codes correctly. Particularly **STATUS_NOT_SAME_DEVICE** which comes from the **EXDEV** which is returned by **rename** when the call is crossing mount points. When a program gets an **EXDEV** it needs to explicitly take an alternate action to accomplish its goal. In the case of **mv** or similar it tries **rename** and on **EXDEV** falls back to a manual copying of data between the two locations and unlinking the source. In these older versions of GVFS-fuse if it received **EXDEV** it would translate that into **EIO**. This would cause **mv** or most any application attempting to move files around on that SMB share to fail with a IO error.

 [GVFS-fuse v1.22.0](https://bugzilla.gnome.org/show_bug.cgi?id=734568) and above fixed this issue but a large number of systems use the older release. On Ubuntu the version can be checked by issuing `apt-cache showpkg gvfs-fuse`. Most distros released in 2015 seem to have the updated release and will work fine but older systems may not. Upgrading gvfs-fuse or the distro in general will address the problem.

@@ -796,7 +847,7 @@ Using the **hard_remove** option will make it so these temporary files are not u

 #### How well does mergerfs scale? Is it "production ready?"

-Users have reported running mergerfs on everything from a Raspberry Pi to dual socket Xeon systems with >20 cores. I'm aware of at least a few companies which use mergerfs in production. [Open Media Vault](https://www.openmediavault.org) includes mergerfs is it's sole solution for pooling drives.
+Users have reported running mergerfs on everything from a Raspberry Pi to dual socket Xeon systems with >20 cores. I'm aware of at least a few companies which use mergerfs in production. [Open Media Vault](https://www.openmediavault.org) includes mergerfs as its sole solution for pooling drives.


 #### Can mergerfs be used with drives which already have data / are in use?
@@ -815,7 +866,7 @@ See the previous question's answer.

 Yes. You need to use `use_ino` to support proper reporting of inodes.

-What mergerfs does not do is fake hard links across branches.  Read the section "rename & link" for how it.
+What mergerfs does not do is fake hard links across branches.  Read the section "rename & link" for how it works.


 #### Does mergerfs support CoW / copy-on-write?
@@ -825,7 +876,7 @@ Not in the sense of a filesystem like BTRFS or ZFS nor in the overlayfs or aufs

 #### Why can't I see my files / directories?

-It's almost always a permissions issue. Unlike mhddfs, which runs as root and attempts to access content as such, mergerfs always changes it's credentials to that of the caller. This means that if the user does not have access to a file or directory than neither will mergerfs. However, because mergerfs is creating a union of paths it may be able to read some files and directories on one drive but not another resulting in an incomplete set.
+It's almost always a permissions issue. Unlike mhddfs, which runs as root and attempts to access content as such, mergerfs always changes its credentials to that of the caller. This means that if the user does not have access to a file or directory than neither will mergerfs. However, because mergerfs is creating a union of paths it may be able to read some files and directories on one drive but not another resulting in an incomplete set.

 Whenever you run into a split permission issue (seeing some but not all files) try using [mergerfs.fsck](https://github.com/trapexit/mergerfs-tools) tool to check for and fix the mismatch. If you aren't seeing anything at all be sure that the basic permissions are correct. The user and group values are correct and that directories have their executable bit set. A common mistake by users new to Linux is to `chmod -R 644` when they should have `chmod -R u=rwX,go=rX`.

@@ -889,7 +940,7 @@ MergerFS is not intended to be a replacement for ZFS. MergerFS is intended to pr

 #### Can drives be written to directly? Outside of mergerfs while pooled?

-Yes, however its not recommended to use the same file from within the pool and from without at the same time. Especially if using caching of any kind (cache.entry, cache.attr, ac_attr_timeout, cache.negative_entry, auto_cache, kernel_cache).
+Yes, however its not recommended to use the same file from within the pool and from without at the same time. Especially if using caching of any kind (cache.files, cache.entry, cache.attr, cache.negative_entry, cache.symlinks, cache.readdir, etc.).


 #### Why do I get an "out of space" / "no space left on device" / ENOSPC error even though there appears to be lots of space available?
@@ -941,7 +992,7 @@ and the kernel use internally (also called the "nodeid").

 #### I notice massive slowdowns of writes over NFS

-Due to how NFS works and interacts with FUSE when not using `direct_io` its possible that a getxattr for `security.capability` will be issued prior to any write. This will usually result in a massive slowdown for writes. Using `direct_io` will keep this from happening (and generally good to enable unless you need the features it disables) but the `security_capability` option can also help by short circuiting the call and returning `ENOATTR`.
+Due to how NFS works and interacts with FUSE when not using `cache.files=off` or `direct_io` its possible that a getxattr for `security.capability` will be issued prior to any write. This will usually result in a massive slowdown for writes. Using `cache.files=off` or `direct_io` will keep this from happening (and generally good to enable unless you need the features it disables) but the `security_capability` option can also help by short circuiting the call and returning `ENOATTR`.

 You could also set `xattr` to `noattr` or `nosys` to short circuit or stop all xattr requests.

@@ -968,15 +1019,17 @@ For non-Linux systems mergerfs uses a read-write lock and changes credentials on

 NOTE: be sure to read about these features before changing them

-* add (or remove) `direct_io`
-* add (or remove) `auto_cache`
-* add (or remove) `kernel_cache`
-* add (or remove) `splice_move`, `splice_read`, and `splice_write`
+* enable (or disable) `splice_move`, `splice_read`, and `splice_write`
 * increase cache timeouts `cache.attr`, `cache.entry`, `cache.negative_entry`
-* enable `cache.open` and/or `cache.statfs`
-* change the number opf worker threads
+* enable (or disable) page caching (`cache.files`)
+* enable `cache.open`
+* enable `cache.statfs`
+* enable `cache.symlinks`
+* enable `cache.readdir`
+* change the number of worker threads
 * disable `security_capability` and/or `xattr`
 * disable `posix_acl`
+* disable `async_read`
 * test theoretical performance using `nullrw` or mounting a ram disk
 * use `symlinkify` if your data is largely static
 * use tiered cache drives
@@ -1,5 +1,5 @@
-VERSION = "2.9.7-mergerfs_2.27.0"
-OPT = -O2
+VERSION = "2.9.7-mergerfs_2.28.0"
+OPTS = -O2

 ifeq ($(DEBUG),1)
 DEBUG_FLAGS := -g
@@ -28,7 +28,7 @@ OBJS = $(SRC:lib/%.c=build/%.o)
 DEPS = $(SRC:lib/%.c=build/%.d)

 CFLAGS += \
-	$(OPT) \
+	$(OPTS) \
 	$(DEBUG_FLAGS) \
 	-Wall \
 	-pipe \
@@ -129,7 +129,7 @@ struct fuse_operations {
        * while open. Helps manage the fact the kernel usually does
        * not send fh with getattr requests.
        */
-        int (*prepare_hide)(const char *name_, uint64_t *fh_, int type_);
+        int (*prepare_hide)(const char *name_, uint64_t *fh_);
        int (*free_hide)(const uint64_t fh_);

 	/** Remove a file */
@@ -608,6 +608,30 @@ struct fuse_operations {
 	 */
 	int (*fallocate) (const char *, int, off_t, off_t,
 			  struct fuse_file_info *);
+
+  /**
+   * Copy a range of data from one file to another
+   *
+   * Performs an optimized copy between two file descriptors without
+   * the
+   * additional cost of transferring data through the FUSE kernel
+   * module
+   * to user space (glibc) and then back into the FUSE filesystem
+   * again.
+   *
+   * In case this method is not implemented, glibc falls back to
+   * reading
+   * data from the source and writing to the destination. Effectively
+   * doing an inefficient copy of the data.
+   */
+  ssize_t (*copy_file_range)(const char            *path_in,
+                             struct fuse_file_info *fi_in,
+                             off_t                  offset_in,
+                             const char            *path_out,
+                             struct fuse_file_info *fi_out,
+                             off_t                  offset_out,
+                             size_t                 size,
+                             int                    flags);
 };

 /** Extra context that may be needed by some filesystems
@@ -921,6 +945,15 @@ int fuse_fs_fallocate(struct fuse_fs *fs, const char *path, int mode,
 void fuse_fs_init(struct fuse_fs *fs, struct fuse_conn_info *conn);
 void fuse_fs_destroy(struct fuse_fs *fs);

+int fuse_fs_prepare_hide(struct fuse_fs *fs, const char *path, uint64_t *fh);
+int fuse_fs_free_hide(struct fuse_fs *fs, uint64_t fh);
+ssize_t fuse_fs_copy_file_range(struct fuse_fs *fs,
+                                const char *path_in,
+                                struct fuse_file_info *fi_in, off_t off_in,
+                                const char *path_out,
+                                struct fuse_file_info *fi_out, off_t off_out,
+                                size_t len, int flags);
+
 int fuse_notify_poll(struct fuse_pollhandle *ph);

 /**
@@ -33,6 +33,9 @@
 #error Please add -D_FILE_OFFSET_BITS=64 to your compile flags!
 #endif

+#define FUSE_DEFAULT_MAX_PAGES_PER_REQ 32
+#define FUSE_MAX_MAX_PAGES 256
+
 #ifdef __cplusplus
 extern "C" {
 #endif
@@ -42,49 +45,53 @@ extern "C" {
 *
 * Changed in version 2.5
 */
-struct fuse_file_info {
-	/** Open flags.	 Available in open() and release() */
-	int flags;
+struct
+fuse_file_info
+{
+  /** Open flags. Available in open() and release() */
+  int flags;

-	/** Old file handle, don't use */
-	unsigned long fh_old;
+  /** In case of a write operation indicates if this was caused by a
+      writepage */
+  uint32_t writepage : 1;

-	/** In case of a write operation indicates if this was caused by a
-	    writepage */
-	int writepage;
+  /** Can be filled in by open, to use direct I/O on this file.
+      Introduced in version 2.4 */
+  uint32_t direct_io : 1;

-	/** Can be filled in by open, to use direct I/O on this file.
-	    Introduced in version 2.4 */
-	unsigned int direct_io : 1;
+  /** Can be filled in by open, to indicate, that cached file data
+      need not be invalidated.  Introduced in version 2.4 */
+  uint32_t keep_cache : 1;

-	/** Can be filled in by open, to indicate, that cached file data
-	    need not be invalidated.  Introduced in version 2.4 */
-	unsigned int keep_cache : 1;
+  /** Indicates a flush operation.  Set in flush operation, also
+      maybe set in highlevel lock operation and lowlevel release
+      operation.	Introduced in version 2.6 */
+  uint32_t flush : 1;

-	/** Indicates a flush operation.  Set in flush operation, also
-	    maybe set in highlevel lock operation and lowlevel release
-	    operation.	Introduced in version 2.6 */
-	unsigned int flush : 1;
+  /** Can be filled in by open, to indicate that the file is not
+      seekable.  Introduced in version 2.8 */
+  uint32_t nonseekable : 1;

-	/** Can be filled in by open, to indicate that the file is not
-	    seekable.  Introduced in version 2.8 */
-	unsigned int nonseekable : 1;
+  /* Indicates that flock locks for this file should be
+     released.  If set, lock_owner shall contain a valid value.
+     May only be set in ->release().  Introduced in version
+     2.9 */
+  uint32_t flock_release : 1;

-	/* Indicates that flock locks for this file should be
-	   released.  If set, lock_owner shall contain a valid value.
-	   May only be set in ->release().  Introduced in version
-	   2.9 */
-	unsigned int flock_release : 1;
+  /* Requests the kernel to cache entries returned by readdir */
+  uint32_t cache_readdir : 1;

-	/** Padding.  Do not use*/
-	unsigned int padding : 27;
+  uint32_t auto_cache : 1;

-	/** File handle.  May be filled in by filesystem in open().
-	    Available in all other file operations */
-	uint64_t fh;
+  /** Padding.  Do not use*/
+  uint32_t padding : 24;

-	/** Lock owner id.  Available in locking operations and flush */
-	uint64_t lock_owner;
+  /** File handle.  May be filled in by filesystem in open().
+      Available in all other file operations */
+  uint64_t fh;
+
+  /** Lock owner id.  Available in locking operations and flush */
+  uint64_t lock_owner;
 };

 /**
@@ -100,6 +107,7 @@ struct fuse_file_info {
 * FUSE_CAP_SPLICE_MOVE: ability to move data to the fuse device with splice()
 * FUSE_CAP_SPLICE_READ: ability to use splice() to read from the fuse device
 * FUSE_CAP_IOCTL_DIR: ioctl support on directories
+ * FUSE_CAP_CACHE_SYMLINKS: cache READLINK responses
 */
 #define FUSE_CAP_ASYNC_READ      (1 << 0)
 #define FUSE_CAP_POSIX_LOCKS     (1 << 1)
@@ -115,6 +123,8 @@ struct fuse_file_info {
 #define FUSE_CAP_ASYNC_DIO       (1 << 15)
 #define FUSE_CAP_PARALLEL_DIROPS (1 << 18)
 #define FUSE_CAP_POSIX_ACL       (1 << 19)
+#define FUSE_CAP_CACHE_SYMLINKS  (1 << 20)
+#define FUSE_CAP_MAX_PAGES       (1 << 21)


 /**
@@ -152,11 +162,6 @@ struct fuse_conn_info {
 	 */
 	unsigned proto_minor;

-	/**
-	 * Is asynchronous read supported (read-write)
-	 */
-	unsigned async_read;
-
 	/**
 	 * Maximum size of the write buffer
 	 */
@@ -187,10 +192,15 @@ struct fuse_conn_info {
 	 */
 	unsigned congestion_threshold;

+        /**
+         * Max pages
+         */
+        uint16_t max_pages;
+
 	/**
 	 * For future use.
 	 */
-	unsigned reserved[23];
+	unsigned reserved[22];
 };

 struct fuse_session;
@@ -207,7 +217,8 @@ struct fuse_pollhandle;
 * @param args argument vector
 * @return the communication channel on success, NULL on failure
 */
-struct fuse_chan *fuse_mount(const char *mountpoint, struct fuse_args *args);
+struct fuse_chan *fuse_mount(const char       *mountpoint,
+                             struct fuse_args *args);

 /**
 * Umount a FUSE mountpoint
@@ -119,6 +119,12 @@
 *
 *  7.28
 *  - add FUSE_COPY_FILE_RANGE
+ *  - add FOPEN_CACHE_DIR
+ *  - add FUSE_MAX_PAGES, add max_pages to init_out
+ *  - add FUSE_CACHE_SYMLINKS
+ *
+ *  7.29
+ *  - add FUSE_NO_OPENDIR_SUPPORT flag
 */

 #ifndef _LINUX_FUSE_H
@@ -154,7 +160,7 @@
 #define FUSE_KERNEL_VERSION 7

 /** Minor version number of this interface */
-#define FUSE_KERNEL_MINOR_VERSION 27
+#define FUSE_KERNEL_MINOR_VERSION 29

 /** The node ID of the root inode */
 #define FUSE_ROOT_ID 1
@@ -222,10 +228,12 @@ struct fuse_file_lock {
 * FOPEN_DIRECT_IO: bypass page cache for this open file
 * FOPEN_KEEP_CACHE: don't invalidate the data cache on open
 * FOPEN_NONSEEKABLE: the file is not seekable
+ * FOPEN_CACHE_DIR: allow caching this directory
 */
 #define FOPEN_DIRECT_IO		(1 << 0)
 #define FOPEN_KEEP_CACHE	(1 << 1)
 #define FOPEN_NONSEEKABLE	(1 << 2)
+#define FOPEN_CACHE_DIR		(1 << 3)

 /**
 * INIT request/reply flags
@@ -252,6 +260,9 @@ struct fuse_file_lock {
 * FUSE_HANDLE_KILLPRIV: fs handles killing suid/sgid/cap on write/chown/trunc
 * FUSE_POSIX_ACL: filesystem supports posix acls
 * FUSE_ABORT_ERROR: reading the device after abort returns ECONNABORTED
+ * FUSE_MAX_PAGES: init_out.max_pages contains the max number of req pages
+ * FUSE_CACHE_SYMLINKS: cache READLINK responses
+ * FUSE_NO_OPENDIR_SUPPORT: kernel supports zero-message opendir
 */
 #define FUSE_ASYNC_READ		(1 << 0)
 #define FUSE_POSIX_LOCKS	(1 << 1)
@@ -275,6 +286,9 @@ struct fuse_file_lock {
 #define FUSE_HANDLE_KILLPRIV	(1 << 19)
 #define FUSE_POSIX_ACL		(1 << 20)
 #define FUSE_ABORT_ERROR	(1 << 21)
+#define FUSE_MAX_PAGES		(1 << 22)
+#define FUSE_CACHE_SYMLINKS	(1 << 23)
+#define FUSE_NO_OPENDIR_SUPPORT (1 << 24)

 /**
 * CUSE INIT request/reply flags
@@ -614,7 +628,9 @@ struct fuse_init_out {
 	uint16_t	congestion_threshold;
 	uint32_t	max_write;
 	uint32_t	time_gran;
-	uint32_t	unused[9];
+	uint16_t	max_pages;
+	uint16_t	padding;
+	uint32_t	unused[8];
 };

 #define CUSE_INIT_INFO_MAX 4096
@@ -1021,6 +1021,52 @@ struct fuse_lowlevel_ops {
 	 */
 	void (*fallocate) (fuse_req_t req, fuse_ino_t ino, int mode,
 		       off_t offset, off_t length, struct fuse_file_info *fi);
+
+  /**
+   * Copy a range of data from one file to another
+   *
+   * Performs an optimized copy between two file descriptors without
+   * the
+   * additional cost of transferring data through the FUSE kernel
+   * module
+   * to user space (glibc) and then back into the FUSE filesystem
+   * again.
+   *
+   * In case this method is not implemented, glibc falls back to
+   * reading
+   * data from the source and writing to the destination. Effectively
+   * doing an inefficient copy of the data.
+   *
+   * If this request is answered with an error code of ENOSYS, this is
+   * treated as a permanent failure with error code EOPNOTSUPP,
+   * i.e. all
+   * future copy_file_range() requests will fail with EOPNOTSUPP
+   * without
+   * being send to the filesystem process.
+   *
+   * Valid replies:
+   *   fuse_reply_write
+   *   fuse_reply_err
+   *
+   * @param req request handle
+   * @param ino_in the inode number of the source file
+   * @param off_in starting point from were the data should be read
+   * @param fi_in file information of the source file
+   * @param ino_out the inode number of the destination file
+   * @param off_out starting point where the data should be written
+   * @param fi_out file information of the destination file
+   * @param len maximum size of the data to copy
+   * @param flags passed along with the copy_file_range() syscall
+   */
+  void (*copy_file_range)(fuse_req_t             req,
+                          fuse_ino_t             ino_in,
+                          off_t                  off_in,
+                          struct fuse_file_info *fi_in,
+                          fuse_ino_t             ino_out,
+                          off_t                  off_out,
+                          struct fuse_file_info *fi_out,
+                          size_t                 len,
+                          int                    flags);
 };

 /**
@@ -19,7 +19,7 @@
 size_t fuse_buf_size(const struct fuse_bufvec *bufv)
 {
 	size_t i;
-	size_t size = 0;
+        size_t size = 0;

 	for (i = 0; i < bufv->count; i++) {
 		if (bufv->buf[i].size == SIZE_MAX)
@@ -229,8 +229,8 @@ static ssize_t fuse_buf_copy_one(const struct fuse_buf *dst, size_t dst_off,
 	int dst_is_fd = dst->flags & FUSE_BUF_IS_FD;

 	if (!src_is_fd && !dst_is_fd) {
-		void *dstmem = dst->mem + dst_off;
-		void *srcmem = src->mem + src_off;
+		char *dstmem = dst->mem + dst_off;
+		char *srcmem = src->mem + src_off;

 		if (dstmem != srcmem) {
 			if (dstmem + len <= srcmem || srcmem + len <= dstmem)
@@ -59,19 +59,14 @@ struct fuse_config {
 	double entry_timeout;
 	double negative_timeout;
 	double attr_timeout;
-	double ac_attr_timeout;
-	int ac_attr_timeout_set;
 	int remember;
 	int nopath;
 	int debug;
-        int hard_remove; /* not used */
 	int use_ino;
 	int readdir_ino;
 	int set_mode;
 	int set_uid;
 	int set_gid;
-	int kernel_cache;
-	int auto_cache;
 	int intr;
 	int intr_signal;
 	int help;
@@ -85,11 +80,6 @@ struct fuse_fs {
 	int debug;
 };

-struct fusemod_so {
-	void *handle;
-	int ctr;
-};
-
 struct lock_queue_element {
 	struct lock_queue_element *next;
 	pthread_cond_t cond;
@@ -162,25 +152,24 @@ struct lock {
 	struct lock *next;
 };

-struct node {
-	struct node *name_next;
-	struct node *id_next;
-	fuse_ino_t nodeid;
-	unsigned int generation;
-	int refctr;
-	struct node *parent;
-	char *name;
-	uint64_t nlookup;
-	int open_count;
-	struct timespec stat_updated;
-	struct timespec mtime;
-	off_t size;
-	struct lock *locks;
-        uint64_t hidden_fh;
-        char is_hidden;
-        char cache_valid;
-	int treelock;
-	char inline_name[32];
+struct node
+{
+  struct node *name_next;
+  struct node *id_next;
+  fuse_ino_t nodeid;
+  unsigned int generation;
+  int refctr;
+  struct node *parent;
+  char *name;
+  uint64_t nlookup;
+  int open_count;
+  struct lock *locks;
+  uint64_t hidden_fh;
+  char is_hidden;
+  int treelock;
+  struct stat stat_cache;
+  char stat_cache_valid;
+  char inline_name[32];
 };

 #define TREELOCK_WRITE -1
@@ -438,11 +427,18 @@ static void set_forget_time(struct fuse *f, struct node *node)
 	curr_time(&lnode->forget_time);
 }

-static void free_node(struct fuse *f, struct node *node)
+static
+void
+free_node(struct fuse *f_,
+          struct node *node_)
 {
-	if (node->name != node->inline_name)
-		free(node->name);
-	free_node_mem(f, node);
+  if(node_->name != node_->inline_name)
+    free(node_->name);
+
+  if(node_->is_hidden)
+    fuse_fs_free_hide(f_->fs,node_->hidden_fh);
+
+  free_node_mem(f_,node_);
 }

 static void node_table_reduce(struct node_table *t)
@@ -1529,13 +1525,12 @@ fuse_fs_rename(struct fuse_fs *fs,
 int
 fuse_fs_prepare_hide(struct fuse_fs *fs_,
                     const char     *path_,
-                     uint64_t       *fh_,
-                     int             type_)
+                     uint64_t       *fh_)
 {
  fuse_get_context()->private_data = fs_->user_data;

  if(fs_->op.prepare_hide)
-    return fs_->op.prepare_hide(path_,fh_,type_);
+    return fs_->op.prepare_hide(path_,fh_);

  return -ENOSYS;
 }
@@ -2301,19 +2296,37 @@ int fuse_fs_fallocate(struct fuse_fs *fs, const char *path, int mode,
 		return -ENOSYS;
 }

-static
-int
-node_open_and_visible(const struct node *node_)
+ssize_t
+fuse_fs_copy_file_range(struct fuse_fs        *fs_,
+                        const char            *path_in_,
+                        struct fuse_file_info *ffi_in_,
+                        off_t                  off_in_,
+                        const char            *path_out_,
+                        struct fuse_file_info *ffi_out_,
+                        off_t                  off_out_,
+                        size_t                 len_,
+                        int                    flags_)
 {
-  return ((node_ != NULL)         &&
-          (node_->open_count > 0) &&
-          (node_->is_hidden == 0));
+  fuse_get_context()->private_data = fs_->user_data;
+
+  if(fs_->op.copy_file_range == NULL)
+    return -ENOSYS;
+
+  return fs_->op.copy_file_range(path_in_,
+                                 ffi_in_,
+                                 off_in_,
+                                 path_out_,
+                                 ffi_out_,
+                                 off_out_,
+                                 len_,
+                                 flags_);
 }

-static int mtime_eq(const struct stat *stbuf, const struct timespec *ts)
+int
+node_open(const struct node *node_)
 {
-	return stbuf->st_mtime == ts->tv_sec &&
-		ST_MTIM_NSEC(stbuf) == ts->tv_nsec;
+  return ((node_ != NULL) &&
+          (node_->open_count > 0));
 }

 #ifndef CLOCK_MONOTONIC
@@ -2334,15 +2347,21 @@ static void curr_time(struct timespec *now)
 	}
 }

-static void update_stat(struct node *node, const struct stat *stbuf)
+static
+void
+update_stat(struct node       *node_,
+            const struct stat *stnew_)
 {
-	if (node->cache_valid && (!mtime_eq(stbuf, &node->mtime) ||
-				  stbuf->st_size != node->size))
-		node->cache_valid = 0;
-	node->mtime.tv_sec = stbuf->st_mtime;
-	node->mtime.tv_nsec = ST_MTIM_NSEC(stbuf);
-	node->size = stbuf->st_size;
-	curr_time(&node->stat_updated);
+  struct stat *stold;
+
+  stold = &node_->stat_cache;
+  if((node_->stat_cache_valid) &&
+     ((stold->st_mtim.tv_sec  != stnew_->st_mtim.tv_sec)  ||
+      (stold->st_mtim.tv_nsec != stnew_->st_mtim.tv_nsec) ||
+      (stold->st_size         != stnew_->st_size)))
+    node_->stat_cache_valid = 0;
+
+  *stold = *stnew_;
 }

 static int lookup_path(struct fuse *f, fuse_ino_t nodeid,
@@ -2367,11 +2386,9 @@ static int lookup_path(struct fuse *f, fuse_ino_t nodeid,
 			e->generation = node->generation;
 			e->entry_timeout = f->conf.entry_timeout;
 			e->attr_timeout = f->conf.attr_timeout;
-			if (f->conf.auto_cache) {
-				pthread_mutex_lock(&f->lock);
-				update_stat(node, &e->attr);
-				pthread_mutex_unlock(&f->lock);
-			}
+                        pthread_mutex_lock(&f->lock);
+                        update_stat(node, &e->attr);
+                        pthread_mutex_unlock(&f->lock);
 			set_stat(f, e->ino, &e->attr);
 			if (f->conf.debug)
 				fprintf(stderr, "   NODEID: %lu\n",
@@ -2621,6 +2638,7 @@ static void fuse_lib_getattr(fuse_req_t req, fuse_ino_t ino,

 	memset(&buf, 0, sizeof(buf));

+        path = NULL;
        err = (((fi == NULL) || (f->fs->op.fgetattr == NULL)) ?
               get_path(f,ino,&path) :
               get_path_nullok(f,ino,&path));
@@ -2640,8 +2658,7 @@ static void fuse_lib_getattr(fuse_req_t req, fuse_ino_t ino,
 	if (!err) {
 		pthread_mutex_lock(&f->lock);
                node = get_node(f, ino);
-		if (f->conf.auto_cache)
-			update_stat(node, &buf);
+                update_stat(node, &buf);
 		pthread_mutex_unlock(&f->lock);
 		set_stat(f, ino, &buf);
 		fuse_reply_attr(req, &buf, f->conf.attr_timeout);
@@ -2695,9 +2712,10 @@ static void fuse_lib_setattr(fuse_req_t req, fuse_ino_t ino, struct stat *attr,

 	memset(&buf, 0, sizeof(buf));

+        path = NULL;
        err = ((fi == NULL) ?
               get_path(f,ino,&path) :
-               get_path_nullok(f, ino, &path));
+               get_path_nullok(f,ino,&path));

 	if (!err) {
 		struct fuse_intr_data d;
@@ -2776,15 +2794,13 @@ static void fuse_lib_setattr(fuse_req_t req, fuse_ino_t ino, struct stat *attr,
 	}

 	if (!err) {
-		if (f->conf.auto_cache) {
-			pthread_mutex_lock(&f->lock);
-			update_stat(get_node(f, ino), &buf);
-			pthread_mutex_unlock(&f->lock);
-		}
-		set_stat(f, ino, &buf);
-		fuse_reply_attr(req, &buf, f->conf.attr_timeout);
+          pthread_mutex_lock(&f->lock);
+          update_stat(get_node(f, ino), &buf);
+          pthread_mutex_unlock(&f->lock);
+          set_stat(f, ino, &buf);
+          fuse_reply_attr(req, &buf, f->conf.attr_timeout);
 	} else {
-		reply_err(req, err);
+          reply_err(req, err);
        }
 }

@@ -2906,15 +2922,18 @@ fuse_lib_unlink(fuse_req_t  req,
  if(!err)
    {
      fuse_prepare_interrupt(f,req,&d);
-      if(node_open_and_visible(wnode))
+
+      pthread_mutex_lock(&f->lock);
+      if(node_open(wnode))
        {
-          err = fuse_fs_prepare_hide(f->fs,path,&wnode->hidden_fh,0);
+          err = fuse_fs_prepare_hide(f->fs,path,&wnode->hidden_fh);
          if(!err)
            wnode->is_hidden = 1;
        }
+      pthread_mutex_unlock(&f->lock);

      err = fuse_fs_unlink(f->fs,path);
-      if(!err && !wnode->is_hidden)
+      if(!err)
        remove_node(f,parent,name);

      fuse_finish_interrupt(f,req,&d);
@@ -2990,12 +3009,15 @@ fuse_lib_rename(fuse_req_t  req,
  if(!err)
    {
      fuse_prepare_interrupt(f,req,&d);
-      if(node_open_and_visible(wnode2))
+
+      pthread_mutex_lock(&f->lock);
+      if(node_open(wnode2))
        {
-          err = fuse_fs_prepare_hide(f->fs,newpath,&wnode2->hidden_fh,1);
+          err = fuse_fs_prepare_hide(f->fs,newpath,&wnode2->hidden_fh);
          if(!err)
            wnode2->is_hidden = 1;
        }
+      pthread_mutex_unlock(&f->lock);

      err = fuse_fs_rename(f->fs,oldpath,newpath);
      if(!err)
@@ -3065,53 +3087,62 @@ static void fuse_do_release(struct fuse *f, fuse_ino_t ino, const char *path,
          fuse_fs_free_hide(f->fs,fh);
 }

-static void fuse_lib_create(fuse_req_t req, fuse_ino_t parent,
-			    const char *name, mode_t mode,
-			    struct fuse_file_info *fi)
+static
+void
+fuse_lib_create(fuse_req_t             req,
+                fuse_ino_t             parent,
+                const char            *name,
+                mode_t                 mode,
+                struct fuse_file_info *fi)
 {
-	struct fuse *f = req_fuse_prepare(req);
-	struct fuse_intr_data d;
-	struct fuse_entry_param e;
-	char *path;
-	int err;
+  int err;
+  char *path;
+  struct fuse *f;
+  struct fuse_intr_data d;
+  struct fuse_entry_param e;

-	err = get_path_name(f, parent, name, &path);
-	if (!err) {
-		fuse_prepare_interrupt(f, req, &d);
-		err = fuse_fs_create(f->fs, path, mode, fi);
-		if (!err) {
-			err = lookup_path(f, parent, name, path, &e, fi);
-			if (err)
-				fuse_fs_release(f->fs, path, fi);
-			else if (!S_ISREG(e.attr.st_mode)) {
-				err = -EIO;
-				fuse_fs_release(f->fs, path, fi);
-				forget_node(f, e.ino, 1);
-			} else {
-				if (f->conf.kernel_cache)
-					fi->keep_cache = 1;
+  f = req_fuse_prepare(req);
+  err = get_path_name(f, parent, name, &path);
+  if(!err)
+    {
+      fuse_prepare_interrupt(f, req, &d);
+      err = fuse_fs_create(f->fs, path, mode, fi);
+      if(!err)
+        {
+          err = lookup_path(f, parent, name, path, &e, fi);
+          if(err)
+            {
+              fuse_fs_release(f->fs, path, fi);
+            }
+          else if(!S_ISREG(e.attr.st_mode))
+            {
+              err = -EIO;
+              fuse_fs_release(f->fs, path, fi);
+              forget_node(f, e.ino, 1);
+            }
+        }
+      fuse_finish_interrupt(f, req, &d);
+    }

-			}
-		}
-		fuse_finish_interrupt(f, req, &d);
-	}
-	if (!err) {
-		pthread_mutex_lock(&f->lock);
-                struct node *n = get_node(f,e.ino);
-                n->open_count++;
-		pthread_mutex_unlock(&f->lock);
+  if(!err)
+    {
+      pthread_mutex_lock(&f->lock);
+      get_node(f,e.ino)->open_count++;
+      pthread_mutex_unlock(&f->lock);

-		if (fuse_reply_create(req, &e, fi) == -ENOENT) {
-			/* The open syscall was interrupted, so it
-			   must be cancelled */
-			fuse_do_release(f, e.ino, path, fi);
-			forget_node(f, e.ino, 1);
-		}
-	} else {
-		reply_err(req, err);
-	}
+      if (fuse_reply_create(req, &e, fi) == -ENOENT) {
+        /* The open syscall was interrupted, so it
+           must be cancelled */
+        fuse_do_release(f, e.ino, path, fi);
+        forget_node(f, e.ino, 1);
+      }
+    }
+  else
+    {
+      reply_err(req, err);
+    }

-	free_path(f, parent, path);
+  free_path(f, parent, path);
 }

 static double diff_timespec(const struct timespec *t1,
@@ -3121,72 +3152,81 @@ static double diff_timespec(const struct timespec *t1,
 		((double) t1->tv_nsec - (double) t2->tv_nsec) / 1000000000.0;
 }

-static void open_auto_cache(struct fuse *f, fuse_ino_t ino, const char *path,
-			    struct fuse_file_info *fi)
+static
+void
+open_auto_cache(struct fuse           *f,
+                fuse_ino_t             ino,
+                const char            *path,
+                struct fuse_file_info *fi)
 {
-	struct node *node;
+  struct node *node;

-	pthread_mutex_lock(&f->lock);
-	node = get_node(f, ino);
-	if (node->cache_valid) {
-		struct timespec now;
+  pthread_mutex_lock(&f->lock);

-		curr_time(&now);
-		if (diff_timespec(&now, &node->stat_updated) >
-		    f->conf.ac_attr_timeout) {
-			struct stat stbuf;
-			int err;
-			pthread_mutex_unlock(&f->lock);
-			err = fuse_fs_fgetattr(f->fs, path, &stbuf, fi);
-			pthread_mutex_lock(&f->lock);
-			if (!err)
-				update_stat(node, &stbuf);
-			else
-				node->cache_valid = 0;
-		}
-	}
-	if (node->cache_valid)
-		fi->keep_cache = 1;
+  node = get_node(f,ino);
+  if(node->stat_cache_valid)
+    {
+      int err;
+      struct stat stbuf;

-	node->cache_valid = 1;
-	pthread_mutex_unlock(&f->lock);
+      pthread_mutex_unlock(&f->lock);
+      err = fuse_fs_fgetattr(f->fs,path,&stbuf,fi);
+      pthread_mutex_lock(&f->lock);
+
+      if(!err)
+        update_stat(node,&stbuf);
+      else
+        node->stat_cache_valid = 0;
+    }
+
+  if(node->stat_cache_valid)
+    fi->keep_cache = 1;
+
+  node->stat_cache_valid = 1;
+
+  pthread_mutex_unlock(&f->lock);
 }

-static void fuse_lib_open(fuse_req_t req, fuse_ino_t ino,
-			  struct fuse_file_info *fi)
+static
+void
+fuse_lib_open(fuse_req_t             req,
+              fuse_ino_t             ino,
+              struct fuse_file_info *fi)
 {
-	struct fuse *f = req_fuse_prepare(req);
-	struct fuse_intr_data d;
-	char *path;
-	int err;
+  int err;
+  char *path;
+  struct fuse *f;
+  struct fuse_intr_data d;

-	err = get_path(f, ino, &path);
-	if (!err) {
-		fuse_prepare_interrupt(f, req, &d);
-		err = fuse_fs_open(f->fs, path, fi);
-		if (!err) {
-			if (f->conf.kernel_cache)
-				fi->keep_cache = 1;
+  f = req_fuse_prepare(req);
+  err = get_path(f, ino, &path);
+  if(!err)
+    {
+      fuse_prepare_interrupt(f, req, &d);
+      err = fuse_fs_open(f->fs, path, fi);
+      if(!err)
+        {
+          if (fi && fi->auto_cache)
+            open_auto_cache(f, ino, path, fi);
+        }
+      fuse_finish_interrupt(f, req, &d);
+    }

-			if (f->conf.auto_cache)
-				open_auto_cache(f, ino, path, fi);
-		}
-		fuse_finish_interrupt(f, req, &d);
-	}
-	if (!err) {
-		pthread_mutex_lock(&f->lock);
-                struct node *n = get_node(f,ino);
-                n->open_count++;
-		pthread_mutex_unlock(&f->lock);
-		if (fuse_reply_open(req, fi) == -ENOENT) {
-			/* The open syscall was interrupted, so it
-			   must be cancelled */
-			fuse_do_release(f, ino, path, fi);
-		}
-	} else
-		reply_err(req, err);
+  if(!err)
+    {
+      pthread_mutex_lock(&f->lock);
+      get_node(f,ino)->open_count++;
+      pthread_mutex_unlock(&f->lock);
+      /* The open syscall was interrupted, so it must be cancelled */
+      if(fuse_reply_open(req, fi) == -ENOENT)
+        fuse_do_release(f, ino, path, fi);
+    }
+  else
+    {
+      reply_err(req, err);
+    }

-	free_path(f, ino, path);
+  free_path(f, ino, path);
 }

 static void fuse_lib_read(fuse_req_t req, fuse_ino_t ino, size_t size,
@@ -3264,7 +3304,6 @@ static struct fuse_dh *get_dirhandle(const struct fuse_file_info *llfi,
 	struct fuse_dh *dh = (struct fuse_dh *) (uintptr_t) llfi->fh;
 	memset(fi, 0, sizeof(struct fuse_file_info));
 	fi->fh = dh->fh;
-	fi->fh_old = dh->fh;
 	return dh;
 }

@@ -3283,6 +3322,7 @@ static void fuse_lib_opendir(fuse_req_t req, fuse_ino_t ino,
 		reply_err(req, -ENOMEM);
 		return;
 	}
+
 	memset(dh, 0, sizeof(struct fuse_dh));
 	dh->fuse = f;
 	dh->contents = NULL;
@@ -3302,7 +3342,10 @@ static void fuse_lib_opendir(fuse_req_t req, fuse_ino_t ino,
 		err = fuse_fs_opendir(f->fs, path, &fi);
 		fuse_finish_interrupt(f, req, &d);
 		dh->fh = fi.fh;
+                llfi->keep_cache    = fi.keep_cache;
+                llfi->cache_readdir = fi.cache_readdir;
 	}
+
 	if (!err) {
 		if (fuse_reply_open(req, llfi) == -ENOENT) {
 			/* The opendir syscall was interrupted, so it
@@ -3659,6 +3702,61 @@ static void fuse_lib_removexattr(fuse_req_t req, fuse_ino_t ino,
 	reply_err(req, err);
 }

+static
+void
+fuse_lib_copy_file_range(fuse_req_t             req_,
+                         fuse_ino_t             nodeid_in_,
+                         off_t                  off_in_,
+                         struct fuse_file_info *ffi_in_,
+                         fuse_ino_t             nodeid_out_,
+                         off_t                  off_out_,
+                         struct fuse_file_info *ffi_out_,
+                         size_t                 len_,
+                         int                    flags_)
+{
+  int err;
+  ssize_t rv;
+  char *path_in;
+  char *path_out;
+  struct fuse *f;
+  struct fuse_intr_data d;
+
+  f = req_fuse_prepare(req_);
+
+  err = get_path_nullok(f,nodeid_in_,&path_in);
+  if(err)
+    return reply_err(req_,err);
+
+  err = get_path_nullok(f,nodeid_out_,&path_out);
+  if(err)
+    {
+      free_path(f,nodeid_in_,path_in);
+      return reply_err(req_,err);
+    }
+
+  fuse_prepare_interrupt(f,req_,&d);
+
+  rv = fuse_fs_copy_file_range(f->fs,
+                               path_in,
+                               ffi_in_,
+                               off_in_,
+                               path_out,
+                               ffi_out_,
+                               off_out_,
+                               len_,
+                               flags_);
+
+  fuse_finish_interrupt(f,req_,&d);
+
+  if(rv >= 0)
+    fuse_reply_write(req_,rv);
+  else
+    reply_err(req_,rv);
+
+  free_path(f,nodeid_in_,path_in);
+  free_path(f,nodeid_out_,path_out);
+}
+
 static struct lock *locks_conflict(struct node *node, const struct lock *lock)
 {
 	struct lock *l;
@@ -4097,45 +4195,46 @@ int fuse_clean_cache(struct fuse *f)
 }

 static struct fuse_lowlevel_ops fuse_path_ops = {
-	.init = fuse_lib_init,
-	.destroy = fuse_lib_destroy,
-	.lookup = fuse_lib_lookup,
-	.forget = fuse_lib_forget,
-	.forget_multi = fuse_lib_forget_multi,
-	.getattr = fuse_lib_getattr,
-	.setattr = fuse_lib_setattr,
-	.access = fuse_lib_access,
-	.readlink = fuse_lib_readlink,
-	.mknod = fuse_lib_mknod,
-	.mkdir = fuse_lib_mkdir,
-	.unlink = fuse_lib_unlink,
-	.rmdir = fuse_lib_rmdir,
-	.symlink = fuse_lib_symlink,
-	.rename = fuse_lib_rename,
-	.link = fuse_lib_link,
-	.create = fuse_lib_create,
-	.open = fuse_lib_open,
-	.read = fuse_lib_read,
-	.write_buf = fuse_lib_write_buf,
-	.flush = fuse_lib_flush,
-	.release = fuse_lib_release,
-	.fsync = fuse_lib_fsync,
-	.opendir = fuse_lib_opendir,
-	.readdir = fuse_lib_readdir,
-	.releasedir = fuse_lib_releasedir,
-	.fsyncdir = fuse_lib_fsyncdir,
-	.statfs = fuse_lib_statfs,
-	.setxattr = fuse_lib_setxattr,
-	.getxattr = fuse_lib_getxattr,
-	.listxattr = fuse_lib_listxattr,
-	.removexattr = fuse_lib_removexattr,
-	.getlk = fuse_lib_getlk,
-	.setlk = fuse_lib_setlk,
-	.flock = fuse_lib_flock,
-	.bmap = fuse_lib_bmap,
-	.ioctl = fuse_lib_ioctl,
-	.poll = fuse_lib_poll,
-	.fallocate = fuse_lib_fallocate,
+	.init            = fuse_lib_init,
+	.destroy         = fuse_lib_destroy,
+	.lookup          = fuse_lib_lookup,
+	.forget          = fuse_lib_forget,
+	.forget_multi    = fuse_lib_forget_multi,
+	.getattr         = fuse_lib_getattr,
+	.setattr         = fuse_lib_setattr,
+	.access          = fuse_lib_access,
+	.readlink        = fuse_lib_readlink,
+	.mknod           = fuse_lib_mknod,
+	.mkdir           = fuse_lib_mkdir,
+	.unlink          = fuse_lib_unlink,
+	.rmdir           = fuse_lib_rmdir,
+	.symlink         = fuse_lib_symlink,
+	.rename          = fuse_lib_rename,
+	.link            = fuse_lib_link,
+	.create          = fuse_lib_create,
+	.open            = fuse_lib_open,
+	.read            = fuse_lib_read,
+	.write_buf       = fuse_lib_write_buf,
+	.flush           = fuse_lib_flush,
+	.release         = fuse_lib_release,
+	.fsync           = fuse_lib_fsync,
+	.opendir         = fuse_lib_opendir,
+	.readdir         = fuse_lib_readdir,
+	.releasedir      = fuse_lib_releasedir,
+	.fsyncdir        = fuse_lib_fsyncdir,
+	.statfs          = fuse_lib_statfs,
+	.setxattr        = fuse_lib_setxattr,
+	.getxattr        = fuse_lib_getxattr,
+	.listxattr       = fuse_lib_listxattr,
+	.removexattr     = fuse_lib_removexattr,
+	.getlk           = fuse_lib_getlk,
+	.setlk           = fuse_lib_setlk,
+	.flock           = fuse_lib_flock,
+	.bmap            = fuse_lib_bmap,
+	.ioctl           = fuse_lib_ioctl,
+	.poll            = fuse_lib_poll,
+	.fallocate       = fuse_lib_fallocate,
+        .copy_file_range = fuse_lib_copy_file_range,
 };

 int fuse_notify_poll(struct fuse_pollhandle *ph)
@@ -4332,12 +4431,8 @@ static const struct fuse_opt fuse_lib_opts[] = {
 	FUSE_OPT_KEY("-d",		      FUSE_OPT_KEY_KEEP),
 	FUSE_LIB_OPT("debug",		      debug, 1),
 	FUSE_LIB_OPT("-d",		      debug, 1),
-	FUSE_LIB_OPT("hard_remove",	      hard_remove, 1),
 	FUSE_LIB_OPT("use_ino",		      use_ino, 1),
 	FUSE_LIB_OPT("readdir_ino",	      readdir_ino, 1),
-	FUSE_LIB_OPT("kernel_cache",	      kernel_cache, 1),
-	FUSE_LIB_OPT("auto_cache",	      auto_cache, 1),
-	FUSE_LIB_OPT("noauto_cache",	      auto_cache, 0),
 	FUSE_LIB_OPT("umask=",		      set_mode, 1),
 	FUSE_LIB_OPT("umask=%o",	      umask, 0),
 	FUSE_LIB_OPT("uid=",		      set_uid, 1),
@@ -4346,8 +4441,6 @@ static const struct fuse_opt fuse_lib_opts[] = {
 	FUSE_LIB_OPT("gid=%d",		      gid, 0),
 	FUSE_LIB_OPT("entry_timeout=%lf",     entry_timeout, 0),
 	FUSE_LIB_OPT("attr_timeout=%lf",      attr_timeout, 0),
-	FUSE_LIB_OPT("ac_attr_timeout=%lf",   ac_attr_timeout, 0),
-	FUSE_LIB_OPT("ac_attr_timeout=",      ac_attr_timeout_set, 1),
 	FUSE_LIB_OPT("negative_timeout=%lf",  negative_timeout, 0),
 	FUSE_LIB_OPT("noforget",              remember, -1),
 	FUSE_LIB_OPT("remember=%u",           remember, 0),
@@ -4363,15 +4456,12 @@ static void fuse_lib_help(void)
 	fprintf(stderr,
 "    -o use_ino             let filesystem set inode numbers\n"
 "    -o readdir_ino         try to fill in d_ino in readdir\n"
-"    -o kernel_cache        cache files in kernel\n"
-"    -o [no]auto_cache      enable caching based on modification times (off)\n"
 "    -o umask=M             set file permissions (octal)\n"
 "    -o uid=N               set file owner\n"
 "    -o gid=N               set file group\n"
 "    -o entry_timeout=T     cache timeout for names (1.0s)\n"
 "    -o negative_timeout=T  cache timeout for deleted names (0.0s)\n"
 "    -o attr_timeout=T      cache timeout for attributes (1.0s)\n"
-"    -o ac_attr_timeout=T   auto cache timeout for attributes (attr_timeout)\n"
 "    -o noforget            never forget cached inodes\n"
 "    -o remember=T          remember cached inodes for T seconds (0s)\n"
 "    -o nopath              don't supply path if not necessary\n"
@@ -4552,9 +4642,6 @@ struct fuse *fuse_new_common(struct fuse_chan *ch, struct fuse_args *args,
 			   fuse_lib_opt_proc) == -1)
 		goto out_free_fs;

-	if (!f->conf.ac_attr_timeout_set)
-		f->conf.ac_attr_timeout = f->conf.attr_timeout;
-
 #if defined(__FreeBSD__) || defined(__NetBSD__)
 	/*
 	 * In FreeBSD, we always use these settings as inode numbers
@@ -4690,9 +4777,11 @@ void fuse_destroy(struct fuse *f)
 	fuse_delete_context_key();
 }

-static struct fuse *fuse_new_common_compat25(int fd, struct fuse_args *args,
-					     const struct fuse_operations *op,
-					     size_t op_size, int compat)
+static
+struct fuse *
+fuse_new_common_compat25(int fd, struct fuse_args *args,
+                         const struct fuse_operations *op,
+                         size_t op_size, int compat)
 {
 	struct fuse *f = NULL;
 	struct fuse_chan *ch = fuse_kern_chan_new(fd);
@@ -4720,6 +4809,7 @@ static struct fuse *fuse_new_common_compat(int fd, const char *opts,
 		fuse_opt_free_args(&args);
 		return NULL;
 	}
+
 	f = fuse_new_common_compat25(fd, &args, op, op_size, compat);
 	fuse_opt_free_args(&args);

@@ -83,16 +83,21 @@ static void fuse_kern_chan_destroy(struct fuse_chan *ch)
 		close(fd);
 }

-#define MIN_BUFSIZE 0x21000
-
-struct fuse_chan *fuse_kern_chan_new(int fd)
+struct fuse_chan *
+fuse_kern_chan_new(int fd_)
 {
-	struct fuse_chan_ops op = {
-		.receive = fuse_kern_chan_receive,
-		.send = fuse_kern_chan_send,
-		.destroy = fuse_kern_chan_destroy,
-	};
-	size_t bufsize = getpagesize() + 0x1000;
-	bufsize = bufsize < MIN_BUFSIZE ? MIN_BUFSIZE : bufsize;
-	return fuse_chan_new(&op, fd, bufsize, NULL);
+  long pagesize;
+  size_t bufsize;
+  struct fuse_chan_ops op =
+    {
+      .receive = fuse_kern_chan_receive,
+      .send    = fuse_kern_chan_send,
+      .destroy = fuse_kern_chan_destroy,
+    };
+
+  pagesize = sysconf(_SC_PAGESIZE);
+
+  bufsize = ((FUSE_MAX_MAX_PAGES * pagesize) + 0x1000);
+
+  return fuse_chan_new(&op, fd_, bufsize, NULL);
 }
@@ -352,6 +352,8 @@ static void fill_open(struct fuse_open_out *arg,
 		arg->open_flags |= FOPEN_KEEP_CACHE;
 	if (f->nonseekable)
 		arg->open_flags |= FOPEN_NONSEEKABLE;
+        if (f->cache_readdir)
+                arg->open_flags |= FOPEN_CACHE_DIR;
 }

 int fuse_reply_entry(fuse_req_t req, const struct fuse_entry_param *e)
@@ -1041,7 +1043,6 @@ static void do_getattr(fuse_req_t req, fuse_ino_t nodeid, const void *inarg)
 		if (arg->getattr_flags & FUSE_GETATTR_FH) {
 			memset(&fi, 0, sizeof(fi));
 			fi.fh = arg->fh;
-			fi.fh_old = fi.fh;
 			fip = &fi;
 		}
 	}
@@ -1067,7 +1068,6 @@ static void do_setattr(fuse_req_t req, fuse_ino_t nodeid, const void *inarg)
 			memset(&fi_store, 0, sizeof(fi_store));
 			fi = &fi_store;
 			fi->fh = arg->fh;
-			fi->fh_old = fi->fh;
 		}
 		arg->valid &=
 			FUSE_SET_ATTR_MODE	|
@@ -1230,7 +1230,6 @@ static void do_read(fuse_req_t req, fuse_ino_t nodeid, const void *inarg)

 		memset(&fi, 0, sizeof(fi));
 		fi.fh = arg->fh;
-		fi.fh_old = fi.fh;
 		if (req->f->conn.proto_minor >= 9) {
 			fi.lock_owner = arg->lock_owner;
 			fi.flags = arg->flags;
@@ -1248,7 +1247,6 @@ static void do_write(fuse_req_t req, fuse_ino_t nodeid, const void *inarg)

 	memset(&fi, 0, sizeof(fi));
 	fi.fh = arg->fh;
-	fi.fh_old = fi.fh;
 	fi.writepage = arg->write_flags & 1;

 	if (req->f->conn.proto_minor < 9) {
@@ -1279,7 +1277,6 @@ static void do_write_buf(fuse_req_t req, fuse_ino_t nodeid, const void *inarg,

 	memset(&fi, 0, sizeof(fi));
 	fi.fh = arg->fh;
-	fi.fh_old = fi.fh;
 	fi.writepage = arg->write_flags & 1;

 	if (req->f->conn.proto_minor < 9) {
@@ -1318,7 +1315,6 @@ static void do_flush(fuse_req_t req, fuse_ino_t nodeid, const void *inarg)

 	memset(&fi, 0, sizeof(fi));
 	fi.fh = arg->fh;
-	fi.fh_old = fi.fh;
 	fi.flush = 1;
 	if (req->f->conn.proto_minor >= 7)
 		fi.lock_owner = arg->lock_owner;
@@ -1337,7 +1333,6 @@ static void do_release(fuse_req_t req, fuse_ino_t nodeid, const void *inarg)
 	memset(&fi, 0, sizeof(fi));
 	fi.flags = arg->flags;
 	fi.fh = arg->fh;
-	fi.fh_old = fi.fh;
 	if (req->f->conn.proto_minor >= 8) {
 		fi.flush = (arg->release_flags & FUSE_RELEASE_FLUSH) ? 1 : 0;
 		fi.lock_owner = arg->lock_owner;
@@ -1360,7 +1355,6 @@ static void do_fsync(fuse_req_t req, fuse_ino_t nodeid, const void *inarg)

 	memset(&fi, 0, sizeof(fi));
 	fi.fh = arg->fh;
-	fi.fh_old = fi.fh;

 	if (req->f->op.fsync)
 		req->f->op.fsync(req, nodeid, arg->fsync_flags & 1, &fi);
@@ -1389,7 +1383,6 @@ static void do_readdir(fuse_req_t req, fuse_ino_t nodeid, const void *inarg)

 	memset(&fi, 0, sizeof(fi));
 	fi.fh = arg->fh;
-	fi.fh_old = fi.fh;

 	if (req->f->op.readdir)
 		req->f->op.readdir(req, nodeid, arg->size, arg->offset, &fi);
@@ -1405,7 +1398,6 @@ static void do_releasedir(fuse_req_t req, fuse_ino_t nodeid, const void *inarg)
 	memset(&fi, 0, sizeof(fi));
 	fi.flags = arg->flags;
 	fi.fh = arg->fh;
-	fi.fh_old = fi.fh;

 	if (req->f->op.releasedir)
 		req->f->op.releasedir(req, nodeid, &fi);
@@ -1420,7 +1412,6 @@ static void do_fsyncdir(fuse_req_t req, fuse_ino_t nodeid, const void *inarg)

 	memset(&fi, 0, sizeof(fi));
 	fi.fh = arg->fh;
-	fi.fh_old = fi.fh;

 	if (req->f->op.fsyncdir)
 		req->f->op.fsyncdir(req, nodeid, arg->fsync_flags & 1, &fi);
@@ -1675,7 +1666,6 @@ static void do_ioctl(fuse_req_t req, fuse_ino_t nodeid, const void *inarg)

 	memset(&fi, 0, sizeof(fi));
 	fi.fh = arg->fh;
-	fi.fh_old = fi.fh;

 	if (sizeof(void *) == 4 && req->f->conn.proto_minor >= 16 &&
 	    !(flags & FUSE_IOCTL_32BIT)) {
@@ -1702,7 +1692,6 @@ static void do_poll(fuse_req_t req, fuse_ino_t nodeid, const void *inarg)

 	memset(&fi, 0, sizeof(fi));
 	fi.fh = arg->fh;
-	fi.fh_old = fi.fh;

 	if (req->f->op.poll) {
 		struct fuse_pollhandle *ph = NULL;
@@ -1760,8 +1749,10 @@ static void do_init(fuse_req_t req, fuse_ino_t nodeid, const void *inarg)
 	f->conn.want = 0;

 	memset(&outarg, 0, sizeof(outarg));
-	outarg.major = FUSE_KERNEL_VERSION;
-	outarg.minor = FUSE_KERNEL_MINOR_VERSION;
+
+	outarg.major     = FUSE_KERNEL_VERSION;
+	outarg.minor     = FUSE_KERNEL_MINOR_VERSION;
+        outarg.max_pages = FUSE_DEFAULT_MAX_PAGES_PER_REQ;

 	if (arg->major < 7) {
 		fprintf(stderr, "fuse: unsupported protocol version: %u.%u\n",
@@ -1777,8 +1768,6 @@ static void do_init(fuse_req_t req, fuse_ino_t nodeid, const void *inarg)
 	}

 	if (arg->minor >= 6) {
-		if (f->conn.async_read)
-			f->conn.async_read = arg->flags & FUSE_ASYNC_READ;
 		if (arg->max_readahead < f->conn.max_readahead)
 			f->conn.max_readahead = arg->max_readahead;
 		if (arg->flags & FUSE_ASYNC_READ)
@@ -1797,12 +1786,16 @@ static void do_init(fuse_req_t req, fuse_ino_t nodeid, const void *inarg)
 			f->conn.capable |= FUSE_CAP_FLOCK_LOCKS;
                if (arg->flags & FUSE_POSIX_ACL)
                        f->conn.capable |= FUSE_CAP_POSIX_ACL;
+                if (arg->flags & FUSE_CACHE_SYMLINKS)
+                        f->conn.capable |= FUSE_CAP_CACHE_SYMLINKS;
                if (arg->flags & FUSE_ASYNC_DIO)
                        f->conn.capable |= FUSE_CAP_ASYNC_DIO;
                if (arg->flags & FUSE_PARALLEL_DIROPS)
                        f->conn.capable |= FUSE_CAP_PARALLEL_DIROPS;
+                if (arg->flags & FUSE_MAX_PAGES)
+                        f->conn.capable |= FUSE_CAP_MAX_PAGES;
 	} else {
-		f->conn.async_read = 0;
+                f->conn.want &= ~FUSE_CAP_ASYNC_READ;
 		f->conn.max_readahead = 0;
 	}

@@ -1823,14 +1816,10 @@ static void do_init(fuse_req_t req, fuse_ino_t nodeid, const void *inarg)
 	if (req->f->conn.proto_minor >= 18)
 		f->conn.capable |= FUSE_CAP_IOCTL_DIR;

-	if (f->atomic_o_trunc)
-		f->conn.want |= FUSE_CAP_ATOMIC_O_TRUNC;
 	if (f->op.getlk && f->op.setlk && !f->no_remote_posix_lock)
 		f->conn.want |= FUSE_CAP_POSIX_LOCKS;
 	if (f->op.flock && !f->no_remote_flock)
 		f->conn.want |= FUSE_CAP_FLOCK_LOCKS;
-	if (f->big_writes)
-		f->conn.want |= FUSE_CAP_BIG_WRITES;

 	if (bufsize < FUSE_MIN_READ_BUFFER) {
 		fprintf(stderr, "fuse: warning: buffer size too small: %zu\n",
@@ -1853,7 +1842,13 @@ static void do_init(fuse_req_t req, fuse_ino_t nodeid, const void *inarg)
 	if (f->no_splice_move)
 		f->conn.want &= ~FUSE_CAP_SPLICE_MOVE;

-	if (f->conn.async_read || (f->conn.want & FUSE_CAP_ASYNC_READ))
+        if ((arg->flags & FUSE_MAX_PAGES) && (f->conn.want & FUSE_CAP_MAX_PAGES))
+          {
+            outarg.flags     |= FUSE_MAX_PAGES;
+            outarg.max_pages  = f->conn.max_pages;
+          }
+
+	if (f->conn.want & FUSE_CAP_ASYNC_READ)
 		outarg.flags |= FUSE_ASYNC_READ;
 	if (f->conn.want & FUSE_CAP_POSIX_LOCKS)
 		outarg.flags |= FUSE_POSIX_LOCKS;
@@ -1869,6 +1864,8 @@ static void do_init(fuse_req_t req, fuse_ino_t nodeid, const void *inarg)
 		outarg.flags |= FUSE_FLOCK_LOCKS;
        if (f->conn.want & FUSE_CAP_POSIX_ACL)
                outarg.flags |= FUSE_POSIX_ACL;
+        if (f->conn.want & FUSE_CAP_CACHE_SYMLINKS)
+                outarg.flags |= FUSE_CACHE_SYMLINKS;
        if (f->conn.want & FUSE_CAP_ASYNC_DIO)
                outarg.flags |= FUSE_ASYNC_DIO;
        if (f->conn.want & FUSE_CAP_PARALLEL_DIROPS)
@@ -1899,6 +1896,7 @@ static void do_init(fuse_req_t req, fuse_ino_t nodeid, const void *inarg)
 			outarg.max_background);
 		fprintf(stderr, "   congestion_threshold=%i\n",
 		        outarg.congestion_threshold);
+                fprintf(stderr, "   max_pages=%d\n",outarg.max_pages);
 	}

        size_t outargsize;
@@ -1971,6 +1969,34 @@ static void do_notify_reply(fuse_req_t req, fuse_ino_t nodeid,
 		nreq->reply(nreq, req, nodeid, inarg, buf);
 }

+static
+void
+do_copy_file_range(fuse_req_t  req_,
+                   fuse_ino_t  nodeid_in_,
+                   const void *arg_)
+
+{
+  struct fuse_file_info ffi_in  = {0};
+  struct fuse_file_info ffi_out = {0};
+  struct fuse_copy_file_range_in *arg = (struct fuse_copy_file_range_in*)arg_;
+
+  ffi_in.fh  = arg->fh_in;
+  ffi_out.fh = arg->fh_out;
+
+  if(req_->f->op.copy_file_range == NULL)
+    fuse_reply_err(req_,ENOSYS);
+  else
+    req_->f->op.copy_file_range(req_,
+                                nodeid_in_,
+                                arg->off_in,
+                                &ffi_in,
+                                arg->nodeid_out,
+                                arg->off_out,
+                                &ffi_out,
+                                arg->len,
+                                arg->flags);
+}
+
 static int send_notify_iov(struct fuse_ll *f, struct fuse_chan *ch,
 			   int notify_code, struct iovec *iov, int count)
 {
@@ -2271,50 +2297,52 @@ int fuse_req_interrupted(fuse_req_t req)
 static struct {
 	void (*func)(fuse_req_t, fuse_ino_t, const void *);
 	const char *name;
-} fuse_ll_ops[] = {
-	[FUSE_LOOKUP]	   = { do_lookup,      "LOOKUP"	     },
-	[FUSE_FORGET]	   = { do_forget,      "FORGET"	     },
-	[FUSE_GETATTR]	   = { do_getattr,     "GETATTR"     },
-	[FUSE_SETATTR]	   = { do_setattr,     "SETATTR"     },
-	[FUSE_READLINK]	   = { do_readlink,    "READLINK"    },
-	[FUSE_SYMLINK]	   = { do_symlink,     "SYMLINK"     },
-	[FUSE_MKNOD]	   = { do_mknod,       "MKNOD"	     },
-	[FUSE_MKDIR]	   = { do_mkdir,       "MKDIR"	     },
-	[FUSE_UNLINK]	   = { do_unlink,      "UNLINK"	     },
-	[FUSE_RMDIR]	   = { do_rmdir,       "RMDIR"	     },
-	[FUSE_RENAME]	   = { do_rename,      "RENAME"	     },
-	[FUSE_LINK]	   = { do_link,	       "LINK"	     },
-	[FUSE_OPEN]	   = { do_open,	       "OPEN"	     },
-	[FUSE_READ]	   = { do_read,	       "READ"	     },
-	[FUSE_WRITE]	   = { do_write,       "WRITE"	     },
-	[FUSE_STATFS]	   = { do_statfs,      "STATFS"	     },
-	[FUSE_RELEASE]	   = { do_release,     "RELEASE"     },
-	[FUSE_FSYNC]	   = { do_fsync,       "FSYNC"	     },
-	[FUSE_SETXATTR]	   = { do_setxattr,    "SETXATTR"    },
-	[FUSE_GETXATTR]	   = { do_getxattr,    "GETXATTR"    },
-	[FUSE_LISTXATTR]   = { do_listxattr,   "LISTXATTR"   },
-	[FUSE_REMOVEXATTR] = { do_removexattr, "REMOVEXATTR" },
-	[FUSE_FLUSH]	   = { do_flush,       "FLUSH"	     },
-	[FUSE_INIT]	   = { do_init,	       "INIT"	     },
-	[FUSE_OPENDIR]	   = { do_opendir,     "OPENDIR"     },
-	[FUSE_READDIR]	   = { do_readdir,     "READDIR"     },
-	[FUSE_RELEASEDIR]  = { do_releasedir,  "RELEASEDIR"  },
-	[FUSE_FSYNCDIR]	   = { do_fsyncdir,    "FSYNCDIR"    },
-	[FUSE_GETLK]	   = { do_getlk,       "GETLK"	     },
-	[FUSE_SETLK]	   = { do_setlk,       "SETLK"	     },
-	[FUSE_SETLKW]	   = { do_setlkw,      "SETLKW"	     },
-	[FUSE_ACCESS]	   = { do_access,      "ACCESS"	     },
-	[FUSE_CREATE]	   = { do_create,      "CREATE"	     },
-	[FUSE_INTERRUPT]   = { do_interrupt,   "INTERRUPT"   },
-	[FUSE_BMAP]	   = { do_bmap,	       "BMAP"	     },
-	[FUSE_IOCTL]	   = { do_ioctl,       "IOCTL"	     },
-	[FUSE_POLL]	   = { do_poll,        "POLL"	     },
-	[FUSE_FALLOCATE]   = { do_fallocate,   "FALLOCATE"   },
-	[FUSE_DESTROY]	   = { do_destroy,     "DESTROY"     },
-	[FUSE_NOTIFY_REPLY] = { (void *) 1,    "NOTIFY_REPLY" },
-	[FUSE_BATCH_FORGET] = { do_batch_forget, "BATCH_FORGET" },
-	[CUSE_INIT]	   = { cuse_lowlevel_init, "CUSE_INIT"   },
-};
+} fuse_ll_ops[] =
+  {
+    [FUSE_LOOKUP]          = { do_lookup,          "LOOKUP"	     },
+    [FUSE_FORGET]          = { do_forget,          "FORGET"	     },
+    [FUSE_GETATTR]         = { do_getattr,         "GETATTR"         },
+    [FUSE_SETATTR]         = { do_setattr,         "SETATTR"         },
+    [FUSE_READLINK]        = { do_readlink,        "READLINK"        },
+    [FUSE_SYMLINK]         = { do_symlink,         "SYMLINK"         },
+    [FUSE_MKNOD]           = { do_mknod,           "MKNOD"	     },
+    [FUSE_MKDIR]           = { do_mkdir,           "MKDIR"	     },
+    [FUSE_UNLINK]          = { do_unlink,          "UNLINK"	     },
+    [FUSE_RMDIR]           = { do_rmdir,           "RMDIR"	     },
+    [FUSE_RENAME]          = { do_rename,          "RENAME"	     },
+    [FUSE_LINK]            = { do_link,	           "LINK"	     },
+    [FUSE_OPEN]            = { do_open,	           "OPEN"	     },
+    [FUSE_READ]            = { do_read,	           "READ"	     },
+    [FUSE_WRITE]           = { do_write,           "WRITE"	     },
+    [FUSE_STATFS]          = { do_statfs,          "STATFS"	     },
+    [FUSE_RELEASE]         = { do_release,         "RELEASE"         },
+    [FUSE_FSYNC]           = { do_fsync,           "FSYNC"	     },
+    [FUSE_SETXATTR]        = { do_setxattr,        "SETXATTR"        },
+    [FUSE_GETXATTR]        = { do_getxattr,        "GETXATTR"        },
+    [FUSE_LISTXATTR]       = { do_listxattr,       "LISTXATTR"       },
+    [FUSE_REMOVEXATTR]     = { do_removexattr,     "REMOVEXATTR"     },
+    [FUSE_FLUSH]           = { do_flush,           "FLUSH"	     },
+    [FUSE_INIT]            = { do_init,	           "INIT"	     },
+    [FUSE_OPENDIR]         = { do_opendir,         "OPENDIR"         },
+    [FUSE_READDIR]         = { do_readdir,         "READDIR"         },
+    [FUSE_RELEASEDIR]      = { do_releasedir,      "RELEASEDIR"      },
+    [FUSE_FSYNCDIR]        = { do_fsyncdir,        "FSYNCDIR"        },
+    [FUSE_GETLK]           = { do_getlk,           "GETLK"	     },
+    [FUSE_SETLK]           = { do_setlk,           "SETLK"	     },
+    [FUSE_SETLKW]          = { do_setlkw,          "SETLKW"	     },
+    [FUSE_ACCESS]          = { do_access,          "ACCESS"	     },
+    [FUSE_CREATE]          = { do_create,          "CREATE"	     },
+    [FUSE_INTERRUPT]       = { do_interrupt,       "INTERRUPT"       },
+    [FUSE_BMAP]            = { do_bmap,	           "BMAP"	     },
+    [FUSE_IOCTL]           = { do_ioctl,           "IOCTL"	     },
+    [FUSE_POLL]            = { do_poll,            "POLL"	     },
+    [FUSE_FALLOCATE]       = { do_fallocate,       "FALLOCATE"       },
+    [FUSE_DESTROY]         = { do_destroy,         "DESTROY"         },
+    [FUSE_NOTIFY_REPLY]    = { (void *) 1,         "NOTIFY_REPLY"    },
+    [FUSE_BATCH_FORGET]    = { do_batch_forget,    "BATCH_FORGET"    },
+    [FUSE_COPY_FILE_RANGE] = { do_copy_file_range, "COPY_FILE_RANGE" },
+    [CUSE_INIT]            = { cuse_lowlevel_init, "CUSE_INIT"       },
+  };

 #define FUSE_MAXOP (sizeof(fuse_ll_ops) / sizeof(fuse_ll_ops[0]))

@@ -2499,19 +2527,14 @@ static const struct fuse_opt fuse_ll_opts[] = {
 	{ "debug", offsetof(struct fuse_ll, debug), 1 },
 	{ "-d", offsetof(struct fuse_ll, debug), 1 },
 	{ "allow_root", offsetof(struct fuse_ll, allow_root), 1 },
-	{ "max_write=%u", offsetof(struct fuse_ll, conn.max_write), 0 },
 	{ "max_readahead=%u", offsetof(struct fuse_ll, conn.max_readahead), 0 },
 	{ "max_background=%u", offsetof(struct fuse_ll, conn.max_background), 0 },
 	{ "congestion_threshold=%u",
 	  offsetof(struct fuse_ll, conn.congestion_threshold), 0 },
-	{ "async_read", offsetof(struct fuse_ll, conn.async_read), 1 },
-	{ "sync_read", offsetof(struct fuse_ll, conn.async_read), 0 },
-	{ "atomic_o_trunc", offsetof(struct fuse_ll, atomic_o_trunc), 1},
 	{ "no_remote_lock", offsetof(struct fuse_ll, no_remote_posix_lock), 1},
 	{ "no_remote_lock", offsetof(struct fuse_ll, no_remote_flock), 1},
 	{ "no_remote_flock", offsetof(struct fuse_ll, no_remote_flock), 1},
 	{ "no_remote_posix_lock", offsetof(struct fuse_ll, no_remote_posix_lock), 1},
-	{ "big_writes", offsetof(struct fuse_ll, big_writes), 1},
 	{ "splice_write", offsetof(struct fuse_ll, splice_write), 1},
 	{ "no_splice_write", offsetof(struct fuse_ll, no_splice_write), 1},
 	{ "splice_move", offsetof(struct fuse_ll, splice_move), 1},
@@ -2535,14 +2558,9 @@ static void fuse_ll_version(void)
 static void fuse_ll_help(void)
 {
 	fprintf(stderr,
-"    -o max_write=N         set maximum size of write requests\n"
 "    -o max_readahead=N     set maximum readahead\n"
 "    -o max_background=N    set number of maximum background requests\n"
 "    -o congestion_threshold=N  set kernel's congestion threshold\n"
-"    -o async_read          perform reads asynchronously (default)\n"
-"    -o sync_read           perform reads synchronously\n"
-"    -o atomic_o_trunc      enable atomic open+truncate support\n"
-"    -o big_writes          enable larger than 4kB writes\n"
 "    -o no_remote_lock      disable remote file locking\n"
 "    -o no_remote_flock     disable remote file locking (BSD)\n"
 "    -o no_remote_posix_lock disable remove file locking (POSIX)\n"
@@ -2745,10 +2763,8 @@ struct fuse_session *fuse_lowlevel_new_common(struct fuse_args *args,
 		goto out;
 	}

-	f->conn.async_read = 1;
 	f->conn.max_write = UINT_MAX;
 	f->conn.max_readahead = UINT_MAX;
-	f->atomic_o_trunc = 0;
 	list_init_req(&f->list);
 	list_init_req(&f->interrupts);
 	list_init_nreq(&f->notify_list);
@@ -28,23 +28,26 @@ enum  {
 	KEY_VERSION,
 };

-struct helper_opts {
-	int singlethread;
-	int foreground;
-	int nodefault_subtype;
-	char *mountpoint;
+struct helper_opts
+{
+  int singlethread;
+  int foreground;
+  int nodefault_subtype;
+  char *mountpoint;
 };

 #define FUSE_HELPER_OPT(t, p) { t, offsetof(struct helper_opts, p), 1 }

-static const struct fuse_opt fuse_helper_opts[] = {
+static
+const
+struct fuse_opt fuse_helper_opts[] =
+  {
 	FUSE_HELPER_OPT("-d",		foreground),
 	FUSE_HELPER_OPT("debug",	foreground),
 	FUSE_HELPER_OPT("-f",		foreground),
 	FUSE_HELPER_OPT("-s",		singlethread),
 	FUSE_HELPER_OPT("fsname=",	nodefault_subtype),
 	FUSE_HELPER_OPT("subtype=",	nodefault_subtype),
-
 	FUSE_OPT_KEY("-h",		KEY_HELP),
 	FUSE_OPT_KEY("--help",		KEY_HELP),
 	FUSE_OPT_KEY("-ho",		KEY_HELP_NOHEADER),
@@ -144,37 +147,46 @@ static int add_default_subtype(const char *progname, struct fuse_args *args)
 	return res;
 }

-int fuse_parse_cmdline(struct fuse_args *args, char **mountpoint,
-		       int *multithreaded, int *foreground)
+int
+fuse_parse_cmdline(struct fuse_args  *args_,
+                   char             **mountpoint_,
+                   int               *multithreaded_,
+                   int               *foreground_)
 {
-	int res;
-	struct helper_opts hopts;
+  int res;
+  struct helper_opts hopts;

-	memset(&hopts, 0, sizeof(hopts));
-	res = fuse_opt_parse(args, &hopts, fuse_helper_opts,
-			     fuse_helper_opt_proc);
-	if (res == -1)
-		return -1;
+  memset(&hopts, 0, sizeof(hopts));

-	if (!hopts.nodefault_subtype) {
-		res = add_default_subtype(args->argv[0], args);
-		if (res == -1)
-			goto err;
-	}
-	if (mountpoint)
-		*mountpoint = hopts.mountpoint;
-	else
-		free(hopts.mountpoint);
+  res = fuse_opt_parse(args_,
+                       &hopts,
+                       fuse_helper_opts,
+                       fuse_helper_opt_proc);
+  if(res == -1)
+    return -1;

-	if (multithreaded)
-		*multithreaded = !hopts.singlethread;
-	if (foreground)
-		*foreground = hopts.foreground;
-	return 0;
+  if(!hopts.nodefault_subtype)
+    {
+      res = add_default_subtype(args_->argv[0], args_);
+      if(res == -1)
+        goto err;
+    }

-err:
-	free(hopts.mountpoint);
-	return -1;
+  if(mountpoint_)
+    *mountpoint_ = hopts.mountpoint;
+  else
+    free(hopts.mountpoint);
+
+  if(multithreaded_)
+    *multithreaded_ = !hopts.singlethread;
+  if(foreground_)
+    *foreground_ = hopts.foreground;
+
+  return 0;
+
+ err:
+  free(hopts.mountpoint);
+  return -1;
 }

 int fuse_daemonize(int foreground)
@@ -229,36 +241,41 @@ int fuse_daemonize(int foreground)
 	return 0;
 }

-static struct fuse_chan *fuse_mount_common(const char *mountpoint,
-					   struct fuse_args *args)
+static
+struct fuse_chan *
+fuse_mount_common(const char       *mountpoint_,
+                  struct fuse_args *args_)
 {
-	struct fuse_chan *ch;
-	int fd;
+  struct fuse_chan *ch;
+  int fd;

-	/*
-	 * Make sure file descriptors 0, 1 and 2 are open, otherwise chaos
-	 * would ensue.
-	 */
-	do {
-		fd = open("/dev/null", O_RDWR);
-		if (fd > 2)
-			close(fd);
-	} while (fd >= 0 && fd <= 2);
+  /*
+   * Make sure file descriptors 0, 1 and 2 are open, otherwise chaos
+   * would ensue.
+   */
+  do
+    {
+      fd = open("/dev/null", O_RDWR);
+      if(fd > 2)
+        close(fd);
+    } while(fd >= 0 && fd <= 2);

-	fd = fuse_mount_compat25(mountpoint, args);
-	if (fd == -1)
-		return NULL;
+  fd = fuse_mount_compat25(mountpoint_, args_);
+  if(fd == -1)
+    return NULL;

-	ch = fuse_kern_chan_new(fd);
-	if (!ch)
-		fuse_kern_unmount(mountpoint, fd);
+  ch = fuse_kern_chan_new(fd);
+  if(!ch)
+    fuse_kern_unmount(mountpoint_, fd);

-	return ch;
+  return ch;
 }

-struct fuse_chan *fuse_mount(const char *mountpoint, struct fuse_args *args)
+struct fuse_chan *
+fuse_mount(const char       *mountpoint_,
+           struct fuse_args *args_)
 {
-	return fuse_mount_common(mountpoint, args);
+  return fuse_mount_common(mountpoint_,args_);
 }

 static void fuse_unmount_common(const char *mountpoint, struct fuse_chan *ch)
@@ -1,7 +1,7 @@
 .\"t
 .\" Automatically generated by Pandoc 1.19.2.4
 .\"
-.TH "mergerfs" "1" "2019\-05\-18" "mergerfs user manual" ""
+.TH "mergerfs" "1" "2019\-09\-03" "mergerfs user manual" ""
 .hy
 .SH NAME
 .PP
@@ -79,65 +79,59 @@ found in \f[B]aufs\f[] and \f[B]overlayfs\f[].
 You can \f[B]not\f[] mount a read\-only filesystem and write to it.
 However, mergerfs will ignore read\-only drives when creating new files
 so you can mix read\-write and read\-only drives.
+It also does \f[B]not\f[] split data across drives.
+It is not RAID0 / striping.
+It is simply a union.
 .SH OPTIONS
 .SS mount options
 .IP \[bu] 2
-\f[B]allow_other\f[]: a libfuse option which allows users besides the
+\f[B]allow_other\f[]: A libfuse option which allows users besides the
 one which ran mergerfs to see the filesystem.
 This is required for most use\-cases.
 .IP \[bu] 2
-\f[B]direct_io\f[]: causes FUSE to bypass caching which can increase
-write speeds at the detriment of reads.
-Note that not enabling \f[C]direct_io\f[] will cause double caching of
-files and therefore less memory for caching generally (enable
-\f[B]dropcacheonclose\f[] to help with this problem).
-However, \f[C]mmap\f[] does not work when \f[C]direct_io\f[] is enabled.
-.IP \[bu] 2
-\f[B]minfreespace=value\f[]: the minimum space value used for creation
+\f[B]minfreespace=SIZE\f[]: The minimum space value used for creation
 policies.
 Understands \[aq]K\[aq], \[aq]M\[aq], and \[aq]G\[aq] to represent
 kilobyte, megabyte, and gigabyte respectively.
 (default: 4G)
 .IP \[bu] 2
-\f[B]moveonenospc=true|false\f[]: when enabled (set to \f[B]true\f[]) if
-a \f[B]write\f[] fails with \f[B]ENOSPC\f[] or \f[B]EDQUOT\f[] a scan of
-all drives will be done looking for the drive with the most free space
-which is at least the size of the file plus the amount which failed to
-write.
+\f[B]moveonenospc=BOOL\f[]: When enabled if a \f[B]write\f[] fails with
+\f[B]ENOSPC\f[] or \f[B]EDQUOT\f[] a scan of all drives will be done
+looking for the drive with the most free space which is at least the
+size of the file plus the amount which failed to write.
 An attempt to move the file to that drive will occur (keeping all
 metadata possible) and if successful the original is unlinked and the
 write retried.
 (default: false)
 .IP \[bu] 2
-\f[B]use_ino\f[]: causes mergerfs to supply file/directory inodes rather
+\f[B]use_ino\f[]: Causes mergerfs to supply file/directory inodes rather
 than libfuse.
 While not a default it is recommended it be enabled so that linked files
 share the same inode value.
 .IP \[bu] 2
-\f[B]dropcacheonclose=true|false\f[]: when a file is requested to be
-closed call \f[C]posix_fadvise\f[] on it first to instruct the kernel
-that we no longer need the data and it can drop its cache.
-Recommended when \f[B]direct_io\f[] is not enabled to limit double
-caching.
+\f[B]dropcacheonclose=BOOL\f[]: When a file is requested to be closed
+call \f[C]posix_fadvise\f[] on it first to instruct the kernel that we
+no longer need the data and it can drop its cache.
+Recommended when \f[B]cache.files=partial|full|auto\-full\f[] to limit
+double caching.
 (default: false)
 .IP \[bu] 2
-\f[B]symlinkify=true|false\f[]: when enabled (set to \f[B]true\f[]) and
-a file is not writable and its mtime or ctime is older than
-\f[B]symlinkify_timeout\f[] files will be reported as symlinks to the
-original files.
+\f[B]symlinkify=BOOL\f[]: When enabled and a file is not writable and
+its mtime or ctime is older than \f[B]symlinkify_timeout\f[] files will
+be reported as symlinks to the original files.
 Please read more below before using.
 (default: false)
 .IP \[bu] 2
-\f[B]symlinkify_timeout=value\f[]: time to wait, in seconds, to activate
+\f[B]symlinkify_timeout=INT\f[]: Time to wait, in seconds, to activate
 the \f[B]symlinkify\f[] behavior.
 (default: 3600)
 .IP \[bu] 2
-\f[B]nullrw=true|false\f[]: turns reads and writes into no\-ops.
+\f[B]nullrw=BOOL\f[]: Turns reads and writes into no\-ops.
 The request will succeed but do nothing.
 Useful for benchmarking mergerfs.
 (default: false)
 .IP \[bu] 2
-\f[B]ignorepponrename=true|false\f[]: ignore path preserving on rename.
+\f[B]ignorepponrename=BOOL\f[]: Ignore path preserving on rename.
 Typically rename and link act differently depending on the policy of
 \f[C]create\f[] (read below).
 Enabling this will cause rename and link to always use the non\-path
@@ -145,8 +139,8 @@ preserving behavior.
 This means files, when renamed or linked, will stay on the same drive.
 (default: false)
 .IP \[bu] 2
-\f[B]security_capability=true|false\f[]: If false return ENOATTR when
-xattr security.capability is queried.
+\f[B]security_capability=BOOL\f[]: If false return ENOATTR when xattr
+security.capability is queried.
 (default: true)
 .IP \[bu] 2
 \f[B]xattr=passthrough|noattr|nosys\f[]: Runtime control of xattrs.
@@ -156,8 +150,8 @@ Default is to passthrough xattr requests.
 or disabled.
 (default: passthrough)
 .IP \[bu] 2
-\f[B]link_cow=true|false\f[]: When enabled if a regular file is opened
-which has a link count > 1 it will copy the file to a temporary file and
+\f[B]link_cow=BOOL\f[]: When enabled if a regular file is opened which
+has a link count > 1 it will copy the file to a temporary file and
 rename over the original.
 Breaking the link and providing a basic copy\-on\-write function similar
 to cow\-shell.
@@ -177,13 +171,24 @@ calculations to ignore available space for branches mounted or tagged as
 create\[aq].
 (default: none)
 .IP \[bu] 2
-\f[B]posix_acl=true|false:\f[] enable POSIX ACL support (if supported by
+\f[B]posix_acl=BOOL\f[]: Enable POSIX ACL support (if supported by
 kernel and underlying filesystem).
 (default: false)
 .IP \[bu] 2
-\f[B]threads=num\f[]: number of threads to use in multithreaded mode.
-When set to zero (the default) it will attempt to discover and use the
-number of logical cores.
+\f[B]async_read=BOOL\f[]: Perform reads asynchronously.
+If disabled or unavailable the kernel will ensure there is at most one
+pending read request per file handle and will attempt to order requests
+by offset.
+(default: true)
+.IP \[bu] 2
+\f[B]fuse_msg_size=INT\f[]: Set the max number of pages per FUSE
+message.
+Only available on Linux >= 4.20 and ignored otherwise.
+(min: 1; max: 256; default: 256)
+.IP \[bu] 2
+\f[B]threads=INT\f[]: Number of threads to use in multithreaded mode.
+When set to zero it will attempt to discover and use the number of
+logical cores.
 If the lookup fails it will fall back to using 4.
 If the thread count is set negative it will look up the number of cores
 then divide by the absolute value.
@@ -192,44 +197,87 @@ threads=\-2 on an 8 core machine will result in 8 / 2 = 4 threads.
 There will always be at least 1 thread.
 NOTE: higher number of threads increases parallelism but usually
 decreases throughput.
-(default: number of cores) \f[I]NOTE2:\f[] the option is unavailable
-when built with system libfuse.
+(default: 0)
 .IP \[bu] 2
-\f[B]fsname=name\f[]: sets the name of the filesystem as seen in
+\f[B]fsname=STR\f[]: Sets the name of the filesystem as seen in
 \f[B]mount\f[], \f[B]df\f[], etc.
 Defaults to a list of the source paths concatenated together with the
 longest common prefix removed.
 .IP \[bu] 2
-\f[B]func.<func>=<policy>\f[]: sets the specific FUSE function\[aq]s
-policy.
+\f[B]func.FUNC=POLICY\f[]: Sets the specific FUSE function\[aq]s policy.
 See below for the list of value types.
 Example: \f[B]func.getattr=newest\f[]
 .IP \[bu] 2
-\f[B]category.<category>=<policy>\f[]: Sets policy of all FUSE functions
-in the provided category.
+\f[B]category.CATEGORY=POLICY\f[]: Sets policy of all FUSE functions in
+the provided category.
+See POLICIES section for defaults.
 Example: \f[B]category.create=mfs\f[]
 .IP \[bu] 2
-\f[B]cache.open=<int>\f[]: \[aq]open\[aq] policy cache timeout in
-seconds.
+\f[B]cache.open=INT\f[]: \[aq]open\[aq] policy cache timeout in seconds.
 (default: 0)
 .IP \[bu] 2
-\f[B]cache.statfs=<int>\f[]: \[aq]statfs\[aq] cache timeout in seconds.
+\f[B]cache.statfs=INT\f[]: \[aq]statfs\[aq] cache timeout in seconds.
 (default: 0)
 .IP \[bu] 2
-\f[B]cache.attr=<int>\f[]: file attribute cache timeout in seconds.
+\f[B]cache.attr=INT\f[]: File attribute cache timeout in seconds.
 (default: 1)
 .IP \[bu] 2
-\f[B]cache.entry=<int>\f[]: file name lookup cache timeout in seconds.
+\f[B]cache.entry=INT\f[]: File name lookup cache timeout in seconds.
 (default: 1)
 .IP \[bu] 2
-\f[B]cache.negative_entry=<int>\f[]: negative file name lookup cache
+\f[B]cache.negative_entry=INT\f[]: Negative file name lookup cache
 timeout in seconds.
 (default: 0)
+.IP \[bu] 2
+\f[B]cache.files=libfuse|off|partial|full|auto\-full\f[]: File page
+caching mode (default: libfuse)
+.IP \[bu] 2
+\f[B]cache.symlinks=BOOL\f[]: Cache symlinks (if supported by kernel)
+(default: false)
+.IP \[bu] 2
+\f[B]cache.readdir=BOOL\f[]: Cache readdir (if supported by kernel)
+(default: false)
+.IP \[bu] 2
+\f[B]direct_io\f[]: deprecated \- Bypass page cache.
+Use \f[C]cache.files=off\f[] instead.
+(default: false)
+.IP \[bu] 2
+\f[B]kernel_cache\f[]: deprecated \- Do not invalidate data cache on
+file open.
+Use \f[C]cache.files=full\f[] instead.
+(default: false)
+.IP \[bu] 2
+\f[B]auto_cache\f[]: deprecated \- Invalidate data cache if file mtime
+or size change.
+Use \f[C]cache.files=auto\-full\f[] instead.
+(default: false)
+.IP \[bu] 2
+\f[B]async_read\f[]: deprecated \- Perform reads asynchronously.
+Use \f[C]async_read=true\f[] instead.
+.IP \[bu] 2
+\f[B]sync_read\f[]: deprecated \- Perform reads synchronously.
+Use \f[C]async_read=false\f[] instead.
 .PP
 \f[B]NOTE:\f[] Options are evaluated in the order listed so if the
 options are \f[B]func.rmdir=rand,category.action=ff\f[] the
 \f[B]action\f[] category setting will override the \f[B]rmdir\f[]
 setting.
+.SS Value Types
+.IP \[bu] 2
+BOOL = \[aq]true\[aq] | \[aq]false\[aq]
+.IP \[bu] 2
+INT = [0,MAX_INT]
+.IP \[bu] 2
+SIZE = \[aq]NNM\[aq]; NN = INT, M = \[aq]K\[aq] | \[aq]M\[aq] |
+\[aq]G\[aq] | \[aq]T\[aq]
+.IP \[bu] 2
+STR = string
+.IP \[bu] 2
+FUNC = FUSE function
+.IP \[bu] 2
+CATEGORY = FUSE function category
+.IP \[bu] 2
+POLICY = mergerfs function policy
 .SS branches
 .PP
 The \[aq]branches\[aq] (formerly \[aq]srcmounts\[aq]) argument is a
@@ -285,6 +333,39 @@ be automatically included.
 \f[B]NOTE:\f[] for mounting via \f[B]fstab\f[] to work you must have
 \f[B]mount.fuse\f[] installed.
 For Ubuntu/Debian it is included in the \f[B]fuse\f[] package.
+.SS fuse_msg_size
+.PP
+FUSE applications communicate with the kernel over a special character
+device: \f[C]/dev/fuse\f[].
+A large portion of the overhead associated with FUSE is the cost of
+going back and forth from user space and kernel space over that device.
+Generally speaking the fewer trips needed the better the performance
+will be.
+Reducing the number of trips can be done a number of ways.
+Kernel level caching and increasing message sizes being two significant
+ones.
+When it comes to reads and writes if the message size is doubled the
+number of trips are appoximately halved.
+.PP
+In Linux 4.20 a new feature was added allowing the negotiation of the
+max message size.
+Since the size is in multiples of
+pages (https://en.wikipedia.org/wiki/Page_(computer_memory)) the feature
+is called \f[C]max_pages\f[].
+There is a maximum \f[C]max_pages\f[] value of 256 (1MiB) and minimum of
+1 (4KiB).
+The default used by Linux >=4.20, and hardcoded value used before 4.20,
+is 32 (128KiB).
+In mergerfs its referred to as \f[C]fuse_msg_size\f[] to make it clear
+what it impacts and provide some abstraction.
+.PP
+Since there should be no downsides to increasing \f[C]fuse_msg_size\f[]
+/ \f[C]max_pages\f[], outside a minor bump in RAM usage due to larger
+message buffers, mergerfs defaults the value to 256.
+On kernels before 4.20 the value has no effect.
+The reason the value is configurable is to enable experimentation and
+benchmarking.
+See the \f[C]nullrw\f[] section for benchmarking examples.
 .SS symlinkify
 .PP
 Due to the levels of indirection introduced by mergerfs and the
@@ -328,22 +409,22 @@ Example:
 .IP
 .nf
 \f[C]
-$\ dd\ if=/dev/zero\ of=/path/to/mergerfs/mount/benchmark\ ibs=1M\ obs=512\ count=1024\ conv=fdatasync
+$\ dd\ if=/dev/zero\ of=/path/to/mergerfs/mount/benchmark\ ibs=1M\ obs=512\ count=1024\ iflag=dsync,nocache\ oflag=dsync,nocache\ conv=fdatasync\ status=progress
 1024+0\ records\ in
 2097152+0\ records\ out
 1073741824\ bytes\ (1.1\ GB,\ 1.0\ GiB)\ copied,\ 15.4067\ s,\ 69.7\ MB/s

-$\ dd\ if=/dev/zero\ of=/path/to/mergerfs/mount/benchmark\ ibs=1M\ obs=1M\ count=1024\ conv=fdatasync
+$\ dd\ if=/dev/zero\ of=/path/to/mergerfs/mount/benchmark\ ibs=1M\ obs=1M\ count=1024\ iflag=dsync,nocache\ oflag=dsync,nocache\ conv=fdatasync\ status=progress
 1024+0\ records\ in
 1024+0\ records\ out
 1073741824\ bytes\ (1.1\ GB,\ 1.0\ GiB)\ copied,\ 0.219585\ s,\ 4.9\ GB/s

-$\ dd\ if=/path/to/mergerfs/mount/benchmark\ of=/dev/null\ bs=512\ count=102400\ conv=fdatasync
+$\ dd\ if=/path/to/mergerfs/mount/benchmark\ of=/dev/null\ bs=512\ count=102400\ iflag=dsync,nocache\ oflag=dsync,nocache\ conv=fdatasync\ status=progress
 102400+0\ records\ in
 102400+0\ records\ out
 52428800\ bytes\ (52\ MB,\ 50\ MiB)\ copied,\ 0.757991\ s,\ 69.2\ MB/s

-$\ dd\ if=/path/to/mergerfs/mount/benchmark\ of=/dev/null\ bs=1M\ count=1024\ conv=fdatasync
+$\ dd\ if=/path/to/mergerfs/mount/benchmark\ of=/dev/null\ bs=1M\ count=1024\ iflag=dsync,nocache\ oflag=dsync,nocache\ conv=fdatasync\ status=progress
 1024+0\ records\ in
 1024+0\ records\ out
 1073741824\ bytes\ (1.1\ GB,\ 1.0\ GiB)\ copied,\ 0.18405\ s,\ 5.8\ GB/s
@@ -442,7 +523,7 @@ T{
 N/A
 T}@T{
 fchmod, fchown, futimens, ftruncate, fallocate, fgetattr, fsync, ioctl
-(files), read, readdir, release, statfs, write
+(files), read, readdir, release, statfs, write, copy_file_range
 T}
 .TE
 .PP
@@ -541,6 +622,7 @@ T{
 eprand (existing path, random)
 T}@T{
 Calls \f[B]epall\f[] and then randomizes.
+Returns 1.
 T}
 T{
 erofs
@@ -586,6 +668,7 @@ T{
 rand (random)
 T}@T{
 Calls \f[B]all\f[] and then randomizes.
+Returns 1.
 T}
 .TE
 .SS Defaults
@@ -754,8 +837,8 @@ returned but it will still be possible.
 .PP
 readdir (http://linux.die.net/man/3/readdir) is different from all other
 filesystem functions.
-While it could have it\[aq]s own set of policies to tweak its behavior
-at this time it provides a simple union of files and directories found.
+While it could have its own set of policies to tweak its behavior at
+this time it provides a simple union of files and directories found.
 Remember that any action or information queried about these files and
 directories come from the respective function.
 For instance: an \f[B]ls\f[] is a \f[B]readdir\f[] and for each
@@ -771,7 +854,7 @@ and inodes.
 This means you will see the combined space of all sources.
 Total, used, and free.
 The sources however are dedupped based on the drive so multiple sources
-on the same drive will not result in double counting it\[aq]s space.
+on the same drive will not result in double counting its space.
 Filesystems mounted further down the tree of the branch will not be
 included when checking the mount\[aq]s stats.
 .PP
@@ -814,7 +897,7 @@ $\ su\ \-
 .fi
 .SS Generically
 .PP
-Have git, g++, make, python, automake, libtool installed.
+Have git, g++, make, python installed.
 .IP
 .nf
 \f[C]
@@ -858,6 +941,10 @@ wherever you configure the mounting of mergerfs (/etc/fstab).
 Use \f[C]xattr\ \-l\ /mountpoint/.mergerfs\f[] to see all supported
 keys.
 Some are informational and therefore read\-only.
+\f[C]setxattr\f[] will return EINVAL on read\-only keys.
+.SS Values
+.PP
+Same as the command line.
 .SS user.mergerfs.branches
 .PP
 \f[B]NOTE:\f[] formerly \f[C]user.mergerfs.srcmounts\f[] but said key is
@@ -912,30 +999,12 @@ T}
 .PP
 The \f[C]=NC\f[], \f[C]=RO\f[], \f[C]=RW\f[] syntax works just as on the
 command line.
-.SS minfreespace
-.PP
-Input: interger with an optional multiplier suffix.
-\f[B]K\f[], \f[B]M\f[], or \f[B]G\f[].
-.PP
-Output: value in bytes
-.SS moveonenospc
-.PP
-Input: \f[B]true\f[] and \f[B]false\f[]
-.PP
-Ouput: \f[B]true\f[] or \f[B]false\f[]
-.SS categories / funcs
-.PP
-Input: short policy string as described elsewhere in this document
-.PP
-Output: the policy string except for categories where its funcs have
-multiple types.
-In that case it will be a comma separated list
 .SS Example
 .IP
 .nf
 \f[C]
 [trapexit:/mnt/mergerfs]\ $\ xattr\ \-l\ .mergerfs
-user.mergerfs.branches:\ /mnt/a:/mnt/b
+user.mergerfs.branches:\ /mnt/a=RW:/mnt/b=RW
 user.mergerfs.minfreespace:\ 4294967295
 user.mergerfs.moveonenospc:\ false
 \&...
@@ -969,16 +1038,16 @@ served.
 To access the values you will need to issue a
 getxattr (http://linux.die.net/man/2/getxattr) for one of the following:
 .IP \[bu] 2
-\f[B]user.mergerfs.basepath:\f[] the base mount point for the file given
+\f[B]user.mergerfs.basepath\f[]: the base mount point for the file given
 the current getattr policy
 .IP \[bu] 2
-\f[B]user.mergerfs.relpath:\f[] the relative path of the file from the
+\f[B]user.mergerfs.relpath\f[]: the relative path of the file from the
 perspective of the mount point
 .IP \[bu] 2
-\f[B]user.mergerfs.fullpath:\f[] the full path of the original file
+\f[B]user.mergerfs.fullpath\f[]: the full path of the original file
 given the getattr policy
 .IP \[bu] 2
-\f[B]user.mergerfs.allpaths:\f[] a NUL (\[aq]\[aq]) separated list of
+\f[B]user.mergerfs.allpaths\f[]: a NUL (\[aq]\[aq]) separated list of
 full paths to all files found
 .IP
 .nf
@@ -996,6 +1065,29 @@ A\ B\ C
 /mnt/b/full/path/to/A
 \f[]
 .fi
+.SH UPGRADE
+.PP
+mergerfs can be upgraded live by mounting on top of the previous
+version.
+Simply install the new version of mergerfs and follow the instructions
+below.
+.PP
+Add \f[C]nonempty\f[] to your mergerfs option list and call mergerfs
+again or if using \f[C]/etc/fstab\f[] call for it to mount again.
+Existing open files and such will continue to work fine though they
+won\[aq]t see runtime changes since any such change would be the new
+mount.
+If you plan on changing settings with the new mount you should / could
+apply those before mounting the new version.
+.IP
+.nf
+\f[C]
+$\ sudo\ mount\ /mnt/mergerfs
+$\ mount\ |\ grep\ mergerfs
+media\ on\ /mnt/mergerfs\ type\ fuse.mergerfs\ (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other)
+media\ on\ /mnt/mergerfs\ type\ fuse.mergerfs\ (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other)
+\f[]
+.fi
 .SH TOOLING
 .IP \[bu] 2
 https://github.com/trapexit/mergerfs\-tools
@@ -1014,6 +1106,9 @@ pool
 mergerfs.balance: Rebalance files across drives by moving them from the
 most filled to the least filled
 .IP \[bu] 2
+mergerfs.consolidate: move files within a single mergerfs directory to
+the drive with most free space
+.IP \[bu] 2
 mergerfs.mktrash: Creates FreeDesktop.org Trash specification compatible
 directories on a mergerfs mount
 .IP \[bu] 2
@@ -1029,40 +1124,64 @@ bad blocks and find the files using those blocks
 .SH CACHING
 .SS page caching
 .PP
-The kernel performs caching of data pages on all files not opened with
-\f[C]O_DIRECT\f[].
-Due to mergerfs using FUSE and therefore being a userland process the
-kernel can double cache the content being read through mergerfs.
-Once from the underlying filesystem and once for mergerfs.
-Using \f[C]direct_io\f[] and/or \f[C]dropcacheonclose\f[] help minimize
-the double caching.
-\f[C]direct_io\f[] will instruct the kernel to bypass the page cache for
-files opened through mergerfs.
-\f[C]dropcacheonclose\f[] will cause mergerfs to instruct the kernel to
-flush a file\[aq]s page cache for which it had opened when closed.
-If most data is read once its probably best to enable both (read above
-for details and limitations).
+https://en.wikipedia.org/wiki/Page_cache
 .PP
-If a cache is desired for mergerfs do not enable \f[C]direct_io\f[] and
-instead possibly use \f[C]auto_cache\f[] or \f[C]kernel_cache\f[].
-By default FUSE will invalidate cached pages when a file is opened.
-By using \f[C]auto_cache\f[] it will instead use \f[C]getattr\f[] to
-check if a file has changed when the file is opened and if so will flush
-the cache.
-\f[C]ac_attr_timeout\f[] is the timeout for keeping said cache.
-Alternatively \f[C]kernel_cache\f[] will keep the cache across opens
-unless invalidated through other means.
-You should only uses these if you do not plan to write/modify the same
-files through mergerfs and the underlying filesystem at the same time.
-It could lead to corruption.
-Then again doing so without caching can also cause issues.
+tl;dr: * cache.files=off: Disables page caching.
+Underlying files cached, mergerfs files are not.
+* cache.files=partial: Enables page caching.
+Underlying files cached, mergerfs files cached while open.
+* cache.files=full: Enables page caching.
+Underlying files cached, mergerfs files cached across opens.
+* cache.files=auto\-full: Enables page caching.
+Underlying files cached, mergerfs files cached across opens if mtime and
+size are unchanged since previous open.
+* cache.files=libfuse: follow traditional libfuse \f[C]direct_io\f[],
+\f[C]kernel_cache\f[], and \f[C]auto_cache\f[] arguments.
 .PP
-It\[aq]s a difficult balance between memory usage, cache bloat &
-duplication, and performance.
+FUSE, which mergerfs uses, offers a number of page caching modes.
+mergerfs tries to simplify their use via the \f[C]cache.files\f[]
+option.
+It can and should replace usage of \f[C]direct_io\f[],
+\f[C]kernel_cache\f[], and \f[C]auto_cache\f[].
+.PP
+Due to mergerfs using FUSE and therefore being a userland process
+proxying existing filesystems the kernel will double cache the content
+being read and written through mergerfs.
+Once from the underlying filesystem and once from mergerfs (it sees them
+as two separate entities).
+Using \f[C]cache.files=off\f[] will keep the double caching from
+happening by disabling caching of mergerfs but this has the side effect
+that \f[I]all\f[] read and write calls will be passed to mergerfs which
+may be slower than enabling caching, you lose shared \f[C]mmap\f[]
+support which can affect apps such as rtorrent, and no read\-ahead will
+take place.
+The kernel will still cache the underlying filesystem data but that only
+helps so much given mergerfs will still process all requests.
+.PP
+If you do enable file page caching,
+\f[C]cache.files=partial|full|auto\-full\f[], you should also enable
+\f[C]dropcacheonclose\f[] which will cause mergerfs to instruct the
+kernel to flush the underlying file\[aq]s page cache when the file is
+closed.
+This behavior is the same as the rsync fadvise / drop cache patch and
+Feh\[aq]s nocache project.
+.PP
+If most files are read once through and closed (like media) it is best
+to enable \f[C]dropcacheonclose\f[] regardless of caching mode in order
+to minimize buffer bloat.
+.PP
+It is difficult to balance memory usage, cache bloat & duplication, and
+performance.
 Ideally mergerfs would be able to disable caching for the files it
 reads/writes but allow page caching for itself.
 That would limit the FUSE overhead.
-However, there isn\[aq]t good way to achieve this.
+However, there isn\[aq]t a good way to achieve this.
+It would need to open all files with O_DIRECT which places limitations
+on the what underlying filesystems would be supported and complicates
+the code.
+.PP
+kernel documenation:
+https://www.kernel.org/doc/Documentation/filesystems/fuse\-io.txt
 .SS entry & attribute caching
 .PP
 Given the relatively high cost of FUSE due to the kernel <\-> userspace
@@ -1113,19 +1232,40 @@ policy will be cached for the number of seconds its set to.
 Example: If the create policy is \f[C]mfs\f[] and the timeout is 60 then
 for that 60 seconds the same drive will be returned as the target for
 creates because the available space won\[aq]t be updated for that time.
+.SS symlink caching
+.PP
+As of version 4.20 Linux supports symlink caching.
+Significant performance increases can be had in workloads which use a
+lot of symlinks.
+Setting \f[C]cache.symlinks=true\f[] will result in requesting symlink
+caching from the kernel only if supported.
+As a result its safe to enable it on systems prior to 4.20.
+That said it is disabled by default for now.
+You can see if caching is enabled by querying the xattr
+\f[C]user.mergerfs.cache.symlinks\f[] but given it must be requested at
+startup you can not change it at runtime.
+.SS readdir caching
+.PP
+As of version 4.20 Linux supports readdir caching.
+This can have a significant impact on directory traversal.
+Especially when combined with entry (\f[C]cache.entry\f[]) and attribute
+(\f[C]cache.attr\f[]) caching.
+Setting \f[C]cache.readdir=true\f[] will result in requesting readdir
+caching from the kernel on each \f[C]opendir\f[].
+If the kernel doesn\[aq]t support readdir caching setting the option to
+\f[C]true\f[] has no effect.
+This option is configuarable at runtime via xattr
+\f[C]user.mergerfs.cache.readdir\f[].
 .SS writeback caching
 .PP
 writeback caching is a technique for improving write speeds by batching
 writes at a faster device and then bulk writing to the slower device.
 With FUSE the kernel will wait for a number of writes to be made and
 then send it to the filesystem as one request.
-mergerfs currently uses a slightly modified and vendored libfuse 2.9.7
-which does not support writeback caching.
-However, a prototype port to libfuse 3.x has been made and the writeback
-cache appears to work as expected (though performance improvements
-greatly depend on the way the client app writes data).
-Once the port is complete and thoroughly tested writeback caching will
-be available.
+mergerfs currently uses a modified and vendored libfuse 2.9.7 which does
+not support writeback caching.
+Adding said feature should not be difficult but benchmarking needs to be
+done to see if what effect it will have.
 .SS tiered caching
 .PP
 Some storage technologies support what some call "tiered" caching.
@@ -1262,11 +1402,14 @@ all the underlying directories.
 Use \f[C]mergerfs.fsck\f[] to audit the drive for out of sync
 permissions.
 .IP \[bu] 2
-Do \f[B]not\f[] use \f[C]direct_io\f[] if you expect applications (such
-as rtorrent) to mmap (http://linux.die.net/man/2/mmap) files.
-It is not currently supported in FUSE w/ \f[C]direct_io\f[] enabled.
+Do \f[B]not\f[] use \f[C]cache.files=off\f[] or \f[C]direct_io\f[] if
+you expect applications (such as rtorrent) to
+mmap (http://linux.die.net/man/2/mmap) files.
+Shared mmap is not currently supported in FUSE w/ \f[C]direct_io\f[]
+enabled.
 Enabling \f[C]dropcacheonclose\f[] is recommended when
-\f[C]direct_io\f[] is disabled.
+\f[C]cache.files=partial|full|auto\-full\f[] or
+\f[C]direct_io=false\f[].
 .IP \[bu] 2
 Since POSIX functions give only a singular error or success its
 difficult to determine the proper behavior when applying the function to
@@ -1344,19 +1487,8 @@ mergerfs.dedup to clean up the old paths or manually remove them from
 the branches directly.
 .SS cached memory appears greater than it should be
 .PP
-Use the \f[C]direct_io\f[] option as described above.
-Due to what mergerfs is doing there ends up being two caches of a file
-under normal usage.
-One from the underlying filesystem and one from mergerfs.
-Enabling \f[C]direct_io\f[] removes the mergerfs cache.
-This saves on memory but means the kernel needs to communicate with
-mergerfs more often and can therefore result in slower speeds.
-.PP
-Since enabling \f[C]direct_io\f[] disables \f[C]mmap\f[] this is not an
-ideal situation however write speeds should be increased.
-.PP
-If \f[C]direct_io\f[] is disabled it is probably a good idea to enable
-\f[C]dropcacheonclose\f[] to minimize double caching.
+Use \f[C]cache.files=off\f[] or \f[C]direct_io=true\f[].
+See the section on page caching.
 .SS NFS clients returning ESTALE / Stale file handle
 .PP
 Be sure to use \f[C]noforget\f[] and \f[C]use_ino\f[] arguments.
@@ -1369,7 +1501,8 @@ Try enabling the \f[C]use_ino\f[] option.
 Some have reported that it fixes the issue.
 .SS rtorrent fails with ENODEV (No such device)
 .PP
-Be sure to turn off \f[C]direct_io\f[].
+Be sure to set \f[C]cache.files=partial|full|auto\-full\f[] or turn off
+\f[C]direct_io\f[].
 rtorrent and some other applications use
 mmap (http://linux.die.net/man/2/mmap) to read and write to files and
 offer no failback to traditional methods.
@@ -1434,7 +1567,7 @@ Particularly \f[B]STATUS_NOT_SAME_DEVICE\f[] which comes from the
 \f[B]EXDEV\f[] which is returned by \f[B]rename\f[] when the call is
 crossing mount points.
 When a program gets an \f[B]EXDEV\f[] it needs to explicitly take an
-alternate action to accomplish it\[aq]s goal.
+alternate action to accomplish its goal.
 In the case of \f[B]mv\f[] or similar it tries \f[B]rename\f[] and on
 \f[B]EXDEV\f[] falls back to a manual copying of data between the two
 locations and unlinking the source.
@@ -1616,8 +1749,8 @@ Users have reported running mergerfs on everything from a Raspberry Pi
 to dual socket Xeon systems with >20 cores.
 I\[aq]m aware of at least a few companies which use mergerfs in
 production.
-Open Media Vault (https://www.openmediavault.org) includes mergerfs is
-it\[aq]s sole solution for pooling drives.
+Open Media Vault (https://www.openmediavault.org) includes mergerfs as
+its sole solution for pooling drives.
 .SS Can mergerfs be used with drives which already have data / are in
 use?
 .PP
@@ -1639,7 +1772,7 @@ Yes.
 You need to use \f[C]use_ino\f[] to support proper reporting of inodes.
 .PP
 What mergerfs does not do is fake hard links across branches.
-Read the section "rename & link" for how it.
+Read the section "rename & link" for how it works.
 .SS Does mergerfs support CoW / copy\-on\-write?
 .PP
 Not in the sense of a filesystem like BTRFS or ZFS nor in the overlayfs
@@ -1654,8 +1787,7 @@ file.
 .PP
 It\[aq]s almost always a permissions issue.
 Unlike mhddfs, which runs as root and attempts to access content as
-such, mergerfs always changes it\[aq]s credentials to that of the
-caller.
+such, mergerfs always changes its credentials to that of the caller.
 This means that if the user does not have access to a file or directory
 than neither will mergerfs.
 However, because mergerfs is creating a union of paths it may be able to
@@ -1777,8 +1909,8 @@ here (http://louwrentius.com/the-hidden-cost-of-using-zfs-for-your-home-nas.html
 .PP
 Yes, however its not recommended to use the same file from within the
 pool and from without at the same time.
-Especially if using caching of any kind (cache.entry, cache.attr,
-ac_attr_timeout, cache.negative_entry, auto_cache, kernel_cache).
+Especially if using caching of any kind (cache.files, cache.entry,
+cache.attr, cache.negative_entry, cache.symlinks, cache.readdir, etc.).
 .SS Why do I get an "out of space" / "no space left on device" / ENOSPC
 error even though there appears to be lots of space available?
 .PP
@@ -1868,13 +2000,14 @@ and\ the\ kernel\ use\ internally\ (also\ called\ the\ "nodeid").
 .SS I notice massive slowdowns of writes over NFS
 .PP
 Due to how NFS works and interacts with FUSE when not using
-\f[C]direct_io\f[] its possible that a getxattr for
-\f[C]security.capability\f[] will be issued prior to any write.
+\f[C]cache.files=off\f[] or \f[C]direct_io\f[] its possible that a
+getxattr for \f[C]security.capability\f[] will be issued prior to any
+write.
 This will usually result in a massive slowdown for writes.
-Using \f[C]direct_io\f[] will keep this from happening (and generally
-good to enable unless you need the features it disables) but the
-\f[C]security_capability\f[] option can also help by short circuiting
-the call and returning \f[C]ENOATTR\f[].
+Using \f[C]cache.files=off\f[] or \f[C]direct_io\f[] will keep this from
+happening (and generally good to enable unless you need the features it
+disables) but the \f[C]security_capability\f[] option can also help by
+short circuiting the call and returning \f[C]ENOATTR\f[].
 .PP
 You could also set \f[C]xattr\f[] to \f[C]noattr\f[] or \f[C]nosys\f[]
 to short circuit or stop all xattr requests.
@@ -1934,26 +2067,30 @@ assuming there are few users.
 .PP
 NOTE: be sure to read about these features before changing them
 .IP \[bu] 2
-add (or remove) \f[C]direct_io\f[]
-.IP \[bu] 2
-add (or remove) \f[C]auto_cache\f[]
-.IP \[bu] 2
-add (or remove) \f[C]kernel_cache\f[]
-.IP \[bu] 2
-add (or remove) \f[C]splice_move\f[], \f[C]splice_read\f[], and
+enable (or disable) \f[C]splice_move\f[], \f[C]splice_read\f[], and
 \f[C]splice_write\f[]
 .IP \[bu] 2
 increase cache timeouts \f[C]cache.attr\f[], \f[C]cache.entry\f[],
 \f[C]cache.negative_entry\f[]
 .IP \[bu] 2
-enable \f[C]cache.open\f[] and/or \f[C]cache.statfs\f[]
+enable (or disable) page caching (\f[C]cache.files\f[])
 .IP \[bu] 2
-change the number opf worker threads
+enable \f[C]cache.open\f[]
+.IP \[bu] 2
+enable \f[C]cache.statfs\f[]
+.IP \[bu] 2
+enable \f[C]cache.symlinks\f[]
+.IP \[bu] 2
+enable \f[C]cache.readdir\f[]
+.IP \[bu] 2
+change the number of worker threads
 .IP \[bu] 2
 disable \f[C]security_capability\f[] and/or \f[C]xattr\f[]
 .IP \[bu] 2
 disable \f[C]posix_acl\f[]
 .IP \[bu] 2
+disable \f[C]async_read\f[]
+.IP \[bu] 2
 test theoretical performance using \f[C]nullrw\f[] or mounting a ram
 disk
 .IP \[bu] 2
@@ -19,12 +19,12 @@
 #include "fs.hpp"
 #include "rwlock.hpp"

-#include <unistd.h>
-#include <sys/stat.h>
-
 #include <string>
 #include <vector>

+#include <unistd.h>
+#include <sys/stat.h>
+
 #define MINFREESPACE_DEFAULT (4294967295ULL)
 #define POLICYINIT(X) X(policies[FuseFunc::Enum::X])

@@ -49,6 +49,11 @@ Config::Config()
    statfs(StatFS::BASE),
    statfs_ignore(StatFSIgnore::NONE),
    posix_acl(false),
+    cache_symlinks(false),
+    cache_readdir(false),
+    async_read(true),
+    cache_files(CacheFiles::LIBFUSE),
+    fuse_msg_size(FUSE_MAX_MAX_PAGES),
    POLICYINIT(access),
    POLICYINIT(chmod),
    POLICYINIT(chown),
@@ -121,3 +126,74 @@ Config::set_category_policy(const string &category_,

  return 0;
 }
+
+Config::CacheFiles::operator int() const
+{
+  return _data;
+}
+
+Config::CacheFiles::operator std::string() const
+{
+  switch(_data)
+    {
+    case OFF:
+      return "off";
+    case PARTIAL:
+      return "partial";
+    case FULL:
+      return "full";
+    case AUTO_FULL:
+      return "auto-full";
+    case LIBFUSE:
+      return "libfuse";
+    case INVALID:
+      break;
+    }
+
+  return "";
+}
+
+Config::CacheFiles::CacheFiles()
+  : _data(INVALID)
+{
+
+}
+
+Config::CacheFiles::CacheFiles(Config::CacheFiles::Enum data_)
+  : _data(data_)
+{
+
+}
+
+bool
+Config::CacheFiles::valid() const
+{
+  return (_data != INVALID);
+}
+
+Config::CacheFiles&
+Config::CacheFiles::operator=(const Config::CacheFiles::Enum data_)
+{
+  _data = data_;
+
+  return *this;
+}
+
+Config::CacheFiles&
+Config::CacheFiles::operator=(const std::string &data_)
+{
+  if(data_ == "off")
+    _data = OFF;
+  else if(data_ == "partial")
+    _data = PARTIAL;
+  else if(data_ == "full")
+    _data = FULL;
+  else if(data_ == "auto-full")
+    _data = AUTO_FULL;
+  else if(data_ == "libfuse")
+    _data = LIBFUSE;
+  else
+    _data = INVALID;
+
+  return *this;
+}
@@ -51,6 +51,34 @@ public:
      };
  };

+  class CacheFiles
+  {
+  public:
+    enum Enum
+      {
+        INVALID = -1,
+        LIBFUSE,
+        OFF,
+        PARTIAL,
+        FULL,
+        AUTO_FULL
+      };
+
+    CacheFiles();
+    CacheFiles(Enum);
+
+    operator int() const;
+    operator std::string() const;
+
+    CacheFiles& operator=(const Enum);
+    CacheFiles& operator=(const std::string&);
+
+    bool valid() const;
+
+  private:
+    Enum _data;
+  };
+
 public:
  Config();

@@ -68,6 +96,8 @@ public:
  uint64_t                 minfreespace;
  bool                     moveonenospc;
  bool                     direct_io;
+  bool                     kernel_cache;
+  bool                     auto_cache;
  bool                     dropcacheonclose;
  bool                     symlinkify;
  time_t                   symlinkify_timeout;
@@ -79,6 +109,11 @@ public:
  StatFS::Enum             statfs;
  StatFSIgnore::Enum       statfs_ignore;
  bool                     posix_acl;
+  bool                     cache_symlinks;
+  bool                     cache_readdir;
+  bool                     async_read;
+  CacheFiles               cache_files;
+  uint16_t                 fuse_msg_size;

 public:
  const Policy  *policies[FuseFunc::Enum::END];
@@ -0,0 +1,71 @@
+/*
+  Copyright (c) 2019, Antonio SJ Musumeci <trapexit@spawn.link>
+
+  Permission to use, copy, modify, and/or distribute this software for any
+  purpose with or without fee is hereby granted, provided that the above
+  copyright notice and this permission notice appear in all copies.
+
+  THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+  WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+  MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+  ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+  WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+  ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+  OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+*/
+
+#include "errno.hpp"
+#include "fileinfo.hpp"
+#include "fs_copy_file_range.hpp"
+
+#include <fuse.h>
+
+#include <stdio.h>
+
+namespace l
+{
+  static
+  ssize_t
+  copy_file_range(const int fd_in_,
+                  off_t     offset_in_,
+                  const int fd_out_,
+                  off_t     offset_out_,
+                  size_t    size_,
+                  int       flags_)
+  {
+    ssize_t rv;
+
+    rv = fs::copy_file_range(fd_in_,
+                             &offset_in_,
+                             fd_out_,
+                             &offset_out_,
+                             size_,
+                             flags_);
+
+    return ((rv == -1) ? -errno : rv);
+  }
+}
+
+namespace FUSE
+{
+  ssize_t
+  copy_file_range(const char            *path_in_,
+                  struct fuse_file_info *ffi_in_,
+                  off_t                  offset_in_,
+                  const char            *path_out_,
+                  struct fuse_file_info *ffi_out_,
+                  off_t                  offset_out_,
+                  size_t                 size_,
+                  int                    flags_)
+  {
+    FileInfo *fi_in  = reinterpret_cast<FileInfo*>(ffi_in_->fh);
+    FileInfo *fi_out = reinterpret_cast<FileInfo*>(ffi_out_->fh);
+
+    return l::copy_file_range(fi_in->fd,
+                              offset_in_,
+                              fi_out->fd,
+                              offset_out_,
+                              size_,
+                              flags_);
+  }
+}
@@ -0,0 +1,30 @@
+/*
+  Copyright (c) 2019, Antonio SJ Musumeci <trapexit@spawn.link>
+
+  Permission to use, copy, modify, and/or distribute this software for any
+  purpose with or without fee is hereby granted, provided that the above
+  copyright notice and this permission notice appear in all copies.
+
+  THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+  WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+  MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+  ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+  WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+  ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+  OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+*/
+
+#pragma once
+
+namespace FUSE
+{
+  ssize_t
+  copy_file_range(const char            *path_in,
+                  struct fuse_file_info *ffi_in,
+                  off_t                  offset_in,
+                  const char            *path_out,
+                  struct fuse_file_info *ffi_out,
+                  off_t                  offset_out,
+                  size_t                 size,
+                  int                    flags);
+}
@@ -31,6 +31,7 @@

 using std::string;
 using std::vector;
+typedef Config::CacheFiles CacheFiles;

 namespace l
 {
@@ -123,7 +124,35 @@ namespace FUSE
    const ugid::Set          ugid(fc->uid,fc->gid);
    const rwlock::ReadGuard  readlock(&config.branches_lock);

-    ffi_->direct_io = config.direct_io;
+    switch(config.cache_files)
+      {
+      case CacheFiles::LIBFUSE:
+        ffi_->direct_io  = config.direct_io;
+        ffi_->keep_cache = config.kernel_cache;
+        ffi_->auto_cache = config.auto_cache;
+        break;
+      case CacheFiles::OFF:
+        ffi_->direct_io  = 1;
+        ffi_->keep_cache = 0;
+        ffi_->auto_cache = 0;
+        break;
+      case CacheFiles::PARTIAL:
+        ffi_->direct_io  = 0;
+        ffi_->keep_cache = 0;
+        ffi_->auto_cache = 0;
+        break;
+      case CacheFiles::FULL:
+        ffi_->direct_io  = 0;
+        ffi_->keep_cache = 1;
+        ffi_->auto_cache = 0;
+        break;
+      case CacheFiles::AUTO_FULL:
+        ffi_->direct_io  = 0;
+        ffi_->keep_cache = 0;
+        ffi_->auto_cache = 1;
+        break;
+      }
+
    return l::create(config.getattr,
                     config.create,
                     config.branches,
@@ -225,6 +225,26 @@ namespace l
      }
  }

+  static
+  void
+  getxattr_controlfile(const Config::CacheFiles &cache_files_,
+                       string                   &attrvalue_)
+  {
+    attrvalue_ = (string)cache_files_;
+  }
+
+  static
+  void
+  getxattr_controlfile(const uint16_t &uint16_,
+                       string         &attrvalue_)
+  {
+    std::ostringstream os;
+
+    os << uint16_;
+
+    attrvalue_ = os.str();
+  }
+
  static
  void
  getxattr_controlfile_policies(const Config &config,
@@ -348,6 +368,10 @@ namespace l
          l::getxattr_controlfile_bool(config.direct_io,attrvalue);
        else if(attr[2] == "posix_acl")
          l::getxattr_controlfile_bool(config.posix_acl,attrvalue);
+        else if(attr[2] == "async_read")
+          l::getxattr_controlfile_bool(config.async_read,attrvalue);
+        else if(attr[2] == "fuse_msg_size")
+          l::getxattr_controlfile(config.fuse_msg_size,attrvalue);
        break;

      case 4:
@@ -365,6 +389,12 @@ namespace l
          l::getxattr_controlfile_cache_entry(attrvalue);
        else if((attr[2] == "cache") && (attr[3] == "negative_entry"))
          l::getxattr_controlfile_cache_negative_entry(attrvalue);
+        else if((attr[2] == "cache") && (attr[3] == "symlinks"))
+          l::getxattr_controlfile_bool(config.cache_symlinks,attrvalue);
+        else if((attr[2] == "cache") && (attr[3] == "readdir"))
+          l::getxattr_controlfile_bool(config.cache_readdir,attrvalue);
+        else if((attr[2] == "cache") && (attr[3] == "files"))
+          l::getxattr_controlfile(config.cache_files,attrvalue);
        break;
      }

@@ -21,12 +21,60 @@

 namespace l
 {
+  static
+  void
+  want(fuse_conn_info *conn_,
+       const int       flag_)
+  {
+    conn_->want |= flag_;
+  }
+
+  static
+  bool
+  capable(fuse_conn_info *conn_,
+          const int       flag_)
+  {
+    return !!(conn_->capable & flag_);
+  }
+
+  static
  void
  want_if_capable(fuse_conn_info *conn_,
                  const int       flag_)
  {
-    if(conn_->capable & flag_)
-      conn_->want |= flag_;
+    if(capable(conn_,flag_))
+      want(conn_,flag_);
+  }
+
+  static
+  void
+  want_if_capable(fuse_conn_info *conn_,
+                  const int       flag_,
+                  bool           *want_)
+  {
+    if(*want_ && l::capable(conn_,flag_))
+      {
+        l::want(conn_,flag_);
+        return;
+      }
+
+    *want_ = false;
+  }
+
+  static
+  void
+  want_if_capable_max_pages(fuse_conn_info *conn_,
+                            Config         &c_)
+  {
+    if(l::capable(conn_,FUSE_CAP_MAX_PAGES))
+      {
+        l::want(conn_,FUSE_CAP_MAX_PAGES);
+        conn_->max_pages = c_.fuse_msg_size;
+      }
+    else
+      {
+        c_.fuse_msg_size = FUSE_DEFAULT_MAX_PAGES_PER_REQ;
+      }
  }
 }

@@ -35,18 +83,21 @@ namespace FUSE
  void *
  init(fuse_conn_info *conn_)
  {
+    Config &c = Config::get_writable();
+
    ugid::init();

-    l::want_if_capable(conn_,FUSE_CAP_ASYNC_READ);
+    l::want_if_capable(conn_,FUSE_CAP_ASYNC_DIO);
+    l::want_if_capable(conn_,FUSE_CAP_ASYNC_READ,&c.async_read);
    l::want_if_capable(conn_,FUSE_CAP_ATOMIC_O_TRUNC);
    l::want_if_capable(conn_,FUSE_CAP_BIG_WRITES);
+    l::want_if_capable(conn_,FUSE_CAP_CACHE_SYMLINKS,&c.cache_symlinks);
    l::want_if_capable(conn_,FUSE_CAP_DONT_MASK);
    l::want_if_capable(conn_,FUSE_CAP_IOCTL_DIR);
-    l::want_if_capable(conn_,FUSE_CAP_ASYNC_DIO);
    l::want_if_capable(conn_,FUSE_CAP_PARALLEL_DIROPS);
-    if(Config::get().posix_acl)
-      l::want_if_capable(conn_,FUSE_CAP_POSIX_ACL);
+    l::want_if_capable(conn_,FUSE_CAP_POSIX_ACL,&c.posix_acl);
+    l::want_if_capable_max_pages(conn_,c);

-    return &Config::get_writable();
+    return &c;
  }
 }
@@ -107,12 +107,13 @@ namespace l
             void           *data_,
             uint32_t       *out_bufsz_)
  {
-    FileInfo *fi = reinterpret_cast<FileInfo*>(ffi_->fh);
+    FileInfo           *fi     = reinterpret_cast<FileInfo*>(ffi_->fh);
+    const fuse_context *fc     = fuse_get_context();
+    const ugid::Set     ugid(fc->uid,fc->gid);

    return l::ioctl(fi->fd,cmd_,data_,out_bufsz_);
  }

-
 #ifndef O_NOATIME
 #define O_NOATIME 0
 #endif
@@ -44,14 +44,19 @@ namespace l
    string xattrs;
    const vector<string> strs =
      buildvector<string>
+      ("user.mergerfs.async_read")
      ("user.mergerfs.branches")
      ("user.mergerfs.cache.attr")
      ("user.mergerfs.cache.entry")
+      ("user.mergerfs.cache.files")
      ("user.mergerfs.cache.negative_entry")
      ("user.mergerfs.cache.open")
+      ("user.mergerfs.cache.readdir")
      ("user.mergerfs.cache.statfs")
+      ("user.mergerfs.cache.symlinks")
      ("user.mergerfs.direct_io")
      ("user.mergerfs.dropcacheonclose")
+      ("user.mergerfs.fuse_msg_size")
      ("user.mergerfs.ignorepponrename")
      ("user.mergerfs.link_cow")
      ("user.mergerfs.minfreespace")
@@ -31,6 +31,7 @@

 using std::string;
 using std::vector;
+typedef Config::CacheFiles CacheFiles;

 namespace l
 {
@@ -92,7 +93,35 @@ namespace FUSE
    const ugid::Set          ugid(fc->uid,fc->gid);
    const rwlock::ReadGuard  readlock(&config.branches_lock);

-    ffi_->direct_io = config.direct_io;
+    switch(config.cache_files)
+      {
+      case CacheFiles::LIBFUSE:
+        ffi_->direct_io  = config.direct_io;
+        ffi_->keep_cache = config.kernel_cache;
+        ffi_->auto_cache = config.auto_cache;
+        break;
+      case CacheFiles::OFF:
+        ffi_->direct_io  = 1;
+        ffi_->keep_cache = 0;
+        ffi_->auto_cache = 0;
+        break;
+      case CacheFiles::PARTIAL:
+        ffi_->direct_io  = 0;
+        ffi_->keep_cache = 0;
+        ffi_->auto_cache = 0;
+        break;
+      case CacheFiles::FULL:
+        ffi_->direct_io  = 0;
+        ffi_->keep_cache = 1;
+        ffi_->auto_cache = 0;
+        break;
+      case CacheFiles::AUTO_FULL:
+        ffi_->direct_io  = 0;
+        ffi_->keep_cache = 0;
+        ffi_->auto_cache = 1;
+        break;
+      }
+
    return l::open(config.open,
                   config.open_cache,
                   config.branches,
@@ -14,6 +14,7 @@
  OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
 */

+#include "config.hpp"
 #include "dirinfo.hpp"

 #include <fuse.h>
@@ -24,8 +25,16 @@ namespace FUSE
  opendir(const char     *fusepath_,
          fuse_file_info *ffi_)
  {
+    const Config &config = Config::get();
+
    ffi_->fh = reinterpret_cast<uint64_t>(new DirInfo(fusepath_));

+    if(config.cache_readdir)
+      {
+        ffi_->keep_cache    = 1;
+        ffi_->cache_readdir = 1;
+      }
+
    return 0;
  }
 }
@@ -26,8 +26,7 @@ namespace FUSE
 {
  int
  prepare_hide(const char *fusepath_,
-               uint64_t   *fh_,
-               int         type_)
+               uint64_t   *fh_)
  {
    int rv;
    struct fuse_file_info ffi = {0};
@@ -24,6 +24,5 @@ namespace FUSE
 {
  int
  prepare_hide(const char *name_,
-               uint64_t   *fh_,
-               int         type_);
+               uint64_t   *fh_);
 }
@@ -178,7 +178,7 @@ namespace l
  int
  setxattr_xattr(const string &attrval_,
                 const int     flags_,
-                 int           xattr_)
+                 int          &xattr_)
  {
    if((flags_ & XATTR_CREATE) == XATTR_CREATE)
      return -EEXIST;
@@ -235,6 +235,26 @@ namespace l
    return 0;
  }

+  static
+  int
+  setxattr(const string       &attrval_,
+           const int           flags_,
+           Config::CacheFiles &cache_files_)
+  {
+    Config::CacheFiles tmp;
+
+    if((flags_ & XATTR_CREATE) == XATTR_CREATE)
+      return -EEXIST;
+
+    tmp = attrval_;
+    if(!tmp.valid())
+      return -EINVAL;
+
+    cache_files_ = tmp;
+
+    return 0;
+  }
+
  static
  int
  setxattr_controlfile_func_policy(Config       &config,
@@ -436,6 +456,10 @@ namespace l
          return l::setxattr_controlfile_cache_entry(attrval,flags);
        else if((attr[2] == "cache") && (attr[3] == "negative_entry"))
          return l::setxattr_controlfile_cache_negative_entry(attrval,flags);
+        else if((attr[2] == "cache") && (attr[3] == "readdir"))
+          return l::setxattr_bool(attrval,flags,config.cache_readdir);
+        else if((attr[2] == "cache") && (attr[3] == "files"))
+          return l::setxattr(attrval,flags,config.cache_files);
        break;

      default:
@@ -22,6 +22,7 @@
 #include "fuse_access.hpp"
 #include "fuse_chmod.hpp"
 #include "fuse_chown.hpp"
+#include "fuse_copy_file_range.hpp"
 #include "fuse_create.hpp"
 #include "fuse_destroy.hpp"
 #include "fuse_fallocate.hpp"
@@ -82,64 +83,56 @@ namespace l
    ops.flag_nopath        = true;
    ops.flag_utime_omit_ok = true;

-    ops.prepare_hide = FUSE::prepare_hide;
-    ops.free_hide    = FUSE::free_hide;
-    ops.fchown       = FUSE::fchown;
-    ops.fchmod       = FUSE::fchmod;
-    ops.futimens     = FUSE::futimens;
-
-    ops.access      = FUSE::access;
-    ops.bmap        = NULL;
-    ops.chmod       = FUSE::chmod;
-    ops.chown       = FUSE::chown;
-    ops.create      = FUSE::create;
-    ops.destroy     = FUSE::destroy;
-    ops.fallocate   = FUSE::fallocate;
-    ops.fgetattr    = FUSE::fgetattr;
-    ops.flock       = NULL; // FUSE::flock;
-    ops.flush       = FUSE::flush;
-    ops.fsync       = FUSE::fsync;
-    ops.fsyncdir    = FUSE::fsyncdir;
-    ops.ftruncate   = FUSE::ftruncate;
-    ops.getattr     = FUSE::getattr;
-    ops.getdir      = NULL;       /* deprecated; use readdir */
-    ops.getxattr    = FUSE::getxattr;
-    ops.init        = FUSE::init;
-    ops.ioctl       = FUSE::ioctl;
-    ops.link        = FUSE::link;
-    ops.listxattr   = FUSE::listxattr;
-    ops.lock        = NULL;
-    ops.mkdir       = FUSE::mkdir;
-    ops.mknod       = FUSE::mknod;
-    ops.open        = FUSE::open;
-    ops.opendir     = FUSE::opendir;
-    ops.poll        = NULL;
-    ops.read        = (nullrw ?
-                       FUSE::read_null :
-                       FUSE::read);
-    ops.read_buf    = (nullrw ?
-                       NULL :
-                       FUSE::read_buf);
-    ops.readdir     = FUSE::readdir;
-    ops.readlink    = FUSE::readlink;
-    ops.release     = FUSE::release;
-    ops.releasedir  = FUSE::releasedir;
-    ops.removexattr = FUSE::removexattr;
-    ops.rename      = FUSE::rename;
-    ops.rmdir       = FUSE::rmdir;
-    ops.setxattr    = FUSE::setxattr;
-    ops.statfs      = FUSE::statfs;
-    ops.symlink     = FUSE::symlink;
-    ops.truncate    = FUSE::truncate;
-    ops.unlink      = FUSE::unlink;
-    ops.utime       = NULL;       /* deprecated; use utimens() */
-    ops.utimens     = FUSE::utimens;
-    ops.write       = (nullrw ?
-                       FUSE::write_null :
-                       FUSE::write);
-    ops.write_buf   = (nullrw ?
-                       FUSE::write_buf_null :
-                       FUSE::write_buf);
+    ops.access          = FUSE::access;
+    ops.bmap            = NULL;
+    ops.chmod           = FUSE::chmod;
+    ops.chown           = FUSE::chown;
+    ops.copy_file_range = FUSE::copy_file_range;
+    ops.create          = FUSE::create;
+    ops.destroy         = FUSE::destroy;
+    ops.fallocate       = FUSE::fallocate;
+    ops.fchmod          = FUSE::fchmod;
+    ops.fchown          = FUSE::fchown;
+    ops.fgetattr        = FUSE::fgetattr;
+    ops.flock           = NULL; // FUSE::flock;
+    ops.flush           = FUSE::flush;
+    ops.free_hide       = FUSE::free_hide;
+    ops.fsync           = FUSE::fsync;
+    ops.fsyncdir        = FUSE::fsyncdir;
+    ops.ftruncate       = FUSE::ftruncate;
+    ops.futimens        = FUSE::futimens;
+    ops.getattr         = FUSE::getattr;
+    ops.getdir          = NULL; /* deprecated; use readdir */
+    ops.getxattr        = FUSE::getxattr;
+    ops.init            = FUSE::init;
+    ops.ioctl           = FUSE::ioctl;
+    ops.link            = FUSE::link;
+    ops.listxattr       = FUSE::listxattr;
+    ops.lock            = NULL;
+    ops.mkdir           = FUSE::mkdir;
+    ops.mknod           = FUSE::mknod;
+    ops.open            = FUSE::open;
+    ops.opendir         = FUSE::opendir;
+    ops.poll            = NULL;
+    ops.prepare_hide    = FUSE::prepare_hide;
+    ops.read            = (nullrw ? FUSE::read_null : FUSE::read);
+    ops.read_buf        = (nullrw ? NULL : FUSE::read_buf);
+    ops.readdir         = FUSE::readdir;
+    ops.readlink        = FUSE::readlink;
+    ops.release         = FUSE::release;
+    ops.releasedir      = FUSE::releasedir;
+    ops.removexattr     = FUSE::removexattr;
+    ops.rename          = FUSE::rename;
+    ops.rmdir           = FUSE::rmdir;
+    ops.setxattr        = FUSE::setxattr;
+    ops.statfs          = FUSE::statfs;
+    ops.symlink         = FUSE::symlink;
+    ops.truncate        = FUSE::truncate;
+    ops.unlink          = FUSE::unlink;
+    ops.utime           = NULL; /* deprecated; use utimens() */
+    ops.utimens         = FUSE::utimens;
+    ops.write           = (nullrw ? FUSE::write_null : FUSE::write);
+    ops.write_buf       = (nullrw ? FUSE::write_buf_null : FUSE::write_buf);

    return;
  }
@@ -99,11 +99,31 @@ static
 void
 set_default_options(fuse_args *args)
 {
-  set_option(args,"atomic_o_trunc");
-  set_option(args,"big_writes");
  set_option(args,"default_permissions");
 }

+static
+int
+parse_and_process(const std::string &value_,
+                  uint16_t          &uint16_,
+                  uint16_t           min_,
+                  uint16_t           max_)
+{
+  int rv;
+  uint64_t uint64;
+
+  rv = num::to_uint64_t(value_,uint64);
+  if(rv == -1)
+    return 1;
+
+  if((uint64 > max_) || (uint64 < min_))
+    return 1;
+
+  uint16_ = uint64;
+
+  return 0;
+}
+
 static
 int
 parse_and_process(const std::string &value_,
@@ -134,13 +154,13 @@ parse_and_process(const std::string &value,

 static
 int
-parse_and_process(const std::string &value,
-                  bool              &boolean)
+parse_and_process(const std::string &value_,
+                  bool              &boolean_)
 {
-  if(value == "false")
-    boolean = false;
-  else if(value == "true")
-    boolean = true;
+  if((value_ == "false") || (value_ == "0") || (value_ == "off"))
+    boolean_ = false;
+  else if((value_ == "true") || (value_ == "1") || (value_ == "on"))
+    boolean_ = true;
  else
    return 1;

@@ -157,6 +177,22 @@ parse_and_process(const std::string &value_,
  return 0;
 }

+static
+int
+parse_and_process(const std::string  &value_,
+                  Config::CacheFiles &cache_files_)
+{
+  Config::CacheFiles tmp;
+
+  tmp = value_;
+  if(!tmp.valid())
+    return 1;
+
+  cache_files_ = tmp;
+
+  return 0;
+}
+
 static
 int
 parse_and_process_errno(const std::string &value_,
@@ -239,6 +275,12 @@ parse_and_process_cache(Config       &config_,
    return (set_kv_option(outargs,"negative_timeout",value_),0);
  else if(func_ == "attr")
    return (set_kv_option(outargs,"attr_timeout",value_),0);
+  else if(func_ == "symlinks")
+    return parse_and_process(value_,config_.cache_symlinks);
+  else if(func_ == "readdir")
+    return parse_and_process(value_,config_.cache_readdir);
+  else if(func_ == "files")
+    return parse_and_process(value_,config_.cache_files);

  return 1;
 }
@@ -250,8 +292,22 @@ parse_and_process_arg(Config            &config,
 {
  if(arg == "defaults")
    return 0;
+  else if(arg == "hard_remove")
+    return 0;
  else if(arg == "direct_io")
    return (config.direct_io=true,0);
+  else if(arg == "kernel_cache")
+    return (config.kernel_cache=true,0);
+  else if(arg == "auto_cache")
+    return (config.auto_cache=true,0);
+  else if(arg == "async_read")
+    return (config.async_read=true,0);
+  else if(arg == "sync_read")
+    return (config.async_read=false,0);
+  else if(arg == "atomic_o_trunc")
+    return 0;
+  else if(arg == "big_writes")
+    return 0;

  return 1;
 }
@@ -307,6 +363,20 @@ parse_and_process_kv_arg(Config            &config,
        rv = parse_and_process(value,config.fsname);
      else if(key == "posix_acl")
        rv = parse_and_process(value,config.posix_acl);
+      else if(key == "direct_io")
+        rv = parse_and_process(value,config.direct_io);
+      else if(key == "kernel_cache")
+        rv = parse_and_process(value,config.kernel_cache);
+      else if(key == "auto_cache")
+        rv = parse_and_process(value,config.auto_cache);
+      else if(key == "async_read")
+        rv = parse_and_process(value,config.async_read);
+      else if(key == "max_write")
+        rv = 0;
+      else if(key == "fuse_msg_size")
+        rv = parse_and_process(value,config.fuse_msg_size,
+                               1,
+                               FUSE_MAX_MAX_PAGES);
    }

  if(rv == -1)
@@ -368,14 +438,14 @@ void
 usage(void)
 {
  std::cout <<
-    "Usage: mergerfs [options] <srcpaths> <destpath>\n"
+    "Usage: mergerfs [options] <branches> <destpath>\n"
    "\n"
    "    -o [opt,...]           mount options\n"
    "    -h --help              print help\n"
    "    -v --version           print version\n"
    "\n"
    "mergerfs options:\n"
-    "    <srcpaths>             ':' delimited list of directories. Supports\n"
+    "    <branches>             ':' delimited list of directories. Supports\n"
    "                           shell globbing (must be escaped in shell)\n"
    "    -o func.<f>=<p>        Set function <f> to policy <p>\n"
    "    -o category.<c>=<p>    Set functions in category <c> to <p>\n"
@@ -383,38 +453,49 @@ usage(void)
    "                           default = 0 (disabled)\n"
    "    -o cache.statfs=<int>  'statfs' cache timeout in seconds. Used by\n"
    "                           policies. default = 0 (disabled)\n"
-    "    -o cache.attr=<int>    file attribute cache timeout in seconds.\n"
+    "    -o cache.files=libfuse|off|partial|full|auto-full\n"
+    "                           * libfuse: Use direct_io, kernel_cache, auto_cache\n"
+    "                             values directly\n"
+    "                           * off: Disable page caching\n"
+    "                           * partial: Clear page cache on file open\n"
+    "                           * full: Keep cache on file open\n"
+    "                           * auto-full: Keep cache if mtime & size not changed\n"
+    "                           default = libfuse\n"
+    "    -o cache.symlinks=<bool>\n"
+    "                           Enable kernel caching of symlinks (if supported)\n"
+    "                           default = false\n"
+    "    -o cache.readdir=<bool>\n"
+    "                           Enable kernel caching readdir (if supported)\n"
+    "                           default = false\n"
+    "    -o cache.attr=<int>    File attribute cache timeout in seconds.\n"
    "                           default = 1\n"
-    "    -o cache.entry=<int>   file name lookup cache timeout in seconds.\n"
+    "    -o cache.entry=<int>   File name lookup cache timeout in seconds.\n"
    "                           default = 1\n"
    "    -o cache.negative_entry=<int>\n"
-    "                           negative file name lookup cache timeout in\n"
+    "                           Negative file name lookup cache timeout in\n"
    "                           seconds. default = 0\n"
-    "    -o direct_io           Bypass page caching, may increase write\n"
-    "                           speeds at the cost of reads. Please read docs\n"
-    "                           for more details as there are tradeoffs.\n"
    "    -o use_ino             Have mergerfs generate inode values rather than\n"
    "                           autogenerated by libfuse. Suggested.\n"
-    "    -o minfreespace=<int>  minimum free space needed for certain policies.\n"
+    "    -o minfreespace=<int>  Minimum free space needed for certain policies.\n"
    "                           default = 4G\n"
    "    -o moveonenospc=<bool> Try to move file to another drive when ENOSPC\n"
    "                           on write. default = false\n"
    "    -o dropcacheonclose=<bool>\n"
    "                           When a file is closed suggest to OS it drop\n"
-    "                           the file's cache. This is useful when direct_io\n"
-    "                           is disabled. default = false\n"
+    "                           the file's cache. This is useful when using\n"
+    "                           'cache.files'. default = false\n"
    "    -o symlinkify=<bool>   Read-only files, after a timeout, will be turned\n"
    "                           into symlinks. Read docs for limitations and\n"
    "                           possible issues. default = false\n"
    "    -o symlinkify_timeout=<int>\n"
-    "                           timeout in seconds before will turn to symlinks.\n"
+    "                           Timeout in seconds before files turn to symlinks.\n"
    "                           default = 3600\n"
    "    -o nullrw=<bool>       Disables reads and writes. For benchmarking.\n"
    "                           default = false\n"
    "    -o ignorepponrename=<bool>\n"
    "                           Ignore path preserving when performing renames\n"
    "                           and links. default = false\n"
-    "    -o link_cow=<bool>     delink/clone file on open to simulate CoW.\n"
+    "    -o link_cow=<bool>     Delink/clone file on open to simulate CoW.\n"
    "                           default = false\n"
    "    -o security_capability=<bool>\n"
    "                           When disabled return ENOATTR when the xattr\n"
@@ -435,7 +516,11 @@ usage(void)
    "                           as 'read only' or 'no create'. 'nc' will ignore\n"
    "                           available space for branches tagged as\n"
    "                           'no create'. default = none\n"
-    "    -o posix_acl=<bool>    enable POSIX ACL support\n"
+    "    -o posix_acl=<bool>    Enable POSIX ACL support. default = false\n"
+    "    -o async_read=<bool>   If disabled or unavailable the kernel will\n"
+    "                           ensure there is at most one pending read \n"
+    "                           request per file and will attempt to order\n"
+    "                           requests by offset. default = true\n"
            << std::endl;
 }

@@ -38,11 +38,15 @@ PolicyCache::Value::Value()
 PolicyCache::PolicyCache(void)
  : timeout(DEFAULT_TIMEOUT)
 {
+  pthread_mutex_init(&_lock,NULL);
 }

 void
 PolicyCache::erase(const char *fusepath_)
 {
+  if(timeout == 0)
+    return;
+
  pthread_mutex_lock(&_lock);

  _cache.erase(fusepath_);
@@ -56,6 +60,9 @@ PolicyCache::cleanup(const int prob_)
  uint64_t now;
  map<string,Value>::iterator i;

+  if(timeout == 0)
+    return;
+
  if(rand() % prob_)
    return;

@@ -25,17 +25,20 @@ using std::string;
 using std::vector;

 int
-Policy::Func::eprand(const Category::Enum::Type  type,
+Policy::Func::eprand(const Category::Enum::Type  type_,
                     const Branches             &branches_,
-                     const char                 *fusepath,
-                     const uint64_t              minfreespace,
-                     vector<const string*>      &paths)
+                     const char                 *fusepath_,
+                     const uint64_t              minfreespace_,
+                     vector<const string*>      &paths_)
 {
  int rv;

-  rv = Policy::Func::epall(type,branches_,fusepath,minfreespace,paths);
+  rv = Policy::Func::epall(type_,branches_,fusepath_,minfreespace_,paths_);
  if(rv == 0)
-    std::random_shuffle(paths.begin(),paths.end());
+    {
+      std::random_shuffle(paths_.begin(),paths_.end());
+      paths_.resize(1);
+    }

  return rv;
 }
@@ -25,17 +25,20 @@ using std::string;
 using std::vector;

 int
-Policy::Func::rand(const Category::Enum::Type  type,
+Policy::Func::rand(const Category::Enum::Type  type_,
                   const Branches             &branches_,
-                   const char                 *fusepath,
-                   const uint64_t              minfreespace,
-                   vector<const string*>      &paths)
+                   const char                 *fusepath_,
+                   const uint64_t              minfreespace_,
+                   vector<const string*>      &paths_)
 {
  int rv;

-  rv = Policy::Func::all(type,branches_,fusepath,minfreespace,paths);
+  rv = Policy::Func::all(type_,branches_,fusepath_,minfreespace_,paths_);
  if(rv == 0)
-    std::random_shuffle(paths.begin(),paths.end());
+    {
+      std::random_shuffle(paths_.begin(),paths_.end());
+      paths_.resize(1);
+    }

  return rv;
 }
Author	SHA1	Message	Date
trapexitandGitHub	0b0f0a1b9e	Merge pull request #663 from trapexit/random only return 1 branch for rand/eprand policies	2019-09-07 17:27:56 -04:00
Antonio SJ Musumeci	0fffabfbe3	only return 1 branch for rand/eprand policies	2019-09-07 17:25:06 -04:00
trapexitandGitHub	6b3f703ba9	Merge pull request #662 from trapexit/unlink initialize mutex to fix lockup	2019-09-07 15:14:10 -04:00
Antonio SJ Musumeci	50ad648ce4	initialize mutex to fix lockup	2019-09-07 14:10:36 -04:00
trapexitandGitHub	2dbcd2b07a	Merge pull request #660 from trapexit/docs add UPGRADE section	2019-09-03 21:43:47 -04:00
Antonio SJ Musumeci	752a15953a	add UPGRADE section	2019-09-03 21:43:18 -04:00
trapexitandGitHub	23d365f67b	Merge pull request #659 from zofrex/relink-mount-mergerfs Force symlink creation (don't error if it already exists)	2019-08-31 13:46:13 -04:00
James Sanderson	f77d1d9929	Force symlink creation (don't error if it already exists)	2019-08-31 18:41:54 +01:00
trapexitandGitHub	4204c807bc	Merge pull request #657 from trapexit/docs update how it works	2019-08-20 20:16:03 -04:00
Antonio SJ Musumeci	e6a6ab967b	update how it works	2019-08-20 20:13:25 -04:00
trapexitandGitHub	52e4d60e7c	Merge pull request #655 from tnilanon/master Update README.md	2019-08-15 06:22:39 -04:00
Tanachat NilanonandGitHub	38919add61	Update README.md Fixed typo: changed ' to `	2019-08-14 23:57:56 -07:00
trapexitandGitHub	a7244184bd	Merge pull request #653 from xlotlu/patch-1 fix some grammar and typos	2019-08-01 08:06:52 -04:00
Ionuț CiocîrlanandGitHub	86bda61aa7	fix some grammar and typos	2019-08-01 14:35:00 +03:00
trapexitandGitHub	10e5829b8e	Merge pull request #643 from trapexit/ioctl_setuid set uid & gid when calling ioctl	2019-06-17 17:09:16 -04:00
Antonio SJ Musumeci	4c4c27a93e	set uid & gid when calling ioctl Turns out certain ioctl calls check the effective user id (FS_IOC_SETFLAGS).	2019-06-17 13:37:15 -04:00
trapexitandGitHub	72ac1fc0d1	Merge pull request #639 from trapexit/args accept old arguments for backwards compatibility	2019-06-10 20:24:36 -04:00
Antonio SJ Musumeci	5ca928e436	accept old arguments for backwards compatibility	2019-06-10 20:20:41 -04:00
trapexitandGitHub	5d013f4a18	Merge pull request #636 from trapexit/max_pages allow setting of 'max_pages' (via 'fuse_msg_size')	2019-06-03 21:16:53 -04:00
Antonio SJ Musumeci	7cbd88ac81	allow setting of 'max_pages' (via 'fuse_msg_size') Linux 4.20 and above allow setting the number of pages per FUSE message upto 256 (4K * 256 = 1MiB). This can greatly increase read and write speeds depending on the workload.	2019-06-03 21:01:19 -04:00
trapexitandGitHub	24af276542	Merge pull request #635 from trapexit/copy_file_range add copy_file_range support	2019-06-03 08:31:22 -04:00
Antonio SJ Musumeci	8cb7195c3e	add copy_file_range support	2019-06-03 07:12:05 -04:00
trapexitandGitHub	c8e178bf71	Merge pull request #633 from trapexit/cache.files add file caching across opens and runtime control	2019-05-30 00:28:28 -04:00
Antonio SJ Musumeci	529a953e30	add file caching across opens and runtime control	2019-05-30 00:10:08 -04:00
trapexitandGitHub	02b657edff	Merge pull request #632 from trapexit/async-read jmake async_read optional again	2019-05-28 17:10:17 -04:00
Antonio SJ Musumeci	ddf6a2f105	make async_read optional again	2019-05-28 15:57:50 -04:00
trapexitandGitHub	40a4762cac	Merge pull request #630 from trapexit/cache-dir add readdir caching	2019-05-23 22:26:19 -04:00
Antonio SJ Musumeci	2323c16316	add readdir caching	2019-05-23 22:11:33 -04:00
trapexitandGitHub	bac4b33ef8	Merge pull request #629 from trapexit/cache-symlink add symlink caching	2019-05-22 19:50:49 -04:00
Antonio SJ Musumeci	1baa706d37	add symlink caching	2019-05-22 19:25:38 -04:00
trapexitandGitHub	9e3331dab4	Merge pull request #628 from trapexit/hide-cleanup fix for unlink race condition	2019-05-21 23:09:13 -04:00
Antonio SJ Musumeci	61cded5b34	fix for unlink race condition	2019-05-21 22:57:02 -04:00