Quick Start

Usage
Help
Config file ↔ CLI parameter mapping
Option list
Examples
Scenario-based tuning
Auto mount

Usage

Mount


us3fs [global options] <bucket> <mountpoint>

<bucket> and <mountpoint> must be the last two arguments in order; otherwise, other options may not take effect.

Unmount


umount <mountpoint>

On Windows, press Ctrl+C in cmd.

Configure access permissions

By default, US3FS uses the current user’s permissions for the mount. If you want other users/groups to access the mountpoint, you can use:

-o allow_other: allow any user to access the filesystem
--uid=xxx: set default owner UID
--gid=xxx: set default owner GID
--mp_mask: mountpoint permission mask. Takes effect only when allow_other is enabled. Default 0000. Usage is similar to umask. For example, to set mountpoint permissions to 770, use -o allow_other --mp_mask 0007. Note: root ignores mp_mask set by non-root mounts.

You can get UID/GID via id. Example:


// Mount as ubuntu user, set default user/group to www
ubuntu:~$ id www
uid=1001(www) gid=1001(www) groups=1001(www)
 
ubuntu:~$ us3fs --uid=1001 --gid=1001 -o allow_other <bucket> <mountpoint>

mp_mask examples:

Mountpoint permission	mp_mask
rwxrwxrwx	0000
rwxrwx---	0007
rwx------	0077
rwxr—r—	0033
rwxrwxr—	0003
rwxr-xr-x	0022

If you see:


stderr:
/bin/fusermount: option allow_other only allowed if 'user_allow_other' is set in /etc/fuse.conf

Add user_allow_other to /etc/fuse.conf.

On Windows, only --uid and --gid are supported.

Mount as read-only

Mount with -o ro.

Not supported on Windows.

Enable logging

--level=info/debug/error: enable US3FS logs at a specific level
--debug_fuse: enable user-space fuse logs
- CentOS: /var/log/messages
- Ubuntu: /var/log/syslog
Mount with -f to run in the foreground and print logs to the screen.

Update

Run:


us3fs --update

The updated binary is placed under /bin/.

Not available on Windows.

Help

Use us3fs -h to view supported options.

Linux


us3fs - a single posix file system based on us3
USAGE
  us3fs [global options] bucket mountpoint
Version
  US3FS Version: v2.0.3
  Commit ID: 96bd92a
  Build: 2024-02-27:17:02:18
  Go Version: go1.17.13 linux/amd64
FUSE
  -o value                      Specify fuse/winfsp option
  --entry_timeout value         How long to cache dentry for inode for fuse. (default: 5m0s)
  --attr_timeout value          How long to cache inode attr for fuse (default: 5m0s)
  --disable_async_read          Disable all read (even read-ahead) operations asynchronously
  --wb                          Enable writeback mode, which is turned off by default
  --max_background value        Specify the max_background parameter of fuse kernel(>=7.13), currently fuse usespace supports up to 1024 (default: 64)
  --congestion_threshold value  Specify the congestion_threshold parameter of fuse kernel(>=7.13), currently fuse usespace supports up to 768 (default: 48)
  --async_dio                   Enable the async_dio parameter of fuse kernel, async_dio is disabled by default
  --keep_pagecache              Turn on pagecache, when the file is opened, it will be decided whether to update according to the modification time of the inode, so please pay attention to the attr_timeout and dcache_timeout parameters will have a certain impact on this
 
OS
  --dcache_timeout value       How long to cache dentry for us3fs (default: 5m0s)
  --retry value                Number of times to retry a failed I/O (default: 5)
  --parallel value             Number of parallel I/O thread (default: 32)
  --disable_remove             Disable remove op, such as unlink, rmdir, rename
  --debug                      Set debug level for fuse/winfsp
  --level value                Set log level: error/warn/info/debug (default: "info")
  --log_dir value              Set log dir
  --log_max_age value          Set log max age (default: 72h0m0s)
  --log_rotation_time value    Set log rotation time (default: 1h0m0s)
  --enable_load_dentries       enable auto init dentries in memory
  --cache_db value             specify cache db path, e.g.: dbtype:dbpath
  --local_write                write file to local and upload async
  --max_local_file_size value  specify local file max size (default: "32m")
  --read_after_write_finish    read operation will wait all write operation done
  --finish_write_when_release  all written data will be uploaded when release
  --readahead value            Readahead size. e.g.: 1m/1k/1  (default: "0")
  --max_cache_per_file value   Max cache per file when enable readahead. e.g.: 32m/64m/128m  (default: "1024m")
  --etag value                 Check etag for part. value is percent(0~100) (default: 50)
  --passwd value               specify access file (default: "/etc/us3fs/us3fs.yaml")
  --enable_md5                 Enalbe md5 in http header
  --uid value                  Specify default uid (default: 0)
  --gid value                  Specify default gid (default: 0)
  --mp_mask value              Specify mountpoint mask (default: 0)
  --disable_check_vdir         disable detection of virtual directories
  --update                     Update us3fs to /bin/us3fs
  -n                           Doesn't check access when mount us3fs
  -l                           Enable local cache for small file
  -p value                     Specify local cache location (default: "/tmp/us3fs/")
  --prefix value               Specify bucket prefix path
  --open_rename                Use rename api rather than copy and delete
  --gfl                        Enable get_file_list
  --direct_read                Enable cache bypass read
  --perf_dump value            How long to output the performance dump (default: 1h0m0s)
  --skip_ne_dir_lookup         Skip non-essential directory checking, such as files ending in ".log",".png",".jpg", etc.
  --storage_class value        Storage type, including "STANDARD", "IA" (default: "STANDARD")
  --enable_remote_cache
  --cache_dirs value           specify cache db path, e.g.: /tmp/read-cache1,/tmp/read-cache2
  --cache_size_limit value     specify cache size limit, unit is GB (default: 4)
  --master_addr value          master server addr
  --data_port value            if data_port is specified, then other clients will connect it to get chunk data (default: 0)
  --page_size value            (default: 1048576)
  --fuse_session_cnt value     (default: 0)
 
MISC
  --help, -h  show help
  -f          foreground

Windows


us3fs - a single posix file system based on us3
USAGE
  us3fs [global options] bucket mountpoint
Version
  US3FS Version: v1.6.8
  Commit ID: c87ec9c
  Build: 2022-08-17:11:01:05
  Go Version: go1.16.3 linux/amd64
 
WinFSP
  -o value                     Specify fuse/winfsp option
  --dir_info_timeout value     The expiration time of the directory information, in seconds (default: 5)
  --file_info_timeout value    File information expiration time, in seconds (default: 5)
  --volume_info_timeout value  Volume information expiration time, in seconds (default: 5)
  --case_insensitive           Is case sensitive
  --keep_filecache             keep filecache
 
OS
  --dcache_timeout value  How long to cache dentry for us3fs (default: 5m0s)
  --retry value           Number of times to retry a failed I/O (default: 5)
  --parallel value        Number of parallel I/O thread (default: 32)
  --debug                 Set debug level for fuse/winfsp
  --level value           Set log level: error/warn/info/debug (default: "info")
  --readahead value       Readahead size. e.g.: 1m/1k/1  (default: "0")
  --etag value            Check etag for part. value is percent(0~100) (default: 50)
  --passwd value          specify access file (default: "/etc/us3fs/us3fs.conf")
  --enable_md5            Enalbe md5 in http header
  --uid value             Specify default uid (default: -1)
  --gid value             Specify default gid (default: -1)
  --disable_check_vdir    disable detection of virtual directories
  --update                Update us3fs to /bin/us3fs
  -n                      Doesn't check access when mount us3fs
  -l                      Enable local cache for small file
  -p value                Specify local cache location (default: "/tmp/us3fs/")
  --prefix value          Specify bucket prefix path
  --gfl                   Enable get_file_list
  --direct_read           Enable cache bypass read
  --perf_dump value       How long to output the performance dump (default: 1h0m0s)
  --skip_ne_dir_lookup    Skip non-essential directory checking, such as files ending in ".log",".png",".jpg", etc.
  --storage_class value   Storage type, including "STANDARD", "IA" (default: "STANDARD")
MISC
  --help, -h  show help
  -f          foreground

The differences between Linux and Windows options are mainly in FUSE vs WinFsp.

Config file ↔ CLI parameter mapping

Some mount parameters can be configured in the config file. Mapping between CLI parameters and config file fields:

CLI parameter	Config field	Example in config
gfl	get_file_list	get_file_list: true
keep_pagecache	keep_pagecache	keep_pagecache: true
n	no_check	no_check: true
disable_check_vdir	disable_check_vdir	disable_check_vdir: true
async_dio	async_dio	async_dio: true
skip_ne_dir_lookup	skip_ne_dir_look	skip_ne_dir_look: true
l	enable_local	enable_local: true
wb	writeback	writeback: true
direct_read	direct_read	direct_read: true
enable_md5	enable_md5	enable_md5: true
debug	debug	debug: true
-o allow_other	allow_other	allow_other: true
enable_load_dentries	enable_load_dentries	enable_load_dentries: true
disable_async_read	disable_async_read	disable_async_read: true
retry	retry	retry: 66
parallel	parallel	parallel: 77
disable_remove	disable_remove	disable_remove: true
congestion_threshold	congestion_threshold	congestion_threshold: 88
max_background	max_background	max_background: 99
uid	uid	uid: 100
gid	gid	gid: 100
etag	etag	etag: 100
dcache_timeout	dcache_timeout	dcache_timeout: 2h
entry_timeout	entry_timeout	entry_timeout: 3h
attr_timeout	attr_timeout	attr_timeout: 4h
perf_dump	perf_dump	perf_dump: 5h
log_max_age	log_max_age	log_max_age: 6h
log_dir	log_dir	log_dir: /a/b/c
prefix	prefix	prefix: /a/b/c
level	level	level: debug
storage_class	storage_class	storage_class: STANDARD
p	local_path	local_path: /a/b/c
readahead	readahead	readahead: 8m
max_cache_per_file	max_cache_per_file	max_cache_per_file: 1024m
cache_db	cache_db	cache_db: leveldb:/data/us3fs_cachedb
local_write	local_write	local_write: true
max_local_file_size	max_local_file_size	max_local_file_size: 32M
finish_write_when_release	finish_write_when_release	finish_write_when_release: true
read_after_write_finish	read_after_write_finish	read_after_write_finish: true
enable_remote_cache	enable_remote_cache	enable_remote_cache: true
cache_dirs	cache_dirs	cache_dirs: /mnt/nvme01,/mnt/nvme02
cache_size_limit	cache_size_limit	cache_size_limit: 100
master_addr	master_addr	master_addr: <master_ip>:<master_port>
data_port	data_port	data_port: 3333
page_size	page_size	page_size: 8388608
fuse_session_cnt	fuse_session_cnt	fuse_session_cnt: 4

Example: configure mount parameters in config file

Edit /etc/us3fs/us3fs.yaml (create directory if needed) and add parameters as required to simplify mount commands:


access_key: ************************************
secret_key: ************************************
endpoint: ufile.cn-north-02.ucloud.cn
hosts: []

Option list

WinFsp

Option	Description
o	WinFsp `-o` option value
dir_info_timeout	Directory info cache timeout (default 5s)
file_info_timeout	File info cache timeout (default 5s)
volume_info_timeout	Volume info cache timeout (default 5s)
keep_filecache	Keep file cache

FUSE

Option	Description
o	FUSE `-o` option value
entry_timeout	How long to cache looked-up names in FUSE Default 5min
attr_timeout	How long to cache file/dir attributes in FUSE Default 5min
disable_async_read	Disable async read-ahead in FUSE kernel (async by default)
wb	Enable writeback mode. Overwrite/append are not supported
max_background	FUSE kernel `max_background` (kernel>=7.13). fuse userspace supports up to 1024 (default 64). Improves direct I/O parallelism
congestion_threshold	FUSE kernel `congestion_threshold` (kernel>=7.13). fuse userspace supports up to 768 (default 48). Triggers congestion control for parallel I/O
async_dio	Enable async direct I/O in FUSE kernel (disabled by default)
keep_pagecache	Keep VFS pagecache until inode modtime/size changes. Note: `entry_timeout` and `dcache_timeout` may delay observing changes

Common FUSE options (use with -o):

Option	Description
allow_other	Allow all users to access the filesystem (default off)
ro	Mount as read-only

Usage


-o option=value

OS (Object Storage)

Option	Description
dcache_timeout	Dentry cache TTL in US3FS Default 5min
retry	Retry count after request failure (default 5)
parallel	Number of parallel I/O threads (default 20)
debug	Set user-space fuse log level (default off)
level	US3FS log level (default Info)
readahead	Readahead size, e.g. 1m/1k/1 (default “0”)
etag	Percentage (0~100) of parts to check ETag for upload integrity (default 50)
passwd	Credential file path (default `/etc/us3fs/us3fs.yaml`; Windows: custom)
enable_md5	Add MD5 check to HTTP headers (default off)
uid	Default file owner user (default current user)
gid	Default file owner group (default current group)
disable_check_vdir	Disable virtual directory detection
update	Update US3FS; new binary path `/bin/us3fs`
n	Do not check bucket access on mount (useful if region has no list service)
l	Enable local cache for small files and upload asynchronously (see Small file scenario)
p	Local cache directory for small-file async upload
prefix	Bucket prefix to mount (default empty)
gfl	For endpoints without ListObjects API, use PrefixFileList API to simulate listing
direct_read	Bypass internal cache and read from US3 directly (may reduce performance for hot files; can reduce latency otherwise)
perf_dump	Output latency statistics periodically (default 1 hour)
skip_ne_dir_lookup	Skip non-essential directory checking for specific suffixes; ensure you don’t have directories ending with those suffixes
storage_class	Storage class for objects written to US3: `STANDARD`, `IA` (default `STANDARD`)
cache_db	Cache metadata on disk, e.g. `leveldb:/data/us3fs_cachedb`
finish_write_when_release	Upload written data when releasing FD (supports multiple flushes per FD)
read_after_write_finish	Reads wait until in-flight writes finish
local_write	Write to local filesystem then upload asynchronously
max_local_file_size	With `local_write`, max file size written locally (default 32M)
enable_remote_cache	Enable remote cache
cache_dirs	Local cache directories, comma-separated
cache_size_limit	Cache size limit (GB)
master_addr	Master server address
data_port	Port for chunk data
page_size	Page size
fuse_session_cnt	FUSE session count

MISC

Option	Description
help, h	Show help
f	Run in foreground and print output to stdout

On Windows, -f has no effect.

Examples

entry_timeout, attr_timeout, dcache_timeout

Setting dcache_timeout increases how long file/dir attributes stay valid in memory, improving UX. We recommend setting entry_timeout and attr_timeout to be smaller than dcache_timeout.

Note: After enabling caching, directory content may be temporarily inconsistent with the actual bucket state. This is enabled by default; set timeouts to 0s to disable.

Example: time to ls a directory containing 10,000 files


[root@10-9-120-211 ~]# us3fs --dcache_timeout=60s --entry_timeout=60s --attr_timeout=60s <your_bucket> <mountpoint>
[root@10-9-120-211 ~]# time ls -la <your_dir> | wc -l
10003

real    0m5.964s
user    0m0.033s
sys     0m0.232s
[root@10-9-120-211 ~]#
[root@10-9-120-211 ~]#
[root@10-9-120-211 ~]# time ls -la <your_dir> | wc -l
10003

real    0m0.872s
user    0m0.029s
sys     0m0.133s

disable_async_read

By default reads are asynchronous; synchronous reads perform worse.

Example:


[root@10-9-120-211 ~]#  us3fs --disable_async_read <your_bucket> <mountpoint>
[root@10-9-120-211 ~]# dd if=<your_file> of=/dev/null bs=4M count=10
10+0 records in
10+0 records out
41943040 bytes (42 MB, 40 MiB) copied, 10.2345 s, 4.1 MB/s

[root@10-9-120-211 ~]#  us3fs <your_bucket> <mountpoint>
[root@10-9-120-211 ~]# dd if=<your_file> of=/dev/null bs=4M count=10
10+0 records in
10+0 records out
41943040 bytes (42 MB, 40 MiB) copied, 0.685801 s, 61.2 MB/s

parallel

Increasing concurrency can improve read/write performance, but also increases system resource usage.

Example:


// Default parallelism is 20
[root@10-9-120-211 ~]# us3fs <your_bucket> <mountpoint>
[root@10-9-120-211 ~]# dd if=/dev/zero of=<your_file> bs=4M count=1024
1024+0 records in
1024+0 records out
4294967296 bytes (4.3 GB, 4.0 GiB) copied, 25.5351 s, 168 MB/s

// Set parallelism to 32
[root@10-9-120-211 ~]# us3fs --parallel=32 <your_bucket> <mountpoint>
[root@10-9-120-211 ~]# dd if=/dev/zero of=<your_file> bs=4M count=1024
1024+0 records in
1024+0 records out
4294967296 bytes (4.3 GB, 4.0 GiB) copied, 18.3614 s, 234 MB/s

readahead

Adjusting readahead has significant impact on sequential reads of large files. We recommend 16m~32m, but it increases memory usage; reduce if needed.

Example:


// Default readahead size is 16MB
[root@10-9-120-211 ~]# dd if=<your_file> of=/dev/null bs=4M count=1024
1024+0 records in
1024+0 records out
4294967296 bytes (4.3 GB, 4.0 GiB) copied, 60.0498 s, 71.5 MB/s
 
// Set readahead size to 32MB
[root@10-9-120-211 ~]# us3fs --readahead=32m <your_bucket> <mountpoint>
[root@10-9-120-211 ~]# dd if=<your_file> of=/dev/null bs=4M count=1024
1024+0 records in
1024+0 records out
4294967296 bytes (4.3 GB, 4.0 GiB) copied, 37.6013 s, 114 MB/s

Scenario-based tuning

parallel: parallel threads; affects CPU load. Recommended 20~40.
critical: enable local ETag validation on write; can increase CPU usage by ~50%.
readahead: readahead window. Recommended 16m~32m; increases memory usage.
keep_pagecache: keep data in VFS pagecache until file modtime/size changes.
wb: merges writes in pagecache and sends larger I/O (default 128K).
fuse_session_cnt: number of connections to the kernel FUSE module; can improve performance but increases CPU/memory usage.

Small file scenario

For large volumes of small files, you can enable local cache with -l. When enabled, US3FS will first upload existing files <=4MB under the cache directory to the bucket, preserving paths. Writes <=4MB are written to local cache first and uploaded asynchronously; if local write fails, it falls back to synchronous writes to US3.

Note: async uploads may fail on the backend; US3FS retries until success.

High read throughput scenario

cache_dirs: local disks for read cache. Prefer local NVMe. You can set multiple disks separated by ,. Avoid using the system disk.
cache_size_limit: cache size upper bound; LRU eviction triggers when reaching the limit.
page_size: when enabling local read cache, set page_size from 1048576 to 8388608.

Shared read cache

To share read cache between multiple mountpoints, start a master node:


us3fs run-master --listen_addr 192.168.0.10:6667

Then set other mountpoints’ master_addr to 192.168.0.10:6667. The mountpoints will heartbeat to the master and form a small read-cache cluster.

enable_remote_cache: when enabled on a mountpoint, other mountpoints may connect to it to read cached data when available.
data_port: the port exposed for remote access when enable_remote_cache is enabled.

Auto mount

The recommended best practice is to use systemctl so US3FS remounts after abnormal exits or machine reboots. Ensure your Linux distribution supports systemctl (Windows is not supported currently):

Ubuntu >= 15.04
CentOS/RHEL >= 7

After the system requirements are met, follow these steps:

Create the service unit

Create /etc/systemd/system/us3fs.service with the following content:


[Unit]
Description=US3FS (User Space FileSystem for US3)
Requires=network-online.target
After=network-online.target

[Service]
Type=forking
User=<user>
Group=<group>
Restart=always
RestartSec=10
ExecStart=/bin/us3fs --passwd=/etc/us3fs/us3fs.conf --keep_pagecache <your_bucket> <mountpoint>
ExecStop=/bin/umount <monutpoint>
[Install]
WantedBy=multi-user.target

User: the user that needs to access the mountpoint (e.g., root)
Group: the group that needs to access the mountpoint (e.g., root)
ExecStart: the mount command; customize as needed

Run systemctl daemon-reload to load the new unit.

Start/stop the service

Start: systemctl start us3fs.service
Stop: systemctl stop us3fs.service
Restart: systemctl restart us3fs.service
Status: systemctl status us3fs.service

Enable auto-start on boot

Run systemctl enable us3fs.service.

Quick Start

Usage

Configure access permissions

Mount as read-only

Enable logging

Update

Help

Linux

Windows

Config file ↔ CLI parameter mapping

Option list

WinFsp

FUSE

OS (Object Storage)

MISC

Examples

Scenario-based tuning

Performance-related parameters

Small file scenario

High read throughput scenario

Shared read cache

Auto mount