☁️
Zus Docs
  • About Züs
  • System
    • Providers and Services
      • Miner
      • Sharder
      • Blobber
      • Validator
      • Authorizer
      • Node Locator (0DNS)
    • Storage
      • Architecture and Data Management
      • Protocol
        • Allocations
        • Reference Objects
        • Challenges
        • Write Markers
          • Chain Hashing
          • Two Commit
        • Blobber Repair Protocol
      • ZS3 Server
        • Backup, Recovery and Replication
        • Encryption and Compression
        • S3FS Setup and Usage
        • Backup & Recovery with Restic on Blimp + ZS3 Server
        • Backup & Recovery with Veeam on Blimp + ZS3 Server
      • File Operations
        • Upload
        • Download
        • File Sharing
        • Partial Error Recovery
        • Streaming
        • Rent a Blobber
    • Smart Contracts
      • Storage S.C.
      • Miner S.C.
      • ZCN S.C.
      • ERC-20 S.C.s
      • Bridge Protocol
    • Blockchain & Consensus
      • Entities
    • User Authentication and Wallet Management System
      • OKTA Integration
      • Key Management System (KMS)
  • APIs
    • 0DNS API
    • JS API
    • Mobile API
  • CLIs
    • Storage CLI
      • Quickstart
      • Configuring the tool
    • Wallet CLI
      • Wallet Configuration
      • Quickstart
      • Configuring the tool
  • SDKs
    • Go SDK
      • GO SDK Microservices
    • JS SDK
  • Tokenomics
    • Staking
    • Reward & Penalty
  • ✨Züs Apps
    • 🗝️Vult
      • Getting Started
        • Web
        • Mobile
      • Vult AI
        • Batch Processing
        • Memory Retention
        • Technical Implementation
        • Architecture Overview
      • Login / Register
      • File Management Pages
      • File Sharing
      • Storage Management Dashboard
      • Storage Maintenance and Troubleshooting
      • Züs Subscription
      • Wallet Management
      • Refer a friend
      • Settings
    • 🏗️Blimp
      • Getting Started
      • Login / Register
      • Configure Storage
        • Create Standard Storage Allocation
        • Create Enterprise Allocation
        • Create S3 Server Allocation
        • Create Cloud Migration Allocation
        • Allocation Maintenance and Troubleshooting
      • File Management Pages
      • File Sharing
      • Manage Allocations
      • Upgrade Storage
      • Blimp Vault
      • Refer a friend
      • Settings
      • Launching ZS3 Server
      • Using CLI to backup files into Blimp + ZS3 Server
    • 🏠Chimney
      • Getting Started
      • Login / Register
      • Create New Deployment
      • Manage Your Deployments
      • Homepage
      • Staking Dashboard
      • Rank Dashboard
      • Monitor Dashboard
      • Stats Dashboard
      • Logs Dashboard
      • Wallet Dashboard
      • Operations on your Deployments
      • Restricted Blobbers
      • Settings
        • Manage Profile
        • Wallet Settings
        • Update Blobber Settings
        • Update Blobber Version
        • Refer a friend
        • Help
    • 🌐Atlus
      • Getting Started
      • Home page
      • Service Providers Page
      • Charts Page
        • Market Charts
        • Network Charts
        • Storage Charts
      • Blockchain Page
      • Server Map Page
      • Storage Explainer Page
      • Details Pages
        • Block Details Page
        • Transaction Details Page
        • Wallet Details Page
        • Miner Details Page
        • Sharder Details Page
        • Blobber Details Page
        • Validator Details Page
        • Authorizer Details Page
        • Allocation Details Page
      • Appendix: Common Components
    • ⚡Bolt
      • Getting Started
        • Web
        • Mobile
      • Login / Register
      • Sign In with external wallet
      • Staking Dashboard
      • Staking/Unstaking a provider
      • Claiming Rewards
      • Send/Receive ZCN tokens
      • Buy ZCN
      • Deposit/Withdraw ZCN tokens
      • Activity Dashboard
      • Refer a friend
      • Settings
  • Releases
    • Hardfork
Powered by GitBook
On this page
  • Mount your bucket to a filesystem directory using S3FS
  • Usage Examples
  • Troubleshooting and checking the logs
  • Note on using your S3FS directory as the storage directory for Postgres DBs (using Docker)
  • Integration with Docker Compose
  • Use Cases and Applications
  1. System
  2. Storage
  3. ZS3 Server

Use Case - FUSE-based S3FS Mountpoint

Use S3FS to interact with your bucket with filesystem commands.

Last updated 1 month ago

S3FS allows you to mount an S3-compatible object storage bucket as a local file system. This enables seamless access to Züs storage for applications such as Docker containers, WordPress, and analytics tools like Grafana.

Key Features:

  • Mounts S3 buckets as directories on the host system.

  • Supports configuration for access keys, permissions, and URL endpoint customization.

  • Provides compatibility for Docker volumes and applications.

After deploying your S3 server, you can integrate to be able to use your allocation as a Data Lake, with S3 support for seamless migration, and lower egress cost compared to other providers.

S3FS makes you operate files and directories in S3 bucket like a local file system, which mean that you can use it in pretty much every use case you can use a typical filesystem directory in (logging, database storage, regular file storage, mount directory for docker containers for data storage or logging).

Moreover, if you have an application that depends on filesystem, you can utilize the power of Züs Storage with zero code change.

Mount your bucket to a filesystem directory using S3FS

Here's a step-by-step guide on how you can apply this change:

  1. Register to

  2. Deploy zs3server as mentioned in . Once deployed, you can find your URL, access key and secret key in the "Use CLI" section in Blimp's S3 operations page, as shown in the image below.

Example of this command (with the credentials part in double square parts)

curl -X GET [[https://blimpsogec.zus.network]]/minioclient/?action=createBucket&[[accessKey=devyetii&secretAccessKey=Admin@123]]

In this example:

  • The URL: https://blimpsogec.zus.network

  • The Access Key: devyetii

  • The secret key: Admin@123

  1. Connect to the machine you'll be using to mount S3FS. This can be your local machine, or any other remote machine that you have. If it's a remote machine, then you can use SSH to connect to it as follows:

ssh <user>@<ip>
  1. Create a path to mount your S3 directory using the command below:

mkdir -p <directory_path>

You can create any mounting path. But make sure you remember it as we'll be using it in Step 7.

  1. Prepare you environment variables as follows:

export AWS_ACCESS_KEY_ID=<your-access-key-in-step2>
export AWS_SECRET_ACCESS_KEY=<your-secret-key-in-step2>  
  1. Create a bucket on your ZS3 server as follows:

aws --endpoint-url https://<zs3server-url-from-step2> s3 mb s3://<bucket-name>        
   sudo apt update
   sudo apt install s3fs
  1. Prepare the password file that you'll be using with S3FS to mount your bucket on the mount point. Run the following command to write the contents of your password file, which will be stored in $HOME/.passwd-s3fs

   echo <your-access-key-from-step2>:<your-secret-key-from-step2> > ${HOME}/.passwd-s3fs 
   chmod 600 ${HOME}/.passwd-s3fs

You should replace <your-access-key-from-step2> and <your-secret-key-from-step2> with the corresponding values.

  1. Mount your bucket to the created directory path using s3fs using the command below:

   s3fs <bucket-name> <mount-directory> -o passwd_file=${HOME}/.passwd-s3fs -o url=<your-zs3server-url-from-step2> -o  use_path_request_style,allow_other,umask=000,complement_stat,nocopyapi,norenameapi

You should replace the placeholders in the command as follows:

  • <bucket-name> should be replaced by the bucket name you used to create the bucket in step 6.

  • <mount-directory> should be replaced by the mount directory you created in step 4.

  • <your-zs3server-url-from-step2> should be replaced by the URL shown in step2.

You can use the command below to unmount the zus-storage from <directory_path>: umount -l <directory_path>

  1. Check if your bucket is mounted successfully to S3FS mount directory as follows:

df -ha | grep "s3fs"

This command displays the mounting path details.

If there is no output, recheck all the steps to ensure they were executed correctly.

  1. Once you mount the Zus storage, any created, copied, or deleted files or directories within the mount will be reflected on your allocation files in Blimp, and vice versa.

Usage Examples

After mounting your bucket to the S3FS mountpoint, you can perform any typical file operation on this mount directory, and it will automatically be reflected to your ZS3 bucket, hence your allocation data showed in Blimp.

For example, write a file and save it to the mount directory (assume it's /mnt/s3fs)

cat "I use Züs!" > /mnt/s3fs/my-tools.txt

To copy a file:

cp ~/my-files/an-awesome-file /mnt/s3fs/my-awesome-files

To move a file:

mv ~/my-files/an-awesome-file /mnt/s3fs/my-awesome-files

To delete a file:

rm /mnt/s3fs/my-awesome-files

Troubleshooting and checking the logs

If you encountered any issue working with your mountpoint (using Ubuntu), you can check syslog as follows:

tail -f /var/logs/syslog | grep s3fs

You can also enable debug logging on S3FS as follows:

s3fs mybucket /path/to/mountpoint -o passwd_file=${HOME}/.passwd-s3fs -o dbglevel=info -f -o curldbg

Note on using your S3FS directory as the storage directory for Postgres DBs (using Docker)

You can configure your Postgres DB deployment (using Docker) to use the mounted S3FS directory as storage directory for the DB.

If you're using docker run to deploy your Postgres DB, attach the S3FS directory as a mount directory to your deployed container as follows:

docker run [...some options...] -v <path-in-your-s3fs-mount-directory>:/var/lib/postgresql/data [...some other options...] postgres:15

If you're using docker-compose you can configure your Postgres DB service to use the mounted S3FS directory as storage directory for the DB as follows:

services:
    ....
    postgres:
        ....
        volumes:
            - <path-in-your-s3fs-mount-directory>:/var/lib/postgresql/data
        ....

In both examples, you should replace <path-in-your-s3fs-mount-directory> with some path in your S3FS mount directory, for example: /mnt/s3fs/data/postgres

Integration with Docker Compose

The mounted paths will be used in Docker Compose files to specify volumes for various components like logs, Grafana, Sharders, Miners, and RocksDB.

Paths:

  1. Sharder Blocks: /mnt/s3fs/sharder${SHARDER}/data/blocks

  2. Miner RocksDB: /mnt/s3fs/miner${MINER}/data

  3. Grafana Loki Logs: /mnt/s3fs/loki

Docker Compose Configuration

Sample configuration for Sharder:

volumes:
  - /mnt/s3fs/sharder${SHARDER}/data/blocks:/0chain/data/blocks

Sample configuration for Grafana Loki:

volumes:
  - /mnt/s3fs/loki:/mnt/s3fs/loki

Sample configuration for Miner:

volumes:
  - /mnt/s3fs/miner${MINER}/data:/0chain/data

Use Cases and Applications

Logs and Monitoring

  • Grafana Loki: Stores log data in /mnt/s3fs/loki. Use this for real-time log analytics.

  • Sharder Logs: Logs from sharder operations are stored in /mnt/s3fs/sharder${SHARDER}/log.

Data Storage for Load Testing

For load testing Züs components like Sharders, Miners, and Blobbers:

  • Configure RocksDB for Miners: /mnt/s3fs/miner${MINER}/data.

  • Configure Blocks for Sharders: /mnt/s3fs/sharder${SHARDER}/data/blocks.

  • Permissions: Always secure your .passwd-s3fs file to avoid exposing access credentials.

  • Mount Optimization: Use -o allow_other and -o use_path_request_style for compatibility with Docker containers.

  • Testing: Regularly verify mounted paths and data accessibility before starting containers.

  • Monitoring: Use Grafana Loki and other monitoring tools to track resource usage and logs.

aws-cli installation and usage is explained in details .

Install S3FS on your machine. Provided down below are commands to install in on an Ubuntu machine. If you have any other distro of Linux or another OS, you can refer to this

here
link
S3FS
https://blimp.software
this page
Fig1: URL, secret and access keys in Using CLI section