# S3FS Setup and Usage S3FS allows you to mount an S3-compatible server as a file system on your local machine using the FUSE-based S3FS utility. This section provides step-by-step instructions for setting up and using S3FS with your ZS3Server. {% embed url="" %} ### Step 1: Create the Password File 1. Create a `.passwd-s3fs` file containing your `ACCESS_KEY_ID` and `SECRET_ACCESS_KEY`: ```bash echo ACCESS_KEY_ID:SECRET_ACCESS_KEY > ${HOME}/.passwd-s3fs ``` Example: ```bash echo test1:rootroot > ${HOME}/.passwd-s3fs ``` 2. Update the file permissions for security: ```bash chmod 600 ${HOME}/.passwd-s3fs ``` ### Step 2: Create a Mount Point 1. Create a directory to use as the mount point: ```bash mkdir -p /mnt/s3fs ``` ### Step 3: Install Tools #### Option A: Using MinIO Client (mc) The MinIO Client allows you to interact with your MinIO server. The MinIO Server provides an S3-compatible API and supports core S3 features. Choose one of the following installation methods for Linux: **RPM (RHEL-based Systems)** * Download the latest stable MinIO RPM package. Install the package using `dnf`: ```bash wget https://dl.min.io/server/minio/release/linux-amd64/archive/minio-20241218131544.0.0-1.x86_64.rpm -O minio.rpm sudo dnf install minio.rpm ``` **DEB (Debian/Ubuntu Systems)** * Download the appropriate `.deb` package from the MinIO Downloads Page. Install the package using `dpkg`: ```bash wget https://dl.min.io/server/minio/release/linux-amd64/archive/minio_20241218131544.0.0_amd64.deb -O minio.deb sudo dpkg -i minio.deb ``` **Binary Installation (Universal Method)** * Download the MinIO Server binary. Make the binary executable and move the binary to a directory in your system PATH: ```bash wget https://dl.min.io/server/minio/release/linux-amd64/minio chmod +x minio sudo mv minio /usr/local/bin/ ``` Installing MinIO Client (mc) The MinIO Client allows command-line interaction with your MinIO server. 1. Download the MinIO Client binary: ```bash wget https://dl.min.io/client/mc/release/linux-amd64/mc ``` 2. Make the binary executable: ```bash chmod +x mc ``` 3. Move the binary to a directory in your system PATH: ```bash sudo mv mc /usr/local/bin/mc ``` 4. Verify the installation: ```bash mc --version ``` 5. Set an alias for your server: ```bash mc alias set zcn https:// --api S3v2 ``` * Replace `` with the domain link from the Blimp output. * Replace `` and `` with your credentials. Example: ```bash mc alias set zcn https://blimpibh5l.zus.network shahnawaz rootroot --api S3v2 ``` 6. Verify the alias setup by listing buckets: ```bash mc ls zcn ``` #### Option B: Using AWS CLI If you prefer AWS CLI over MinIO: **Install AWS CLI** Follow installation from [AWS CLI Docs](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html). **Set Environment Variables** ``` export AWS_ACCESS_KEY_ID=devuser export AWS_SECRET_ACCESS_KEY=Admin@123 ``` **Create a Bucket** ``` aws --endpoint-url https://blimpsogec.zus.network s3 mb s3://s3fs ``` ### Step 4: Create and List Buckets 1. Use MinIO Client to create a bucket: ```bash ./mc mb zcn/s3fs ``` 2. List the contents of the S3 server to verify: ```bash ./mc ls zcn ``` The bucket `s3fs` should now be visible. ### Step 5: Mount the Server 1. Mount the S3 bucket as a local file system: ```bash s3fs mybucket /path/to/mountpoint -o passwd_file=${HOME}/.passwd-s3fs -o url=https://url.to.s3/ -o use_path_request_style,allow_other,umask=000,complement_stat,nocopyapi,norenameapi ``` Replace the placeholders: * `mybucket` with the name of your bucket (`s3fs`). * `/path/to/mountpoint` with `/mnt/s3fs`. 2. Example using Blimp URL: ```bash s3fs s3fs /mnt/s3fs -o passwd_file=${HOME}/.passwd-s3fs -o url=https://blimp7fyct.zus.network -o use_path_request_style,allow_other,umask=000,complement_stat,nocopyapi,norenameapi ``` ### Step 6: Verify the Mount 1. Check the mounted file system: ```bash df -ha ``` 2. Navigate to the mount point: ```bash cd /mnt/s3fs ls ``` ### Step 7: Test the Setup 1. Create a file in the mounted directory: ```bash touch file1.txt ``` 2. List the contents to verify the file: ```bash ls ``` ### Step 8: Verify on Blimp 1. Open the Blimp dashboard. Navigate to the bucket associated with the mounted directory (`s3fs`). 2. Confirm that the file (`file1.txt`) and other contents of the mounted directory are visible on Blimp. ### Additional Examples #### File Operations Examples ```bash # Write a file echo "I use Züs!" > /mnt/s3fs/my-tools.txt # Copy a file cp ~/my-files/testfile /mnt/s3fs/ # Move a file mv ~/my-files/testfile /mnt/s3fs/ # Delete a file rm /mnt/s3fs/testfile ``` All file operations will reflect in your ZS3 bucket via Blimp. #### Docker Integration Docker Run Example ```bash docker run -v /mnt/s3fs/data/postgres:/var/lib/postgresql/data postgres:15 ``` Docker Compose Example ``` services: postgres: image: postgres:15 volumes: - /mnt/s3fs/data/postgres:/var/lib/postgresql/data ``` Additional Volume Mappings * Grafana Loki Logs: `/mnt/s3fs/loki` * Sharder Blocks: `/mnt/s3fs/sharder${SHARDER}/data/blocks` * Miner RocksDB: `/mnt/s3fs/miner${MINER}/data` #### Troubleshooting Check Mount ```bash df -ha | grep s3fs ``` Check Logs (Ubuntu) ```bash tail -f /var/log/syslog | grep s3fs ``` Enable Debug Logging ```bash s3fs s3fs /mnt/s3fs -o passwd_file=${HOME}/.passwd-s3fs -o dbglevel=info -f -o curldbg ``` Unmount ZS3 ```bash umount -l /mnt/s3fs ``` ### Additional Notes * The S3FS utility uses FUSE (Filesystem in Userspace) to mount S3 servers as a local file system. * The `s3fs` command includes options for security and compatibility: * `-o passwd_file=${HOME}/.passwd-s3fs`: Specifies the password file. * `-o url=https://url.to.s3/`: Points to the S3-compatible server URL. * `-o use_path_request_style,allow_other,umask=000,complement_stat`: Enables compatibility and allows other users to access the mounted file system. * **Compression**: If you are using compression, it's recommended to use the MinIO Client for efficient file operations.