ZS3 Architecture and File Operations

ZS3 (Zus S3) is a gateway interface layer that allows users to interact with the Zus decentralized storage network using standard S3-compatible APIs. It translates traditional object storage operations into blockchain-compliant transactions on the Zus network. This simplifies decentralized file storage and enables familiar developer workflows using tools like AWS CLI, MinIO, or custom S3 clients.

Overview

ZS3 is designed to:

• Bridge Traditional & Decentralized Storage: By supporting S3 APIs, ZS3 allows users to interact with decentralized storage as if they were using traditional cloud storage like AWS S3.

• Enable Seamless User Experience: End-users and developers don’t need to learn new interfaces or protocols.

• Ensure Blockchain Accountability: All storage actions result in write markers that are committed to the blockchain, ensuring state integrity and traceability.

• Support Enterprise Admin Controls: Admins can control access, configure environments, and manage resource allocation.

• Optimize Performance: Built-in caching and batch processing minimize latency and blockchain load.

Core Components

1. Admin-Deployed ZS3 Server (VM-based)

• Deployed using provisioning tools like Blend on any cloud infrastructure (AWS, Azure, etc.).

• The admin connects via SSH and initializes the ZS3 node.

• The server runs both the MinIO Gateway and MinIO Server components.

• The admin controls access permissions, typically granting external users (teams or apps) access to ZS3 via S3 credentials.

• Each ZS3 instance is linked to one or more Zus allocations which map to actual decentralized storage across blobbers.

2. MinIO Gateway

• Exposes the S3-compatible REST API to clients.

• Clients use standard S3 commands (GET, PUT, POST, DELETE, etc.) to interact with storage.

• All S3 operations are received and passed along to the internal cache and GoSDK components.

• Serves as a compatibility layer between traditional cloud tooling and the Zus protocol.

3. ZS3 Cache Layer

• Acts as a temporary buffer for incoming file operations (uploads, deletions, metadata changes).

• Aggregates operations into logical batches to reduce frequent, small blockchain updates.

• Ensures that files are stored locally until they are validated and committed to the network.

• Prevents duplicate writes and reduces latency.

4. GoSDK Adapter (zS3 Adapter)

• Communicates with the Zus network using the Zus protocol.

• Fetches file operation batches from the cache.

• Converts them into valid updates to the Zus allocation, applying changes like file additions, deletions, etc.

• Generates a write marker and submits it to the blockchain to reflect the new state.

• Handles signature generation, validation, and error management.

5. Zus Allocation & Blockchain State

• A Zus allocation is a logical grouping of data stored on the decentralized network.

• Files are erasure-coded, distributed across multiple blobbers, and validated cryptographically.

• Each update to the allocation (e.g. file uploads) generates a write marker, which commits the change to the blockchain.

• Write markers ensure consensus, data integrity, and enable auditing.

---

Workflow: Uploading Files with ZS3

1. User Interaction via S3 Client

   • A user issues standard S3 commands (e.g., aws s3 cp, mc alias set, or SDK requests).

   • The request is sent to the ZS3 MinIO Gateway endpoint.

2. Request Processing

   • MinIO Gateway accepts the request and sends it to the ZS3 Cache.

   • The cache temporarily stores uploaded files and tracks them in-memory.

3. Batching & GoSDK Processing

   • Once multiple operations (e.g. uploads of multiple files or folders) are ready, the cache passes them as a single batch to the GoSDK.

   • GoSDK validates the batch, updates the corresponding Zus allocation with the new data.

4. Blockchain Commit via Write Marker

   • A new write marker is generated that includes the updated allocation root.

   • The marker is signed and submitted to the blockchain.

   • Only a single blockchain transaction is required, reducing cost and improving speed.

---

Docker Integration & Volume Support

ZS3 supports Docker-based file system extension using the zS3 FS Adapter, which enables:

• Docker Volume Mounting: ZS3 allocations can be mounted as volumes inside Docker containers.

• Automated Volume Expansion: Admins can use the S3 interface to dynamically attach new file systems or disks.

• ZFS Mount Management: Admins can configure the server to recognize and mount storage volumes as part of the S3-compatible backend.

• S3 FS Interface: Handles mounting/unmounting of volumes in Linux-based environments for containerized or server workloads.

---

Advantages of ZS3

Feature	Benefit
S3-Compatible APIs	Zero learning curve for developers familiar with AWS S3
Blockchain Commit Layer	Ensures decentralized, verifiable storage updates
Batch Write Markers	Reduces blockchain transaction volume and cost
Caching Layer	Improves speed and upload throughput
GoSDK Integration	Seamlessly bridges S3 commands with Zus network storage operations
Docker Volume Support	Mount Zus storage in Docker containers for dynamic workflows
Multi-tenant Admin Control	Admins manage access, provisioning, and updates for multiple users

---

Real-World Example: Uploading 10 Files in 5 Batches

• A user uploads 10 files, split across 5 S3 requests.

• ZS3 Cache temporarily holds the files locally.

• The GoSDK reads and batches the operations.

• A single write marker is created, referencing the new allocation root.

• One blockchain transaction is made, efficiently committing the update.

Summary

ZS3 is a scalable, efficient, and developer-friendly interface for interacting with decentralized storage via the Zus protocol. It retains the usability of traditional cloud APIs while providing the benefits of blockchain-based trust, redundancy, and security. Whether you're running enterprise apps or containerized systems, ZS3 enables seamless decentralized data operations with minimal complexity.