Architecture and Data Management

Zus storage operates on a distributed and decentralized model, ensuring high availability, fault tolerance, and optimized performance.

This document outlines how writes, reads, data repair, rollback mechanisms, file size constraints, encryption, and performance-based selection work within the Zus storage ecosystem.

1. Write Process – Data +1 Consensus

To ensure data reliability and durability, Zus enforces a data+1 consensus requirement for every write operation:

• A write is only considered successful when all required data shards have been stored plus one additional blobber acknowledges the write.

• This extra redundancy enhances fault tolerance and ensures that even if a blobber fails during the write process, the data can still be reconstructed.

• This approach guarantees strong data consistency and durability while minimizing risks of partial writes.

2. Read Process – Data Shard Requirement

Zus follows an erasure coding model, where reads require retrieving only the minimum number of data shards needed to reconstruct a file:

• The exact number of required shards depends on the erasure coding scheme chosen by the user during allocation setup.

• Clients can tune redundancy and performance based on their desired balance between storage efficiency and fault tolerance.

• This approach optimizes bandwidth usage and download speed while ensuring that files remain accessible even if some blobbers go offline.

3. Automatic Data Repair – Self-Healing Mechanism

To maintain long-term data durability, Zus includes an automated repair mechanism:

• If parity blobbers go offline or become unreachable, the system automatically reconstructs lost data shards.

• The missing shards are redistributed to healthy blobbers, ensuring the allocation remains intact.

• This self-healing mechanism prevents permanent data loss, keeping the storage network highly available and resilient.

4. Rollback Mechanism – Handling Partial Commits

To prevent data inconsistency during failed writes, Zus employs an atomic rollback mechanism:

• If a write operation partially succeeds (i.e., some blobbers accept the data while others fail), the system reverts to the last known consistent state.

• This ensures strong consistency guarantees, preventing corruption, orphaned data, or incomplete writes.

• Users are assured that their stored data always reflects a coherent and valid state, regardless of network failures.

5. File Count and File Size Limits

Zus offers flexibility in terms of file storage:

• There are no hard limits on the number of files within an allocation.

• The maximum file size is capped by the allocation size itself.

  • If an allocation is 1000 GB, a single file can be as large as 1000 GB, provided no other files consume space.

• This model enables large-scale data storage, allowing users to store and manage massive files efficiently.

6. Encryption and Secure File Sharing

Zus supports end-to-end encryption (E2EE) to protect data at rest and in transit:

• Clients can encrypt their data before upload, ensuring that even storage providers (blobbers) cannot access raw content.

• Proxy re-encryption technology allows users to securely share encrypted files without exposing their encryption keys.

• This ensures confidentiality and controlled access, even when files are being shared between multiple users.

7. Data Verification – Merkle Tree Proofs

To maintain data integrity and verifiability, Zus utilizes Merkle Tree-based validation:

• Each uploaded file is split into fixed-size chunks, with a Merkle Tree constructed over these chunks.

• Each blobber stores its respective Merkle Tree, allowing efficient proof-of-storage verification.

• During downloads, Merkle proofs are checked against the root hash stored on the blockchain:

  • Ensures no tampering or modification has occurred.

  • Guarantees that retrieved data matches the originally uploaded version.

• This cryptographic auditability enhances trust in storage integrity.

8. Performance-Based Blobber Selection

To optimize download performance, Zus dynamically selects the best-performing blobbers based on real-time response metrics:

• During initial download requests, the system tracks blobber response times.

• Clients prioritize faster-performing blobbers for subsequent read operations.

• This approach balances speed, redundancy, and reliability, ensuring users receive the best available performance.

9. Batch Processing of Write Markers

Each data commit generates a Write Marker, a cryptographic proof of successful storage:

• The Write Marker includes:

  • Client ID

  • Blobber ID

  • Allocation ID

  • Size of data committed

  • Allocation root hash (post-commit)

  • Timestamp and signature

• These Write Markers serve as proof-of-storage, ensuring blobbers are eligible for payment.

Write Marker Chain – Batch Submission and Verification

To optimize blockchain interactions, Zus employs a Write Marker Chain mechanism:

• Instead of submitting each Write Marker individually, blobbers batch multiple markers into a single Write Marker Chain.

• Each batch contains:

  • A sequence of write markers.

  • A cumulative root hash of the allocation.

  • A batched root hash, computed by hashing all root hashes in the batch.

• The blockchain verifies the integrity of the entire chain, ensuring:

  • Proof-of-storage validation for blobbers.

  • Reduced transaction costs by minimizing on-chain interactions.

  • Efficient cryptographic auditing, allowing individual verification of each write operation.

This batch submission approach significantly enhances scalability, performance, and economic efficiency for both users and storage providers.

10. Geolocation-Aware Allocation Strategy

Zus optimizes storage allocation based on geolocation diversity, ensuring:

• Global redundancy by distributing shards across multiple regions.

• Enhanced data availability, reducing risks of localized outages.

• Optimized performance, ensuring reads and writes are routed to the closest, best-performing blobbers.

For reads, Zus prioritizes selecting the best-performing blobbers based on latency, reliability, and response times and minimizing data retrieval time by choosing blobbers that optimize efficiency for the user’s location.

---