> For the complete documentation index, see [llms.txt](https://docs.zus.network/zus-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.zus.network/zus-docs/system-overview/providers-and-services/blobber.md).

# Blobber

<figure><img src="/files/Ji9MVYIKv90xWsy60Xmw" alt=""><figcaption><p>blobber docker-compose-based deployment example</p></figcaption></figure>

The blobber is the most important provider when it comes to hosting users' data. It provides actual storage space to host this data on its file system and gets rewards in return. Users can host their own blobber on premises or on our dedicated servers through the [Chimney](https://chimney.software/) Webapp.

### Core responsibilities of the Blobber

* Host users' data on its file system
* Maintain the correctness and availability of the users data on-demand.
* Provide a highly-available interface (currently an HTTP API) for the users to access or modify their data.
* Answer the challenges so it can get rewards

### Logical component of the Blobber's Process

The Sharder Process consists of multiple workers (technically, lightweight execution threads or [goroutines](https://go.dev/doc/effective_go#goroutines) in the Go literature). Following are the most important workers running the miner operations

#### Allocation Sync Worker

*An Allocation* is like a contract between the user and Züs network about some data the user need to host on the Network. One of the terms that the user choose during creating the allocation is the blobbers that will host their data based on this allocation. Since this coordination occurs between the user and the blockchain network (miners and sharders to simplify), the blobber knows nothing about the allocation unless it requests this data from the network using one of the endpoints of the Sharder API. This worker periodically polls this endpoint for updates and persists the information of the new allocations

#### Challenge Workers

*Challenges* is how the blobbers get time-based rewards for storage. In simple words, a challenge is a validation inquiry of a portion of the data that should be hosted by some blobber i.e. trying to check if the blobber is really hosting this portion of data or not. They're generated by the network using a special type of transactions called *Built-in Transactions* that are autogenerated by the network on each block generation. A challenge is generated:

* For a specific Blobber
* Asking about a specific Allocation
* And a specific *Block* of data in this allocation

Similar to Allocations, Challenges are generated and kept in the blockchain network and the blobber knows nothing about it until it requests this data from the network also using one of the endpoints of the [Sharder API](/zus-docs/http-apis.md#sharder-node-api). It then answers those challenges and send its response back to the network, which in turn validates the answer and either rewards or penalizes the blobber, based on the answer. These tasks: polling the network for new challenges, answering the challenges and sending the answers to the network, are the responsibility of these workers. More about challenges are to come in the [Challenges](/zus-docs/system-overview/storage/protocol/challenges.md)[ ](/zus-docs/system-overview/storage/protocol/challenges.md)page.

#### Read/Write Markers Workers

*Read Markers* are chique-like structures sent by the blobbers to the network in order to get fees of read operations from the client's wallet, same for *Write Markers* for write operations. Both will be discussed in more details in the [Read/Write Markers](/zus-docs/system-overview/storage/protocol/read-write-markers.md) page. Those workers handle the markers flow until the marker is fulfilled and redeemed.

#### Health Check Worker

A worker responsible for the heartbeats of the blobber which are signs of the blobber liveness. It works by generating a health check transaction periodically and communicating it with the network.

#### Update Settings Worker

The owner of the blobber - the user who created the blobber on Chimney - can update the settings of the blobber. However, they will not be communicating with the blobber directly, this settings update operation is submitted as a transaction to the network, thus unseen by the blobber unless it requests this data from the network using the [Sharder API](/zus-docs/http-apis.md#sharder-node-api). So the responsibility of this worker is to sync blobber settings with the network periodically.

### The Blobber HTTP API

The blobber exposes an HTTP API as an interface of the file operations the users need to execute on their data. It's discussed in more detail in the [Blobber API](/zus-docs/http-apis.md#blobber-api) page.

### Blobber Settings

These are the settings that the owners can change about their blobbers.

> Those settings can only be changed by the owner's wallet.

#### Capacity (Bytes)

This is the maximum capacity that the blobber offers to use for hosting users' data.

#### Read Price (SAS/GB/Time Unit)

> SAS is the smallest countable unit of the ZCN token. One ZCN is 10^10 SAS

This is how much the user pays for read operations against this blobber, it's calculates per GB per *Time Unit* where *Time Unit* is one of the storage configuration elements.

#### Write Price (SAS/GB/Time Unit)

This is how much the user pays for write operations against this blobber, it's calculated per GB per *Time Unit* where *Time Unit* is one of the storage configuration elements

#### Max Offer Duration (duration string e.g. 700h - 3m - ..)

This is how long the blobber can keep its prices without change. After this duration elapses, the blobber's prices change and all allocations need to be updated.

#### Min Stake (SAS)

This is the least amount the blobber can be staked with by a single client. Staking is discussed in more detail in the [Tokenomics](/zus-docs/tokenomics.md) page.

#### Max Stake (SAS)

This is the largest amount the blobber can be staked by a single client. Staking is discussed in more detail in the [Tokenomics](/zus-docs/tokenomics.md) page.

#### Max Number of Delegates (no unit)

This is the maximum number of clients that can be delegated to the blobber by staking. Delegation is discussed in more detail in the [Tokenomics](/zus-docs/tokenomics.md) page.

#### Service Charge (Decimal, 0 to 1)

This is the ratio the blobber receives from each storage reward it earns before the rest is distributed over its delegates.

#### URL (text)

The URL of the homepage of the blobber.

#### Not Available (boolean)

Whether the blobber is available for hosting users' data or not.

### Blobber Protocols

#### File Hash

There are 3 hashes for a file on Blobber:

* Actual File Hash

  > it is used to verify the checksum of downloaded file on clients. It is the hash of original file. It is `reference_objects.actual_file_hash` in database.
* Content Hash

  > it is used to verify the checksum of uploaded data blocks on blobber server. It is hash of data blocks that is sharded by `ErasureEncoder` on client and uploaded on a blobber. It is `reference_objects.content_hash` in database.
* Challenge Hash

  > it is used to verify challenge on validator based on challenge protocol. It is `reference_objects.merkle_root` in database.

#### Hash Method

* `Actual File Hash` is computed by sha256
* `Content Hash` is computed by [MerkleTree](https://en.wikipedia.org/wiki/Merkle_tree).

> A hash tree or Merkle tree is a tree in which every leaf node is labelled with the cryptographic hash of a data block, and every non-leaf node is labelled with the cryptographic hash of the labels of its child nodes.

For better performance on memory and persistence, we have a `CompactMerkleTree` here.

**What is `CompactMerkleTree`?**

It is a `MerkleTree` which nodes are labelled and removed as soon as possible.

In `MerkleTree`, all leaf nodes are added and kept in memory before computing hash. It computes hash from leaf nodes to root node level by level.

In `CompactMerkleTree`, they will be computed instantly once two child nodes are added, save the binary hash on their parent node. The two child nodes are removed from memory.

For example:

* the size of the data block is chunk\_size.
* a data block can be released from memory once it's hash is computed and added into MerkleTree as leaf node.
* `In Memory` means it has to be kept in memory for computing hash.
* `Out Memory` means it is safe to release from memory or is not added in memory yet.

1. Trees in memory when 3 hash leaves are added

<figure><img src="https://github.com/0chain/blobber/wiki/assests/hash/merkle_tree_vs_shrinked_merkle_tree_1.png" alt=""><figcaption></figcaption></figure>

2. Trees in memory when 5 hash leaves are added

<figure><img src="https://github.com/0chain/blobber/wiki/assests/hash/merkle_tree_vs_shrinked_merkle_tree_2.png" alt=""><figcaption></figcaption></figure>

3. Trees in memory when 7 hash leaves are added

<figure><img src="https://github.com/0chain/blobber/wiki/assests/hash/merkle_tree_vs_shrinked_merkle_tree_3.png" alt=""><figcaption></figcaption></figure>

`CompactMerkleTree` is better than `MerkleTree` to persist the tree to disk or keep it in memory.

`Challenge Hash` is also computed by `MerkleTree`. For Outsourcing Attack, we have a `FixedMerkleTree` here.

{% hint style="info" %}

#### Outsourcing Attack

To ensure each storage server commits resources and doesn’t outsource its tasks, our protocol requires that the content for verification is 64 kB, derived from the full file fragment. Here’s how it works:

1. **File Division**: The file is split into ( n ) 64 kB fragments, each assigned to a storage server.
2. **Chunking**: Each 64 kB fragment is further divided into 64-byte chunks, resulting in 1024 chunks per fragment.
3. **Hashing**: Data at each index across the blocks is treated as a continuous message and hashed. These 1024 hashes form the leaf nodes of a Merkle Tree.
4. **Merkle Tree**: The root of the Merkle Tree is used to aggregate file hashes up to the directory/allocation level. The Merkle proof provides the path from the leaf to the file root and then to the allocation level.

To pass a challenge for a specific index (1 to 1024), a dishonest server would need to download all content and construct the leaf hash, discouraging outsourcing and ensuring genuine resource commitment.
{% endhint %}

**What is `FixedMerkleTree`?**

It is a `MerkleTree` in which every leaf node is labelled with the `MerkleTree` hash of a data block and every non-leaf node is labelled with the cryptographic hash of the labels of its child nodes. It always has fixed 1024 leaf nodes. That is why we call it `FixedMerkleTree`.

See detail on example:

<figure><img src="https://github.com/0chain/blobber/wiki/assests/hash/challenge_hash.png" alt=""><figcaption></figcaption></figure>

{% hint style="success" %}
`Challenge Hash` was computed hash with `MerkleTree` in which every leaf node is labelled with `sha3`, and every non-leaf node is labelled with the cryptographic hash of the labels of its child nodes. `sha3` asks fully load `block..1-n`,`block..2-n`...`block..1024-n` on memory before computing the hash for the leaf node `L..n`. I have the same issue as `sha1` in `Actual File Hash` and `Content Hash`. That is why every leaf node is labelled with `CompactMerkleTree` instead of `sha3` now.
{% endhint %}