> For the complete documentation index, see [llms.txt](https://docs.zus.network/zus-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.zus.network/zus-docs/webapps/blimp.md). # Blimp Blimp is an ACID cache and autonomous materialized view (MV) on Iceberg — an efficient per-core cache (2–4 GB/s per node) that keeps your AI/ML context fresh with CDC delta-merges in seconds. One small, simple, scalable node runs in your own AWS account and VPC — your engines, your catalog, and your data never leave your cloud. ## Introduction Blimp sits between your compute engines and S3 as a smart, ACID cache with an autonomous materialized view — keeping your AI/ML context fresh, without you migrating data or changing your engines. Point your engines at it, simplify your pipeline, and lower cost. You point your existing tools (Spark, Trino, Snowflake, Ray, PyTorch, …) and your Iceberg catalog at a Blimp node. Blimp takes your query, materializes the right Iceberg views automatically, and serves reads from a high-throughput NVMe cache in front of S3. It is **non-invasive**: the node is provisioned into *your* AWS account, your data stays in *your* S3 bucket, and nothing leaves your cloud. * **Open & standard** — Iceberg tables, S3-compatible endpoints, Glue / Polaris / Nessie-compatible catalog. * **Fully ACID** — every write commits as an atomic data entity; no orphaned files, no partial reads. * **Yours end-to-end** — runs on Amazon EC2 in your VPC; you keep ownership of the data, keys, and bucket. ## Features * **Autonomous materialized views** — Blimp takes your query, then authors and materializes Iceberg materialized views (MVs) for you and keeps them fresh via CDC delta-merges in a few seconds (3–4s). Materialized queries return **sub-second** — and because the MVs are standard Iceberg tables, any existing pipeline gets the same sub-second reads through its **current query engine** (Trino, Spark, Snowflake, DuckDB, …). No SQL rewrites, no schema changes. * **Storage Cache** — an **efficient cache per core (2–4 GB/s per node)**: an NVMe cache in front of S3, exposed over fast S3 and Mountpoint-S3 endpoints, serving fresh data for inference and keeping GPUs fed for training with no cross-region latency. Fewer cores for the same throughput means lower cost and a greener footprint, and it scales up or down with any workload — analytics, inference, or training. * **Cache Router** — a transparent read-through cache (the blimp-router, `:8088`) in front of your S3. On a miss it pulls the object from your origin once, stores it on the node blobbers, and serves it; every repeat read is then served from the blobbers. Cached data is reachable two ways: an **S3 path** (gateway `S3:9000`) for objects, and **Mountpoint for Amazon S3** as a filesystem mount over the same endpoint, so apps and inference loops use whichever they prefer. * **CDC engine** — a change-data-capture pipeline that tracks Iceberg snapshots and keeps your MVs and cached data consistent with the source tables, so optimizer MVs refresh automatically as your data changes. * **Storage benchmarks** — built-in fio, warp, and MLPerf benchmarks let you measure a node's throughput before you commit a workload. * **Unlimited nodes** — spin up as many independent nodes as you need, each in your VPC, and add more to scale throughput. ## Architecture A Blimp deployment has two planes. **Control plane (Manager)** — a multi-tenant, highly-available control plane authenticates users, provisions nodes into your AWS account, and proxies privileged operations. Application calls carry your login token in an `x-app-id-token` header; the node's gateway admin token never leaves the server side. The Manager exposes a REST control-plane API used by both the web app and your own automation. **Data plane (the node)** — each node runs entirely inside your VPC: | Component | Role | Endpoint | | ------------------- | ------------------------------------------------------------------------------------------------------------ | ------------------------------- | | **Gateway** | S3-compatible gateway + DuckDB query optimizer; authors/materializes MVs, encodes/decodes erasure-coded data | S3 `:9000`, Iceberg MV `:19123` | | **blimp-router** | Transparent read-through cache in front of origin S3 | `:8088` | | **Blobbers** | Erasure-coded storage backed by local NVMe; hold cached objects, MVs, and results | internal | | **Iceberg catalog** | Catalog for the materialized views (Nessie / Polaris / Glue-compatible) | via gateway | | **CDC engine** | Tracks Iceberg snapshots and refreshes dependent MVs and cached data | internal | The gateway fans out to every blobber on write (PUT) and reconstructs from any available shard on read (GET), so its vCPU and network scale with the data-shard count while blobbers stay uniform. #### Cache read paths — your S3 Every read goes through the **blimp-router**: a **hit** is served straight from the local blobbers with no cross-region hop, and a **miss** pulls the object from your origin S3 once, caches it on the blobbers, and serves it. The Gateway exposes the cache two ways — S3 and Mountpoint-S3 — so your app can read it however it prefers. ![Blimp Cache Router — read through cache: a hit is served from the local blobbers, a miss pulls once from your origin S3](/files/atFJXbbgsiXnukZwqtEl) #### Query Optimizer — Iceberg MVs authored on demand A query registers its source table into the Iceberg catalog, Harness/LLM authors the matching MV shape, and the Gateway materializes it by pulling straight from S3 and caching the result on the blobbers — Blimp then returns a link to the result. CDC watches the source afterward and keeps the MV fresh as it changes. ![Query Optimizer — Harness/LLM-authored Iceberg MV flow, kept fresh by CDC](/files/SvbKQHwWnnhNWZhwF9GN) ## ACID — consistent under load Every write to a Blimp node is erasure-coded across independent blobbers and commits as a single atomic unit — nothing is ever half-written. If a write is interrupted, there's no orphaned file left behind for a later reader to trip over; the write either lands completely or not at all. Each write is row-hash verified, and because the data is erasure-coded (2 data + 1 parity by default), losing or corrupting a single blobber shard doesn't corrupt your data — it's reconstructed transparently from the remaining shards, with no data loss and no manual repair. Reads stay consistent under concurrent load too: a value you just wrote is the value the next reader gets, and a read racing an overwrite never returns a stale or torn mix of old and new bytes — whether you reach the node through the raw gateway S3 API, the read-through cache router, or the mounted filesystem. This is validated with the same linearizability model-checker used in Jepsen-style distributed-systems testing, run against all three read paths under both single-writer and multi-writer load — see run-blimp's [`blimp --acid`](https://github.com/0chain/run-blimp#d--testing-acid-blimp---acid). The result: you don't have to reason about orphaned files, torn writes, silent corruption, or lost transactions — the node guarantees them away. ## Security — blockchain zero-trust Blimp is **breach-resistant by architecture**. It runs on **zero standing trust** — nothing in the pipeline is trusted by default. Most breaches start with a spoofed key or a stolen identity, an attacker impersonating a trusted entity; Blimp anchors every identity on the blockchain, so a leaked key alone can't be replayed to forge an identity. * **Blockchain-anchored, zero-trust identity** — every entity (gateway, blobbers, clients) has an identity anchored on the blockchain, so a leaked or stolen key can't on its own be used to impersonate it. No one can spoof a key or steal an identity to impersonate an entity in your pipeline. * **Signed messages between entities** — every message between the gateway, blobbers, and clients is cryptographically signed. No unsigned traffic, no shared secrets, no implicit trust anywhere in the system. * **ACID data integrity** — every write commits as an atomic data entity: no partial writes, no torn reads, no orphaned files. Your AI inference always reads complete, consistent data. * **Customer-owned infrastructure** — every node runs inside your own AWS account and VPC; your engines, catalog, and data never leave your cloud. ## Quickstart — try Blimp end to end in \~15 minutes The fastest way to see what Blimp does: sign up, launch the prefilled dev node, and run the built-in demos — then point it at your own data. No data migration, no SQL rewrites. 1. **Sign up with Gmail.** Go to [blimp.software](https://blimp.software) and continue with Google — a key is created for you automatically. 2. **Launch the 2/1 node.** On the empty Home screen click **Launch a node**. The form comes **prefilled** for a **2/1** node (region, data/parity ratio, and storage already set) — just enter your **wallet password** and submit. The first time, connect the AWS account Blimp provisions into; after that it's remembered. You'd need to click **Create ARN**, which takes you to the AWS login page and then to a Quick Create Stack template — tick the **Acknowledge** box, click **Create Stack**, and on the resulting page select the **Output** tab and click the **Blimp** link to return to the app (or simply copy & paste the ARN value). 3. **Wait \~8 minutes** while the node forms — watch the deployment progress bar. When it turns **green**, you'll see two demo areas, the **Query Optimizer** and the **Storage Cache**. 4. **Run the SF1 query.** In **Query Optimizer → Benchmark (TPC-DS)**, click **Run TPC-DS q88** against the self-seeded SF1 (\~1 GB) dataset — note the author / materialize times; a warm re-run is sub-second. 5. **Check the Storage Cache.** Open **Storage Cache → Benchmarks** and run **warp**, **fio**, or **MLPerf** to measure the node's S3, Mountpoint-S3, and AI data-loader throughput tested internally on the gateway — each test takes 1-2 minutes, with MLPerf the longest as it builds and writes the data first to conduct the test. ## Storage Performance Measure how much data the **gateway** can pull from the **blobbers** and serve. Open a node → **Storage Cache** → **Benchmarks**, choose a tool + concurrency / object size, and run — each streams a live log and a summary: * **warp** — S3 throughput against the gateway's S3 endpoint (S3:9000). Parallel GET/PUT, reporting aggregate MB/s + ops/s — the S3-protocol data rate served from the blobbers. * **fio** — Mountpoint-S3 throughput against the gateway's Mountpoint-S3 mount. Sequential / random read + write bandwidth and IOPS — what an app or data loader sees mounting the node as a filesystem. * **MLPerf Storage** — an ML training data-loader benchmark against the node (samples/s, achieved utilization) — an end-to-end measure of whether the cache keeps GPUs fed under an AI workload. The cache product can be used for **high-compute analytics, datalake compaction, and AI inference and training** — anywhere you need speed of execution while lowering CPU and GPU compute cost. It runs on commodity network-optimized instances and delivers an efficient cache per core — fewer cores for the same throughput, which is both cheaper and greener — and scales up or down with the workload. Expected performance (measured on live AWS nodes): | | **2/1 node** | | --------------------- | -------------------------------- | | Gateway | c6in.4xlarge (16 vCPU) | | Blobber nodes | 3 × c6in.xlarge | | Fault tolerance | 2 data + 1 parity | | Capacity | Up to 128 TB | | Total vCPUs | 28 (14 cores) | | TTFB (p50 / p99) | 1-2 ms | | S3 — warp | 0.9 GB/s PUT · 4.3 GB/s GET | | AI training — MLPerf | 1.6-2 GB/s at 95% AU | | GET throughput / vCPU | \~153 MB/s (\~306 MB/s per core) | Results persist on the gateway with hosted log links, so a run survives a browser refresh — compare tools and concurrency levels to find the node's ceiling before you commit a workload. ## Query Mechanics ### How it works We take the original query, strip its literals, and generalise the grain so the MV isn't tied to one set of filter values — `d_year = 2000` becomes `d_year` in the `GROUP BY`. That materialised view holds pre-aggregated rows at that grain. The rewrite then re-applies the literals on top of the MV and joins whatever dimensions it still needs, producing the original query's exact answer — verified by row-hashing the original against the rewrite so a wrong MV is rejected rather than served. Three constraints make it actually work: * **Measures must be re-aggregatable.** SUM and COUNT roll up; AVG doesn't. An MV storing an average is unusable, so we store `SUM(x)` and `COUNT(x)` per measure and let the rewrite divide. * **Refresh is O(delta), not O(fact).** When new rows land, we aggregate only the delta at the same grain and write it as a separate parquet part beside the MV; readers fold base + delta at query time. The base is never rewritten. This holds only when each UNION branch of the recipe reads exactly one fact — with k facts in a branch, the algebra needs 2^k−1 inclusion-exclusion terms and the unchanged ones bind to the full table, so the "merge" silently becomes a full scan. * **Grain must be scale-invariant.** A grain on a bounded column (`ss_quantity`, 100 values) gives the same MV at SF1 and SF1000. A grain on a surrogate key (`ss_customer_sk`) is 100k rows at SF1 and 12M at SF1000 — which is how q4 ends up with a 53M-row MV that costs more to read than the query it replaces. Example — TPC-DS **q9**, removing non-linearity from a query so its MV can delta-merge. Measured on a single r6i.4xlarge (16 vCPU / 123 GB) against TPC-DS SF1000 — `store_sales` 2,879,987,999 rows — on 2026-08-05. This is the clearest example of the technique, because q9 packs three different kinds of non-linearity into one query and all three have to come out before the view can be maintained incrementally. It asks five independent questions of the same table: each is a `CASE` whose branch is chosen by a count threshold, and whose value is an average over a quantity band. **1 · Original query** ```sql CASE WHEN (SELECT count(*) FROM store_sales WHERE ss_quantity BETWEEN 1 AND 20) > 74129 THEN (SELECT avg(ss_ext_discount_amt) FROM store_sales WHERE ss_quantity BETWEEN 1 AND 20) ELSE (SELECT avg(ss_net_paid) FROM store_sales WHERE ss_quantity BETWEEN 1 AND 20) END bucket1, ... the same shape for buckets 2-5, bands 21-40, 41-60, 61-80, 81-100 ... FROM reason WHERE r_reason_sk = 1 ``` Three things here cannot live in a materialized view: | construct | why it breaks incremental maintenance | | ------------------ | ----------------------------------------------------------------------- | | `avg(x)` | not additive — `avg(A ∪ B)` is not derivable from `avg(A)` and `avg(B)` | | `CASE ... > 74129` | a literal threshold; baking it in fixes the view to one binding | | `BETWEEN 1 AND 20` | a literal band; same problem, five times over | **2 · The linear core** — MV definition Strip all three out and what remains is a plain SPJG aggregate — one table, one grouping column, additive measures only: ```sql SELECT ss_quantity, COUNT(*) AS cnt, SUM(ss_ext_discount_amt) AS sum_disc, COUNT(ss_ext_discount_amt) AS cnt_disc, SUM(ss_net_paid) AS sum_net, COUNT(ss_net_paid) AS cnt_net FROM store_sales GROUP BY ss_quantity ``` 101 rows. Every non-linear construct is gone. Note what replaced `avg`: a **SUM and a COUNT stored separately**, so the division can happen later. The `COUNT` is of the measure expression, not `COUNT(*)`. That distinction is not cosmetic — on q14 the same recipe used `COUNT(*)` as an AVG denominator when the true one excluded NULL factors, and the two differed by **129,390 rows** at SF1. The row-hash verifier caught it; a stored average would have hidden it. **3 · The rewrite** — where the non-linearity goes ```sql SELECT CASE WHEN (SELECT SUM(cnt) FROM tpcds_mv.store_sales_qty_rollup WHERE ss_quantity BETWEEN 1 AND 20) > 74129 THEN (SELECT SUM(sum_disc)/SUM(cnt_disc) FROM tpcds_mv.store_sales_qty_rollup WHERE ss_quantity BETWEEN 1 AND 20) ELSE (SELECT SUM(sum_net)/SUM(cnt_net) FROM tpcds_mv.store_sales_qty_rollup WHERE ss_quantity BETWEEN 1 AND 20) END bucket1, CASE WHEN (SELECT SUM(cnt) FROM tpcds_mv.store_sales_qty_rollup WHERE ss_quantity BETWEEN 21 AND 40) > 122840 THEN (SELECT SUM(sum_disc)/SUM(cnt_disc) FROM tpcds_mv.store_sales_qty_rollup WHERE ss_quantity BETWEEN 21 AND 40) ELSE (SELECT SUM(sum_net)/SUM(cnt_net) FROM tpcds_mv.store_sales_qty_rollup WHERE ss_quantity BETWEEN 21 AND 40) END bucket2, CASE WHEN (SELECT SUM(cnt) FROM tpcds_mv.store_sales_qty_rollup WHERE ss_quantity BETWEEN 41 AND 60) > 56580 THEN (SELECT SUM(sum_disc)/SUM(cnt_disc) FROM tpcds_mv.store_sales_qty_rollup WHERE ss_quantity BETWEEN 41 AND 60) ELSE (SELECT SUM(sum_net)/SUM(cnt_net) FROM tpcds_mv.store_sales_qty_rollup WHERE ss_quantity BETWEEN 41 AND 60) END bucket3, CASE WHEN (SELECT SUM(cnt) FROM tpcds_mv.store_sales_qty_rollup WHERE ss_quantity BETWEEN 61 AND 80) > 10097 THEN (SELECT SUM(sum_disc)/SUM(cnt_disc) FROM tpcds_mv.store_sales_qty_rollup WHERE ss_quantity BETWEEN 61 AND 80) ELSE (SELECT SUM(sum_net)/SUM(cnt_net) FROM tpcds_mv.store_sales_qty_rollup WHERE ss_quantity BETWEEN 61 AND 80) END bucket4, CASE WHEN (SELECT SUM(cnt) FROM tpcds_mv.store_sales_qty_rollup WHERE ss_quantity BETWEEN 81 AND 100) > 165306 THEN (SELECT SUM(sum_disc)/SUM(cnt_disc) FROM tpcds_mv.store_sales_qty_rollup WHERE ss_quantity BETWEEN 81 AND 100) ELSE (SELECT SUM(sum_net)/SUM(cnt_net) FROM tpcds_mv.store_sales_qty_rollup WHERE ss_quantity BETWEEN 81 AND 100) END bucket5 FROM (SELECT 1) dummy WHERE 1=1 AND (SELECT r_reason_sk FROM reason WHERE r_reason_sk = 1) = 1 ``` ## Performance Comparison A Blimp node is in its own category — a query *and* cache optimizer with unparalleled cost efficiency through liquid scaling of data acceleration based on demand, a uniquely secure data environment, easy scalability on the cloud, and full ACID compliance. It also holds up directly against pure-play systems: roughly double the cache throughput per core of Alluxio and VAST Data, and faster query performance than StarRocks on TPC-DS (Iceberg). | Cache throughput | Cores | GB/s | GB/s per core | | ------------------------------------------------------------------------------------------------------------- | ----- | ---- | ------------- | | Blimp | 14 | 4 | 0.29 | | [VAST Data](https://kb.vastdata.com/docs/s3-benchmarking-using-elbencho#choosing-the-right-number-of-clients) | 64 | 10 | 0.16 | | [Alluxio](https://documentation.alluxio.io/ee-ai-en/benchmark/s3-api#id-6-node-httpbench-on-aws) | 72 | 11 | 0.15 | Sources: Alluxio 6-node HTTPBench on AWS (single client/worker pair, peak throughput, 72-vCPU c5n.18xlarge worker) · VAST Data S3 benchmarking guide (per-CNode figure). TPC-DS — StarRocks (Iceberg catalog) vs Blimp, total query time: | Query | StarRocks — Iceberg | Blimp | | ----- | ------------------- | ------ | | Q09 | 23.2 s | 0.83 s | | Q88 | 19.5 s | 1.14 s | | Q04 | 16.7 s | 8.44 s | | Q14 | 15.3 s | 2.71 s | | Q24 | 11.9 s | 5.74 s | | Q59 | 3.2 s | 0.90 s | | Q47 | 3.1 s | 0.93 s | | Q13 | 2.2 s | 1.08 s | Source: [StarRocks TPC-DS benchmark](https://docs.starrocks.io/docs/benchmarking/TPC_DS_Benchmark/#test-results) — Iceberg catalog results, 1 FE + 4 BE × m6id.4xlarge (16 vCPU each). Blimp — measured on the 2/1 node above. ## Production setup - use run-blimp repo The production paths mirror the [**run-blimp**](https://github.com/0chain/run-blimp) client, which is a repo to automate testing out the Blimp node for cache and query performance from your app hosted on a machine that's close to you, preferably in the same VPC, same zone, same region in order to maximize performance. Test the node end-to-end with the **run-blimp** repo's `blimp` CLI: `blimp --setup` wires the kit to your data in about 10-15 minutes, `blimp --query` runs the query-optimizer + CDC suite (author → append → delta-merge) in about 5 minutes, and `blimp --storage` runs the storage/cache suite (warp, fio, MLPerf, cache) in about 30 minutes. Add `--verify` to `blimp --query` to row-hash verify each result against a full DuckDB recompute of the source. Cache objects pulled in through the **blimp-router** and **materialized views created by our harness, LLM, and DuckDB** evict independently: the cache ages out under its own eviction-% (share of free disk), while MVs — auto-created as incoming queries touch new shapes, so every hot query has an MV in place — age out under a separate MV eviction policy. You can use the scripts in **run-blimp** to integrate with your app or orchestrator and scale your data pipeline's performance. The node is a **liquid** resource: add TB/s of throughput and thousands of queries per second by spawning nodes on demand, pause them to cut cost, resize the Gateway vCPUs to tune performance, and add storage as capacity needs grow. For production, reserve a base set of paused nodes you can restart at any time to cover steady demand at low cost, and spawn new nodes within **\~8 minutes** to absorb a spike. #### Production query — your Iceberg 1. **Connect your Iceberg and S3.** Fill in the Iceberg REST URL, Warehouse, S3 endpoint / access key / secret, and Namespace. 2. **Run a query.** Enter SQL via node panel UI or script and click **▶ Run Production Query**. Blimp pulls the source into the node's blobbers, materializes the MV + result, and returns hosted viewer URLs. After that, your Iceberg commit trigger drives CDC and auto-refreshes the MV. 3. **Govern access.** Each node has CIDR allow-lists for its Iceberg catalog and S3 gateway endpoints — restrict them to the networks that should reach the node. Drive the same prod query from scripts and dashboards with the control-plane API — see **API & integration** below. Query performance is largely dependent on the size of the dataset, so the gateway needs to scale with capacity demand. For example, a `2xlarge` is adequate for SF1 (\~1 GB); SF10 and SF100 need a `4xlarge` or larger; and a SF1000 dataset starts at an `8xlarge`. #### Production cache — your S3 1. **Point the cache at your S3.** Open a node → **Storage Cache** → **Production (your S3 endpoint)** and supply your endpoint + bucket. 2. **Read through the node.** A miss pulls the object from your bucket into the local blobbers once; every hit after that is served locally — never leaving the region. 3. **Measure it.** **▶ Measure PUT/GET** reports the observed last-60s data rate through the cache. 4. **Govern access.** Lock every endpoint (S3 `:9000`, router `:8088`) to your networks with the node CIDR allow-lists. Wire the same read-through cache into your own scripts and data loaders — see **API & integration** below. ## API & integration Two things are worth wiring into your own scripts, dashboards, and ML jobs: **running optimized queries on your data**, and **reading your data through the node cache**. (The demo and benchmark buttons are app-only — nothing to integrate there.) ### 1 · Run optimized queries on your Iceberg ![Query integration — A: the managed prod-query API authors + materializes an MV and returns hosted URLs; B: your own engine reads the MVs directly via the Iceberg MV endpoint (:19123).](/files/yK1XHjTtpPwtDDQxLkCL) POST your SQL to the control-plane API with your login token in the `x-app-id-token` header. The gateway ingests only the tables your SQL touches (in parallel), authors + materializes a view, and returns hosted result/MV URLs; a warm re-run reuses the MV and is sub-second. ```bash # enable once (one-way) curl -X POST https://datalake.blimp.software/api/v1/clusters/optimizer/prod-query/enable \ -H 'x-app-id-token: ' -H 'Content-Type: application/json' \ -d '{"cluster_id":""}' # run a query — scope the SQL to the tables you actually need curl -X POST https://datalake.blimp.software/api/v1/clusters/optimizer/prod-query \ -H 'x-app-id-token: ' -H 'Content-Type: application/json' \ -d '{ "cluster_id":"", "iceberg_url":"https://", "warehouse":"s3://", "s3_endpoint":"https://s3..amazonaws.com", "s3_key":"", "s3_secret":"", "namespace":"", "sql":"SELECT d_year, COUNT(*) FROM date_dim GROUP BY d_year" }' ``` Prefer your own engine? Point Trino / Spark / DuckDB at the read-only **Iceberg MV endpoint** `http://:19123/iceberg` (warehouse `mv`) and query the materialized views directly. ### 2 · Read your data through the node cache ![Cache integration — your app reads through the blimp-router into the Gateway's S3/Mountpoint-S3 endpoints; a hit is served from the local blobbers, and a miss pulls once from your origin S3.](/files/LR3Yp1D09JN8oB5VxI0q) Set it up once from the app — **Storage Cache → Production** — which points the node's read-through cache at your S3. After that, read *through the node* instead of origin S3: on a miss it pulls the object from your bucket into the local blobbers, and every warm read is served locally — never leaving the region. Throughput **scales with concurrency**; a single stream is capped by your origin S3 and the gateway instance network, so use parallel readers across many files. ```bash # Mountpoint-S3: mount the read-through cache as a filesystem (inference / data access) mount-s3 /mnt/blimp --endpoint-url http://:8088 # S3-style: read through the node router (miss -> origin S3 -> cached) curl http://:8088// -o out ``` Lock every endpoint (`S3:9000`, Iceberg `:19123`, router `:8088`) to your networks with the node **CIDR allow-lists**. Tune the cache size with the eviction-% (share of free disk) and validate the speed-up under **Storage Cache → Production** before you cut a workload over. ## Deployment & sizing Nodes are provisioned into your AWS account on the EC2 `c6in` family. A one-time setup connects the AWS account (via a CloudFormation Quick Create Stack that returns an IAM role ARN); subsequent launches reuse it. Capacity is described by an **erasure-coding ratio** (data shards / parity shards). The gateway scales with the data-shard fan-out; blobbers are uniform. The node tier offered is: | Tier | Erasure coding | Gateway | Blobbers | Capacity | | ---- | ---------------- | ------------------------ | ----------------- | ------------ | | 2/1 | 2 data, 1 parity | `c6in.4xlarge` (16 vCPU) | 3 × `c6in.xlarge` | Up to 128 TB | Each blobber's EBS data volume is provisioned for high throughput (gp3, 8000 IOPS / 2000 MiB/s) independent of instance size. Total usable storage is split across the data shards at deploy time. Query performance is largely dependent on the size of the dataset, so the gateway needs to scale with capacity demand. For example, a `2xlarge` is adequate for SF1 (\~1 GB); SF10 and SF100 need a `4xlarge` or larger; and a SF1000 dataset starts at an `8xlarge`. ## Scaling with nodes Blimp scales horizontally — you are not limited to one node: * **Add unlimited nodes** from **Manage Nodes** at any time. Each node is independent and runs in your VPC. * **Scale throughput** by adding nodes — more nodes mean more aggregate cache and query bandwidth to serve more engines and workloads in parallel. * **Manage a demand-based lifecycle** — start, stop, resize storage and instances, and destroy nodes from the same screen; the deployment status and disk-usage gauges keep you informed. * **Pricing** follows a simple per-GB-provisioned-hour model of node storage plus the per-vCPU-hour based on Gateway instance. We do not charge during stops, or for blobber vCPUs, and do not charge for Gateway storage which is used by DuckDB for large dataset spills. You pay AWS directly for the underlying infrastructure (it runs in your account); Blimp meters its software through the [AWS Marketplace listing](https://aws.amazon.com/marketplace/pp/prodview-zn364bubzupaq) for current rates. We do not charge for LLMs used for certain queries, but have a restriction on the number of calls per day. Pause nodes and resize the Gateway vCPUs to match demand, keep a small base of paused nodes for steady load, and spawn new nodes within **\~8 minutes** for spikes — cost tracks actual usage, not peak capacity. *** Need help? Reach the team via **Help → Discord** in the settings menu, email , or [book a meeting](https://calendly.com/saswata_basu/15min).