Docs
Quickstart
Spilled is a network observability platform for collecting, storing, and querying network telemetry. It is designed for interactive investigation: instead of relying solely on predefined dashboards, operators can ask questions of their network data as an incident unfolds.
Spilled uses a Bring Your Own Cloud (BYOC) deployment model. Fusion, our stateless executable responsible for both flow ingestion and query execution, runs in your cloud while the Spilled control plane remains managed by Spilled in our cloud. Raw telemetry data is stored in your object storage, such as Amazon S3. Building on cloud object storage follows the modern pattern for scalable data systems making petabyte-scale retention practical while keeping storage costs manageable.
Deploying Fusion
Fusion is the all-in-one runtime for smaller workloads. It combines flow ingestion and query execution in a single deployment.
Create an account
To get started, create a Spilled account.
Create API keys
Once you have an account, navigate to the API keys page found under Team Settings and create both a query and an ingest API key.
Get your query cluster ID
Query clusters are logical groups of instances used to execute queries. On sign up, a cluster, default, is created for you. Go to the Query Clusters page to get the cluster ID.
Deploy Fusion
Fusion deployments are stateless. They can be replaced or rescheduled freely. Fusion is distributed as a container image and can be deployed anywhere you can run container images including Kubernetes, Nomad, Docker and cloud platforms that support container workloads.
As an example, to run on Docker:
docker run -d \
-p 2055:2055/udp \
-p 6343:6343/udp \
-e SPX_BUCKET_URL=<bucket-url> \
-e SPX_INGEST_API_KEY=<ingest-api-key> \
-e SPX_QUERY_API_KEY=<query-api-key> \
-e SPX_CLUSTER_ID=<cluster-id> \
-e AWS_ACCESS_KEY_ID=<access-key> \
-e AWS_SECRET_ACCESS_KEY=<secret-key> \
ghcr.io/liquidspill/fusion:latest
Ingress ports
By default, Fusion listens on the following UDP ports.
| Port | Protocol | Purpose |
|---|---|---|
2055 | UDP | Netflow (v5/v9) & IPFIX ingestion |
6343 | UDP | sFlow ingestion |
Exporting flows
Configure your network device to export flows to your Fusion instance. This will require your instance having a publicly accessible IP address, either directly attached to your Fusion instance or behind a L4 load balancer (commonly known as a network load balancer (NLB) on most public clouds).
Ensure that your exporter is configured to have an active timeout of 60 seconds.
Required configuration
The following configuration is required for a standard Fusion deployment.
Bucket URL
SPX_BUCKET_URL identifies the bucket Spilled should use for persisted data. The
exact URL shape depends on the object store implementation.
Currently, Spilled only supports AWS S3 and S3-compatible object stores.
- Format
-
s3://{BUCKET_NAME}?region={REGION} - Example
s3://mybucket?region=us-east-1
- Format
-
s3://{BUCKET_NAME}?region={REGION}&endpoint={ENDPOINT}&s3ForcePathStyle=true - Example
-
s3://mybucket?region=auto&endpoint=https://XXX.r2.cloudflarestorage.com&s3ForcePathStyle=true
Environment Variables
The following environment variables are required to deploy Fusion.
| Variable | Description |
|---|---|
AWS_ACCESS_KEY_ID | Access key for the S3-compatible object store. |
AWS_SECRET_ACCESS_KEY | Secret key for the S3-compatible object store. |
SPX_INGEST_API_KEY | Credential used by the ingest path. |
SPX_QUERY_API_KEY | Credential used by the query path. |
SPX_CLUSTER_ID | Spilled cluster identifier. |
SPX_BUCKET_URL | Bucket URL used for persisted flow data. |
Object storage
Object storage is the durable backing store for Spilled. We recommend using a dedicated bucket for your Spilled data. However, all data written by Spilled is stored under the /spilled prefix in your object storage bucket.
Do not enable object retention policies, object versioning, or soft deletion on the bucket used by Spilled. Retention policies can cause data corruption if the bucket state diverges from the files Spilled still considers active.
Permissions
Fusion needs permission to create, read, list, and delete objects in the bucket. Those operations are required for normal writes, reads and maintenance workflows such as compaction and retention.