Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.trynebula.ai/llms.txt

Use this file to discover all available pages before exploring further.

By default the Enterprise stack runs Postgres (Nebula app + Hatchet) and MinIO inside the deploy boundary. For any production AWS deployment we recommend backing these onto RDS Postgres and real S3 instead — better durability, point-in-time recovery, IAM-driven credentials, and operational ownership by AWS. This page covers the AWS-side setup and the config knobs. The same managed-resource path works on both the Compose and EKS deploys.

​
RDS Postgres

​
Instance setup

  • Engine: PostgreSQL 16 (Aurora PostgreSQL 16 also works)
  • Databases: pre-create the two databases (nebula and hatchet) as the master user before bootstrap. The bundled create-hatchet-db.sh runs createdb as HATCHET_POSTGRES_USER against managed Postgres only as an idempotent no-op; it does NOT assume CREATEDB on the application role.
  • Parameter group: rds.extensions must include vector. The database initialization path must run CREATE EXTENSION IF NOT EXISTS vector before API traffic reaches the cluster.
  • Network: private subnet in the same VPC as the compose host or EKS cluster. The host’s / cluster’s security group must be allowed inbound on :5432 from the application security group.
  • Storage: gp3, 100 GB minimum to start. Enable autoscaling up to ~1 TB; Nebula’s working set scales linearly with the number of collections + total ingested document volume.
  • Backups: RDS automatic backups, 7-day retention minimum. PITR is enabled by default.

​
Database / user provisioning

Two logical databases on the same RDS instance — separate Postgres users for blast-radius isolation: 
-- Connect as the master user (e.g. via psql against the RDS endpoint)
CREATE DATABASE nebula;
CREATE USER nebula WITH PASSWORD '<generate-32-bytes-random>';
GRANT ALL PRIVILEGES ON DATABASE nebula TO nebula;

CREATE DATABASE hatchet;
CREATE USER hatchet WITH PASSWORD '<generate-32-bytes-random>';
GRANT ALL PRIVILEGES ON DATABASE hatchet TO hatchet;
Hatchet’s hatchet-create-db one-shot expects to be able to create its database on first boot; if you’ve already created it above, the one-shot is a no-op (it checks for existence).

​
Compose: wire up via .env.enterprise

Append to env/.env.enterprise: 
NEBULA_POSTGRES_HOST=<rds-endpoint>.us-east-1.rds.amazonaws.com
NEBULA_POSTGRES_PORT=5432
NEBULA_POSTGRES_USER=nebula
NEBULA_POSTGRES_DBNAME=nebula
NEBULA_POSTGRES_SSL_MODE=require
NEBULA_POSTGRES_PASSWORD=<password from CREATE USER above>

HATCHET_POSTGRES_HOST=<rds-endpoint>.us-east-1.rds.amazonaws.com
HATCHET_POSTGRES_PORT=5432
HATCHET_POSTGRES_USER=hatchet
HATCHET_POSTGRES_DBNAME=hatchet
HATCHET_POSTGRES_SSL_MODE=require
HATCHET_POSTGRES_PASSWORD=<password from CREATE USER above>
bootstrap.sh auto-detects the override and skips the in-stack postgres + hatchet-postgres containers via compose profiles. No other changes required.

​
EKS: wire up via Helm values

In your your-values.yaml (copied from helm/examples/eks/values.yaml): 
postgres:
  mode: external
  host: <rds-endpoint>.us-east-1.rds.amazonaws.com
  port: 5432
  database: nebula
  credentialsSecret: nebula-postgres-credentials

hatchetPostgres:
  mode: external
  host: <rds-endpoint>.us-east-1.rds.amazonaws.com
  port: 5432
  database: hatchet
  credentialsSecret: hatchet-postgres-credentials
Each credentialsSecret references a Kubernetes Secret with username and password keys (those exact lowercase key names — the chart reads them via secretKeyRef.key: username / .key: password). Create them via ESO (sync from AWS Secrets Manager) or kubectl create secret generic <name> --from-literal=username=<u> --from-literal=password=<p>.

​
S3 (object storage)

​
Bucket setup

  • Region: same as the compose host / EKS cluster (cross-region adds latency to every snapshot read)
  • Versioning: enabled (recommended; protects against accidental deletes)
  • Encryption: SSE-S3 or SSE-KMS
  • Public access: blocked at the account level
  • Lifecycle: optional — Nebula doesn’t expire its own objects, but you can set a policy on the incomplete-multipart-uploads prefix to clean up failed ingests after 7 days

​
IAM policy

The principal accessing S3 (instance profile / ECS task role / EKS IRSA role / IAM user) needs: 
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::<your-bucket>",
        "arn:aws:s3:::<your-bucket>/*"
      ]
    }
  ]
}
If you’re using SSE-KMS, also grant kms:Encrypt, kms:Decrypt, kms:GenerateDataKey on the KMS key ARN.

​
Compose: wire up via .env.enterprise

Append to env/.env.enterprise: 
NEBULA_USE_EXTERNAL_S3=1

# Graph-engine (Rust) S3 config:
NEBULA_S3_BUCKET=<your-bucket>
NEBULA_S3_ENDPOINT_URL=
NEBULA_S3_REGION=us-east-1

# Python service S3 config (must reference the same bucket):
S3_BUCKET_NAME=<your-bucket>
S3_GRAPH_STORAGE_BUCKET=<your-bucket>
S3_ENDPOINT_URL=

# Empty AWS_*/S3_* credentials → SDK falls through to instance/task role:
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
S3_ACCESS_KEY_ID=
S3_SECRET_ACCESS_KEY=
The = with no value (e.g. NEBULA_S3_ENDPOINT_URL=) is intentional — it tells the AWS SDK to resolve the regional endpoint instead of falling back to the in-stack MinIO URL, and tells boto3 to use the default credential chain (instance profile, ECS task role) instead of static keys. bootstrap.sh auto-detects NEBULA_USE_EXTERNAL_S3=1 and skips the in-stack minio + minio-init containers.

​
EKS: wire up via Helm values

The example values file at helm/examples/eks/values.yaml has this pre-wired: 
objectStorage:
  endpoint: ""              # empty → AWS SDK regional default
  bucket: <your-bucket>
  region: us-east-1
  forcePathStyle: false     # MUST be false for real AWS S3
  credentialsSecret: ""     # empty → SDK uses IRSA role on the SA

serviceAccount:
  create: true
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::<account-id>:role/nebula-irsa

​
Sanity checks

After re-bootstrapping with managed resources, verify:
CheckCommandExpected
App reaches RDScurl -fsS http://localhost:7272/v1/healthHealth endpoint returns OK
Migrations appliedAlembic head matches bundleTables collections, memories, entities exist in the nebula DB
pgvector loaded\dx vector in psqlshows vector | 0.x.x
S3 writes succeedIngest a small docNew objects appear under <your-bucket>/nebula-graphs/
Hatchet workflows runhttp://localhost:7274Dashboard shows enqueued tasks

​
Migrating an existing in-stack deploy to managed resources

If you’ve been running with in-stack Postgres + MinIO and want to move to RDS + S3 without losing data:
  1. Dump in-stack Postgres — docker exec -t <postgres-container> pg_dumpall -U postgres > nebula-dump.sql
  2. Restore into RDS — psql -h <rds-endpoint> -U postgres < nebula-dump.sql
  3. Copy MinIO contents to S3 — aws s3 sync s3://nebula-files/ s3://<your-bucket>/ --source-region us-east-1 --region us-east-1 (with appropriate MinIO/S3 credentials)
  4. Update .env.enterprise with the managed-resource overrides above
  5. Restart: ./enterprise/bootstrap.sh — the script will pick up the overrides and skip the in-stack services
Test on a staging deployment before doing this in production; the cutover involves a brief downtime window between the in-stack stop and the managed-resource start.