Skip to main content

Upgrades, Backups, and Recovery

Updating Aivory should cover four phases: decide the target version, create a restorable backup, deploy and verify, and roll back if necessary. Do not mix pulling the latest image, deleting containers, and deleting data volumes into a single operation.

The current application branch is based on v2.4.7 and includes the post-release domain enrollment changes. Its startup migration creates the passkeys, registration_domains, and domain_users tables automatically; no manual SQL is required.

Pin Release Versions

Testing may use IMAGE_TAG=latest, but stable environments should pin a fully released semantic version, for example:

IMAGE_TAG=3.0.0

Image tags carry no v prefix. A full deployment makes the app, the sandbox sidecar, and the sandbox runtime share the same version tag; before upgrading, confirm all three images can be pulled, to avoid mixing partially published versions.

Render the actual current configuration rather than checking only the template:

cd /opt/aivory/deploy
docker compose --env-file .env -f docker-compose.prod.yml config --images

For the Personal edition, replace .env and the Compose file with .env.personal and docker-compose.personal.yml. If the Personal edition's local sandbox profile is enabled, the commands also need --profile sandbox.

For historical versions predating unified sandbox version tags, you may explicitly use a compatibility override, for example:

IMAGE_TAG=2.2.6
SANDBOX_IMAGE_TAG=latest

Use it only when the target historical version genuinely has no matching sandbox image; do not pin SANDBOX_IMAGE_TAG=latest long-term on newer versions, or you may silently mix different versions of the app and the sandbox.

Pre-Upgrade Checks

Before every upgrade, complete the following:

  1. Read the target version's release notes and check for database migrations, environment variable changes, or feature deprecations.
  2. Record the current IMAGE_TAG, the deployment mode, container status, and a baseline of key features.
  3. Create a backup and confirm it is accessible; do not rely solely on "the backup job was submitted".
  4. Confirm DATA_DIR and the named volumes still have enough free disk space, and that the certificates and domain are healthy.
  5. For instances with real users, work within a maintenance window or during off-peak hours, and have a definite rollback version ready.
  6. If domain enrollment is enabled, export the current domain rules and record which workspaces are locked. Plan a test registration and a locked-user switch attempt after the upgrade.

Do not run docker compose down -v before upgrading. -v deletes named volumes, which may delete PostgreSQL, Redis, Qdrant, or sandbox archive data.

Updating the Personal Edition

The Personal edition has only the app container by default:

cd /opt/aivory/deploy
docker compose --env-file .env.personal -f docker-compose.personal.yml config --images
docker compose --env-file .env.personal -f docker-compose.personal.yml pull
docker compose --env-file .env.personal -f docker-compose.personal.yml up -d --no-build
docker compose --env-file .env.personal -f docker-compose.personal.yml ps
docker compose --env-file .env.personal -f docker-compose.personal.yml logs --tail=200 app

If the local sandbox is enabled, add --profile sandbox to every Compose command. The Personal edition's data lives in DATA_DIR: the SQLite database, vectors, uploaded files, artifacts, and background backups must all be preserved together.

Updating the Full Edition

cd /opt/aivory/deploy
docker compose --env-file .env -f docker-compose.prod.yml config --images
docker compose --env-file .env -f docker-compose.prod.yml pull
docker compose --env-file .env -f docker-compose.prod.yml up -d --no-build
docker compose --env-file .env -f docker-compose.prod.yml ps
docker compose --env-file .env -f docker-compose.prod.yml logs --tail=200 app

The first pull or an upgrade of the sandbox runtime image can be slow. sandbox-image-keepalive exists to keep the runtime image referenced so that image-pruning tasks do not leave the next Python call with a cold pull and a timeout; do not remove it just because it looks idle.

Post-Upgrade Acceptance

Containers showing as running is not enough. Perform at least these checks:

curl -fsS http://127.0.0.1/api/health

Then verify in a browser:

  • Administrators and regular users can log in and existing sessions are still accessible.
  • The default model completes a short conversation; channels and the model selector work normally.
  • Upload a small file and confirm the preview or download works.
  • With a knowledge base present, retrieve a known item; on the Full edition, also check Qdrant status.
  • With Python enabled, run a harmless small task to confirm the sandbox works.
  • With OAuth, email, or payments enabled, at least verify the corresponding callback/send/order flow on a test account.
  • With passkeys enabled, verify one existing passkey login and device removal from Account settings on HTTPS (or localhost).
  • With domain enrollment enabled, verify a matching registration joins the configured workspace, a locked user cannot switch to personal space, and an explicit personal-space override still works.

When you find a problem, first save redacted logs from the app and related services. If the issue is tied to the new version and cannot be fixed quickly, follow the rollback procedure below; do not experiment with deletions, imports, or forced rebuilds on production data.

Rollback

Change IMAGE_TAG in the actual env file to the previous fully released version you have verified, then repeat config --images, pull, and up -d --no-build. If the rollback crosses versions that included data migrations, check the release notes first: a rollback-able application binary does not mean the new data structures safely roll back too.

When post-upgrade data writes have already occurred and compatibility is unconfirmed, prefer restoring the pre-upgrade backup in an isolated environment rather than blindly running the old image. After rolling back, run the full acceptance pass again.

Backup Scope

Deployment modeWhat must be protectedNotes
Personal editionThe entire DATA_DIRContains aivory.db, embedded vectors, uploads, artifacts, local objects, passkey credentials, domain rules, and backup archives
Full editionpgdata, redisdata, qdrantdata, sandbox-archives, DATA_DIRRelational data (including passkeys and domain bindings), vectors, cache persistence, and durable workspaces and files must stay consistent

The admin "Backup and Migration" page can create a migration-oriented full archive, typically including a logical database backup, vectors, and optional files; "Configuration Export" copies admin settings. Both may contain API keys and OAuth/SMTP/storage/payment credentials, so they must be encrypted at rest, access-restricted, and never uploaded to tickets, public repositories, or chat logs.

Keep both physical backups and admin logical backups whenever possible: the former for full disaster recovery, the latter for easier cross-server or cross-deployment-mode migrations. Archives produced by backup jobs also live in persistent locations and should not be kept only inside temporary containers about to be replaced.

Recovery Principles and Procedure

Recovery is a destructive write operation. Before starting, stop new writes, save an independent copy of the current deployment, record the original version and the recovery target, and — if possible — rehearse in an isolated environment first.

Recommended order:

  1. Prepare an empty deployment on the target host that matches or is compatible with the backup, and confirm it can start.
  2. Stop writes on the target application and preserve its existing data directory and volume snapshots.
  3. Restore the full archive through the admin console, or restore the data directory and all relevant volumes using whatever backup tool you use.
  4. Start the services and review the database, vector, application, and sandbox logs.
  5. Verify the health endpoint, admin login, normal conversations, file downloads, knowledge base retrieval, workspace permissions, domain enrollment/lock exceptions, passkey login (if enabled), and any required integrations.
  6. Only after acceptance is complete should you update DNS, shut down the old instance, or clean up old data.

Do not hand a SQLite file actively used by a Personal edition to multiple containers or multiple servers; do not restore PostgreSQL while leaving behind Qdrant and files; do not treat Redis, Qdrant, or sandbox archive volumes as disposable "cache" unless you have assessed the recovery consequences.

Backup Drills and Retention

At least periodically, restore a recent backup in a non-production environment. Record the time and disk space required, any missing items, and the acceptance results; when a real failure happens, that information is more valuable than "the backup succeeded yesterday".

A retention policy should satisfy business, regulatory, and cost requirements at once: keep multiple points in time, at least one off-host or cross-region copy, encrypted archives, access auditing, and regular deletion of expired backups. Requests to delete users or workspaces must also be explicitly reconciled with backup retention rules.