Local-First Architecture

Offline-first data management with real-time synchronization

The platform uses a local-first architecture built on SignalDB with IndexedDB persistence. All data interactions happen locally first, with bi-directional synchronization to the server via REST APIs.

Overview

  • Immediate Responsiveness: All reads and writes target local IndexedDB-backed collections. UI updates are instant.
  • Offline Functionality: Collections persist in IndexedDB across page reloads. When offline, users can browse cached data and make edits; changes are queued and pushed when connectivity returns.
  • Automatic Synchronization: The SyncManager handles incremental pull (via updatedAt timestamps) and push (local mutations).
  • Conflict Resolution: Server-side baseRevision checking detects concurrent edits. A Conflict Resolution UI lets users choose "Accept Theirs" or "Keep Mine".
  • Per-Entity Sync State: An EntitySyncStateService derives per-entity status (synced, localChanges, conflict).
graph TB
    subgraph "Client"
        UI[Svelte Components] --> Collections[SignalDB Collections]
        Collections --> IDB[IndexedDB Persistence]
        SM[SmallstackSyncManager] --> Collections
        SM --> API[REST API calls]
        NS[NetworkStatusService] --> SM
        ESS[EntitySyncStateService] --> SM
    end

    subgraph "Server"
        API --> Auth[Better Auth]
        Auth --> Endpoints[Generic CRUD Endpoints]
        Endpoints --> MongoDB
    end

Core Services

SignalDbService

Located at packages/client/src/services/signal-db.service.svelte.ts. The central service for data management.

Key responsibilities:

  • Creates and manages SignalDB collections with IndexedDB persistence adapters
  • Configures the SmallstackSyncManager for bi-directional sync
  • Tracks initial sync state per collection (initiallySynced)
  • Tracks last sync timestamps per collection (lastSyncedAt)
  • Manages sync conflicts and their resolution
  • Handles configurable polling for opted-in collections (default 5s; drops to 30s heartbeat when Ably is active)
// Getting a collection (creates it on first access, starts sync)
const collection = signalDbService.getCollection<MyEntity>("tenant-abc-types");

// With real-time polling enabled
const collection = signalDbService.getCollection<MyEntity>("tenant-abc-types", { realtime: true });

// Wait for initial data to be available
await signalDbService.isInitiallySynced("tenant-abc-types");

// Adjust polling interval (e.g., reduce to 30s heartbeat while Ably is active)
signalDbService.setPollingInterval(30_000);

// Check whether a collection has been registered
signalDbService.hasCollection("tenant-abc-types"); // boolean

// Check whether a collection has realtime polling enabled
signalDbService.isRealtimeCollection("tenant-abc-types"); // boolean

// Manually trigger a pull sync for a collection
await signalDbService.refreshCollection("tenant-abc-types");

SmallstackSyncManager

Located at packages/client/src/services/smallstack-sync-manager.ts. A subclass of SignalDB's SyncManager that exposes pending change information:

// Get IDs of entities with un-pushed local changes
const pendingIds: Set<string> = syncManager.getPendingChangeIds("my-collection");

// Get detailed pending changes for a specific entity
const changes = syncManager.getPendingChanges("my-collection", "entity-123");

// Get total pending change count across all collections
const count: number = syncManager.getPendingChangeCount();

NetworkStatusService

Located at packages/client/src/services/network-status.service.svelte.ts. Reactive online/offline detection using browser APIs:

import { networkStatusService } from "@smallstack/client";

// Reactive boolean ($state)
networkStatusService.isOnline; // true or false

// Listen for status changes
const unsubscribe = networkStatusService.onStatusChange((isOnline) => {
    console.log("Network status changed:", isOnline);
});

EntitySyncStateService

Located at packages/client/src/services/entity-sync-state.service.svelte.ts. Derives per-entity sync state:

import { entitySyncStateService } from "@smallstack/client";

// Returns: "synced" | "localChanges" | "conflict"
const state = entitySyncStateService.getState("entity-id", "collection-name");

// Total pending count
const count = entitySyncStateService.getTotalPendingCount();

Synchronization

How Pull Works

The SyncManager pull handler uses incremental sync:

  1. Initial sync (lastFinishedSyncEnd is null): Fetches all entities via GET /api/...?updatedAt=0, returns { items: [...] }
  2. Subsequent syncs: Fetches only entities modified since the last sync via GET /api/...?updatedAt=<timestamp>, categorizes into added/modified/removed based on revision and deletedAt
  3. Persistence: The SyncManager itself uses an IndexedDB persistence adapter (__signaldb_sync__), so sync metadata (snapshots, change tracking, sync operations) survive page reloads

How Push Works

Local mutations are captured by SignalDB's change tracking and pushed via:

  • Inserts: POST /api/... with the full entity
  • Updates: PATCH /api/.../entityId with { modifier, baseRevision }
  • Deletes: DELETE /api/.../entityId

Conflict Detection

The server checks baseRevision against the current document revision:

  1. Client sends { modifier, baseRevision: N } in PATCH request
  2. Server validates that the document's current revision === N
  3. If mismatch, server returns HTTP 409 with conflict details
  4. Client stores the conflict in signalDbService.conflicts
  5. User resolves via Conflict Resolution UI

Reconnect Behavior

When the network comes back online, networkStatusService.onStatusChange triggers syncManager.syncAll() to:

  • Push any queued local changes
  • Pull any server-side updates missed while offline

UI Components

OfflineBannerComponent

Fixed banner that appears when offline, auto-dismissing "Back online" notification on reconnect.

ConflictResolutionContainer / ConflictResolutionDialog

Renders active conflicts from signalDbService.conflicts as side-by-side cards with "Accept Theirs" / "Keep Mine" buttons.

EntitySyncBadge

Inline badge showing conflict or pending-changes status for a specific entity. Used in the entity editor.

PendingChangesCounter

Displays total pending change count. Placed in both app layouts.

StaleBadgeComponent

Shows per-collection staleness indicators based on lastSyncedAt timestamps.

Collection Naming & API Resolution

Collections are named using a convention that maps to API paths:

tenant-{projectId}-{typeId}     → /api/projects/{projectId}/types/{typeId}/data
tenant-{projectId}-types        → /api/projects/{projectId}/types
tenant-{projectId}-applications → /api/projects/{projectId}/applications

The describeCollection() utility from @smallstack/shared handles this mapping.

Type-Level Sync Configuration

A Type can opt out of server synchronization entirely via the syncType field. The field is configurable in the type editor.

Value Behavior
"remote" Default. Entities sync to the server via the standard pull/push flow.
"localOnly" Entities live only in IndexedDB. The SyncManager never POSTs them to the server.

"localOnly" is intended for ephemeral or device-bound state — draft caches, UI preferences, scratch data. Choose it when the data should never leave the browser.

Data-loss caveat: "localOnly" data lives only in the current browser profile. Clearing site data, switching browsers, or signing in on another device wipes it. There is no server-side recovery. If a type is later switched back to "remote", data that accumulated while it was "localOnly" is not retroactively pushed to the server and is permanently lost.

Offline Auth

When offline, server-side auth hooks (hooks.server.ts) catch network errors and allow the request to proceed with the locally cached session. Better Auth uses database-backed sessions with a 30-day expiry; the locally cached session remains valid offline up to that absolute expiry, not for 30 days from the moment the device went offline. A session already close to its expiry will still expire while offline.

Application-level switch: the services above keep data available offline. To keep the application UI itself working offline, an application must opt in via its Offline Capable setting, which eagerly preloads every widget chunk after the first page settles. It is off by default. See Offline capability under Applications.

Real-Time Updates

When an ABLY_API_KEY is configured server-side, the platform uses Ably-driven push notifications to trigger collection syncs instantly when data changes elsewhere — replacing the continuous 5s polling loop with a 30s safety-net heartbeat.

Without Ably (dev environments or missing key), polling remains at the default 5s interval.

Ably-driven sync behaviour

  • Realtime collections ({ realtime: true }) auto-sync silently within ~1.5 s of a remote change.
  • Manual collections (no realtime flag) show a badge on the refresh button so users decide when to reload.
  • Self-notifications are filtered: changes made in the current browser window never trigger a redundant re-fetch.
  • If Ably is unavailable (e.g. token endpoint returns 503), the system continues on polling with no errors.

CollectionSyncNotificationService

Located at packages/client/src/services/collection-sync-notification.service.svelte.ts. Subscribes to the Ably project:{projectId}:sync channel and coordinates with SignalDbService.

// Subscribe when entering a project context (called from layout)
const connected = await syncNotificationService.subscribeToProject(projectId, userId);
if (connected) signalDbService.setPollingInterval(30_000); // reduce heartbeat while Ably is active

// Unsubscribe and restore polling on teardown
await syncNotificationService.unsubscribeFromProject();
signalDbService.setPollingInterval(5000);

// Check whether a manual collection has unseen remote changes
syncNotificationService.hasRemoteChanges("tenant-abc-contacts"); // boolean ($state)

// Clear the badge after a manual refresh
syncNotificationService.clearRemoteChanges("tenant-abc-contacts");

Collections opt in to the polling-based fallback as before:

signalDbService.getCollection("my-collection", { realtime: true });
// or
signalDbService.startRealtime("my-collection");

Best Practices

  1. Use getCollection() lazily: Collections sync on first access. Don't pre-register collections you don't need yet.
  2. Wait for initial sync: Use await signalDbService.isInitiallySynced(name) before rendering data-dependent UI.
  3. Soft deletes: Entities use deletedAt field. The sync system tracks deletions via this field.
  4. baseRevision: Always include baseRevision in PATCH requests for conflict detection.
  5. Avoid raw fetch for synced data: Use collection.find() / collection.findOne() for reads. The sync layer handles server communication.

Debugging

In development, you can inspect:

  • IndexedDB: Open Browser DevTools → Application → IndexedDB to see collections and the __signaldb_sync__ metadata stores
  • Console logs: Collection creation and sync operations are logged at debug level
  • SignalDB DevTools: Uncomment the devtools import in signal-db.service.svelte.ts
Markdown | llm.txt