gobridge

Configuration Stores

This guide documents the configuration store implementations that load, watch, and persist BridgeConfig documents. For the YAML field reference, see Configuration Reference. For the high-level lifecycle overview, see Configuration Overview.

Architecture

GoBridge separates configuration concerns into composable contracts:

classDiagram
    class Loader {
        <<interface>>
        +Load(ctx) BridgeConfig, error
    }
    class Watcher {
        <<interface>>
        +Watch(ctx) chan BridgeConfig, error
    }
    class Reloader {
        <<interface>>
    }

    Reloader --|> Loader
    Reloader --|> Watcher

    class FileSource {
        +Load(ctx) BridgeConfig, error
    }
    class FileWatcher {
        +Watch(ctx) chan BridgeConfig, error
        +Stop()
    }
    class DynamoDBLoader {
        +Load(ctx) BridgeConfig, error
        +Watch(ctx) chan BridgeConfig, error
        +Save(ctx, cfg) error
        +EnsureTable(ctx) error
    }
    class Manager {
        +Load(ctx) BridgeConfig, error
        +Watch(ctx) chan BridgeConfig, error
        +Stop()
    }

    FileSource ..|> Loader
    FileWatcher ..|> Watcher
    DynamoDBLoader ..|> Reloader
    Manager ..|> Reloader
    Manager o-- Loader : base + overlays
    Manager o-- Watcher : watches all layers

The loader/watcher contracts live in ports (ports/blueprint_loader.go) and operate on *ports.BridgeConfig. Adapters and external consumers implement them directly; the config package owns only the orchestration and on-disk write path:

Package Types Role
ports Loader, Watcher, Reloader, BridgeConfig Contracts implemented by adapters, consumed by Manager and httpapi
config Manager, Layer, MergeFunc, DefaultMerge, WriteFile Layered orchestration, merge strategy, atomic file persistence

This keeps the contracts free of any config dependency.

The admin API consumes ports.ConfigStore for load, save, validation, and merge. Save assigns the current stored version plus one (or 1 for a new document), rather than trusting the incoming BridgeConfig.Version. A successful save updates the caller’s Version; a failed save leaves it unchanged. Restoring earlier content is another commit and advances the version too.

parser.FileStore implements this contract with atomic file replacement. It still requires a single writer: the version read and the replacement are not an atomic compare-and-swap. parser.WriteFile is the lower-level serialization helper and preserves the supplied version.

File Source

Package: github.com/mariotoffia/gobridge/adapters/native/config/file

Reads a YAML or JSON configuration file from disk. Format is auto-detected from the file extension (.yaml, .yml, .json), or can be overridden.

API

import (
    fileconfig "github.com/mariotoffia/gobridge/adapters/native/config/file"
    "github.com/mariotoffia/gobridge/ports"
)

// registry carries the plugin decoders (transports, stores, processors) that
// decode each options block. It is a required argument; build it once at the
// composition root and register each adapter's decoder.
source := fileconfig.NewSource("bridge.yaml", registry)
cfg, err := source.Load(ctx)

Options

Option Description
WithSourceFormat(f) Override format auto-detection (parser.FormatYAML, parser.FormatJSON)

Behaviour

File Watcher

Package: github.com/mariotoffia/gobridge/adapters/native/config/file

Watches a configuration file for changes and re-parses it when modifications are detected. Supports two watch modes.

API

watcher := fileconfig.NewWatcher("bridge.yaml", registry,
    fileconfig.WithWatchConfig(baseCfg.ConfigWatch),
    fileconfig.WithLogger(logger),
)
ch, err := watcher.Watch(ctx)
// ch receives *ports.BridgeConfig on each detected change

Options

Option Description Default
WithMode(m) ModeNotify (hybrid: directory-scoped fsnotify + periodic hash-resync backstop) or ModePoll (SHA-256 hash) ModeNotify
WithDebounce(d) Debounce interval for ModeNotify 100ms
WithPollInterval(d) Poll interval for ModePoll 30s
WithResyncInterval(d) Notify-mode hash-reconciliation cadence: periodic SHA-256 comparison catching changes fsnotify missed. Non-positive values are ignored 30s
WithBaselineHash(h) Seed the change-detection baseline with the hash the caller actually loaded (Source.LoadHash), closing the Load↔Watch race hash file at Watch
WithClock(c) Inject the clock used for timers and tickers (nil ignored) clock.System
WithFormat(f) Override format auto-detection FormatAuto
WithLogger(l) Logger for diagnostics nil
WithWatchConfig(def) Apply settings from a ConfigWatchDef (from the YAML config_watch section) –

Watch Modes

Mode Mechanism Best for
Notify (default) Hybrid: filesystem events via fsnotify on the containing directory, debounced to coalesce rapid writes, plus a periodic SHA-256 hash-resync backstop (default 30s) that catches changes fsnotify missed — this makes Kubernetes ConfigMap ..data symlink swaps and kernel event-queue overflow safe Local disks and K8s ConfigMap volume mounts, fast change detection
Poll Periodic SHA-256 content hash comparison NFS, EFS, network mounts, subPath mounts

Behaviour

YAML-Driven Configuration

The watcher can be configured directly from the YAML config file itself:

config_watch:
  mode: notify        # hybrid: fsnotify + hash-resync backstop
  poll_interval: 30s  # notify mode: resync cadence; poll mode: poll cadence
  debounce: 200ms     # for notify mode

In notify mode poll_interval doubles as the hash-resync cadence. Pass this to the watcher with WithWatchConfig(baseCfg.ConfigWatch).

DynamoDB Loader

Package: github.com/mariotoffia/gobridge/adapters/aws/config/dynamodb

Stores the full BridgeConfig as a single DynamoDB item with version-based change detection. Useful for centralized configuration management in AWS environments.

A DynamoDB layer is a base, not an overlay on a file. Wire it as the config source programmatically, or select config_source: dynamodb in the shipped AWS deployment profile’s bootstrap config. Single and DynamoDB HA support it; filesystem_replicated supports file only. The profile runs exactly one base layer, either file (the default) or dynamodb, and gives the admin API that same source’s store. It does not support overlays: an overlay changing underneath a transaction could make the running config differ from the document the API commits. See AWS configuration and Overlays and the admin config API do not compose.

API

import ddbconfig "github.com/mariotoffia/gobridge/adapters/aws/config/dynamodb"

loader := ddbconfig.NewLoader(ddbClient,
    ddbconfig.WithTableName("gobridge-config"),
    ddbconfig.WithBridgeID("production"),
    ddbconfig.WithPollInterval(30 * time.Second),
)

// Load
cfg, err := loader.Load(ctx)

// Watch for changes
ch, err := loader.Watch(ctx)

// Save (admin/test tooling)
err = loader.Save(ctx, cfg)

// Create table if missing (dev/test)
err = loader.EnsureTable(ctx)

Options

Option Description Default
WithTableName(name) DynamoDB table name "gobridge-config"
WithBridgeID(id) Bridge identifier (partition key prefix) "default"
WithPollInterval(d) Watch polling interval in ModePoll 30s
WithWatchMode(m) ModePoll or ModeStreams change detection ModePoll
WithStreamPollInterval(d) GetRecords interval in ModeStreams —
WithStreamsClient(c) DynamoDB Streams client (required for ModeStreams) nil
WithRegistry(r) Plugin registry used to decode the stored config’s options blocks nil

Watch Modes

Mode Mechanism Notes
Poll (default) One strongly-consistent GetItem per instance per interval, comparing the version attribute Predictable cost; use for clustered deployments
Streams DynamoDB Streams GetRecords, sub-second propagation Streams throughput is ~5 GetRecords/sec per shard shared across all consumers; falls back to ModePoll (with a warning) when a streams client is absent or streams are not enabled on the table. Failure semantics: a throttled GetRecords keeps the iterator (no LATEST reset) and sheds load via equal-jittered exponential backoff up to 30s; a genuinely invalid iterator (or 3 consecutive unknown failures) is re-acquired at LATEST followed by a version-check reconciliation covering the gap; 5 consecutive acquisition failures switch to poll fallback for the rest of the Watch with a single Warn

Table Schema

Attribute Type Description
PK (partition key) String "config#<bridge-id>"
SK (sort key) String "current"
data String Full BridgeConfig serialized as JSON
version Number Monotonically increasing version counter

The table uses pay-per-request billing. EnsureTable creates the table idempotently (safe to call multiple times). When the loader runs in ModeStreams, EnsureTable provisions the new table with a KEYS_ONLY stream specification so a self-provisioned deployment actually gets the streams-based Watch it configured; an existing table’s stream settings are left untouched.

Behaviour

Cost Characteristics

Operation DynamoDB Cost
Watch poll (no change) ~1 RCU per poll (strongly-consistent read of the item, ≤ 4 KB)
Watch poll (change detected) ~1 RCU + full item re-parse
Save 1 WCU per save

Configuration Manager

Package: github.com/mariotoffia/gobridge/config

The Manager orchestrates multiple configuration sources in a layered stack. A base layer is loaded first, then overlays are merged on top in order. The merged result is validated before being returned.

API

mgr := config.NewManager(
    config.Layer{Name: "file", Loader: fileSource, Watcher: fileWatcher},
    config.WithOverlay(config.Layer{Name: "env", Loader: ddbLoader, Watcher: ddbLoader}),
    config.WithManagerLogger(logger),
)

// Load all layers, merge, validate
cfg, err := mgr.Load(ctx)

// Watch all layers for changes, re-merge on any change
ch, err := mgr.Watch(ctx)
defer mgr.Stop()

Options

Option Description
WithOverlay(layer) Add an overlay layer (applied in registration order)
WithMergeFunc(fn) Override the default merge strategy
WithManagerLogger(l) Logger for diagnostics

Layer Structure

type Layer struct {
    Name    string
    Loader  Loader   // required
    Watcher Watcher  // optional (nil if source doesn't support watching)
}

Merge Rules (DefaultMerge)

Section Merge Behaviour
bridge Overlay non-zero fields replace base; cluster replaces base when non-nil (endpoint map cloned)
config_watch Overlay replaces base entirely if non-nil
http Field-level – non-empty overlay scalar fields win, empty fields keep the base value; the admin_api_key / monitor_api_key secrets are preserved when the overlay omits them or echoes back the "[REDACTED]" marker (a partial PATCH never wipes a configured key)
stores Overlay replaces per role (lease, outbox, dlq individually)
sessions, receivers, senders, bindings Merge by ID – new IDs append; a matching ID is merged field-level, and the base entry’s typed plugin Config (broker URL, credentials, options) is carried forward unless the overlay changes the transport/discriminator
routes Merge by ID – new IDs append; a matching ID is wholesale-replaced (routes carry no plugin Config, so nothing can be lost)

Behaviour

Layered Configuration Patterns

flowchart TD
    Base["Base Layer<br/>(file: defaults.yaml)"]
    Env["Overlay: Environment<br/>(DynamoDB: staging)"]
    Inst["Overlay: Instance<br/>(file: instance.yaml)"]

    Base --> M[DefaultMerge]
    Env --> M
    Inst --> M
    M --> V[Validate]
    V --> Merged["Merged BridgeConfig"]

Example: File base + DynamoDB overlay

fileSource := fileconfig.NewSource("defaults.yaml", registry)
fileWatcher := fileconfig.NewWatcher("defaults.yaml", registry,
    fileconfig.WithMode(fileconfig.ModePoll),
    fileconfig.WithPollInterval(30*time.Second),
)

ddbLoader := ddbconfig.NewLoader(ddbClient,
    ddbconfig.WithBridgeID("staging"),
    ddbconfig.WithRegistry(registry),
)

mgr := config.NewManager(
    config.Layer{Name: "defaults", Loader: fileSource, Watcher: fileWatcher},
    config.WithOverlay(config.Layer{Name: "ddb-staging", Loader: ddbLoader, Watcher: ddbLoader}),
)

cfg, _ := mgr.Load(ctx)
watchCh, _ := mgr.Watch(ctx)

Config Persistence (Write)

Package: github.com/mariotoffia/gobridge/config/parser

The parser.WriteFile function provides atomic YAML writes with permission preservation:

err := parser.WriteFile("bridge.yaml", cfg)

HTTP Admin API (Config Transactions)

The HTTP admin API provides transactional config editing over the file-based store:

Endpoint Method Description
/api/v1/admin/config GET Read the current effective config (redacted)
/api/v1/admin/config/transactions POST Open a transaction against the current version
/api/v1/admin/config/transactions/{txnID} GET Preview the merged config
/api/v1/admin/config/transactions/{txnID} PATCH Apply a config overlay (merge)
/api/v1/admin/config/transactions/{txnID}/commit POST Validate, CAS-check version, write to disk
/api/v1/admin/config/transactions/{txnID} DELETE Roll back (discard) the transaction

The transaction manager uses config.DefaultMerge to apply patches and config.WriteFile for atomic commits with version-based CAS (compare-and-swap) to prevent lost updates.

See the HTTP API Reference for the full endpoint table, status codes, and merge semantics, and Credentials & HTTP API for authentication.

Custom Configuration Sources

Implement the ports.Loader and optionally ports.Watcher interfaces:

type Loader interface {
    Load(ctx context.Context) (*ports.BridgeConfig, error)
}

type Watcher interface {
    Watch(ctx context.Context) (<-chan *ports.BridgeConfig, error)
}

Guidelines:

Example: Consul-backed loader (sketch)

type ConsulLoader struct {
    client   *consulapi.Client
    registry *ports.Registry
}

func (l *ConsulLoader) Load(ctx context.Context) (*ports.BridgeConfig, error) {
    kv, _, err := l.client.KV().Get("gobridge/config", nil)
    if err != nil { return nil, err }
    return parser.Parse(bytes.NewReader(kv.Value), parser.FormatJSON, l.registry)
}

Capability Matrix

Implementation Load Watch Persist Watch Mechanism
file.Source Yes – – –
file.Watcher – Yes – Hybrid fsnotify + hash-resync, or SHA-256 poll
dynamodb.Loader Yes Yes Yes (Save) Version-number poll (default) or DynamoDB Streams (opt-in, auto-fallback to poll)
config.Manager Yes Yes – Multiplexes all layer watchers
config.WriteFile – – Yes –
Document Description
Configuration Overview Lifecycle, layering, bootstrap pattern
Configuration Reference Every YAML/JSON field documented
Transport Configuration MQTT, SQS, AMQP, Azure SB, HTTP transport options
Credentials & HTTP API Secret management and admin API
Scenario: Layered DynamoDB Config Walkthrough of file + DynamoDB layering
Scenario: Dynamic Reconfiguration Live config reload patterns
This site is open source. Improve this page.