Docker Observability Language Specification

Status: stable

DOL is a domain-specific language for querying, observing, and analyzing Docker infrastructure. The language defines a rich, implementable core that can be parsed into a typed AST and executed by a Rust CLI.

1. Design Goals

DOL treats Docker as a live data system:

DOL improves the day-to-day workflow where users otherwise combine docker ps, docker stats, docker events, shell scripts, Prometheus queries, and dashboard tools. The language does not replace Prometheus or Grafana; it provides a Docker-native query surface that can later export to those systems.

The language prioritizes:

2. Query Families

DOL has nine top-level query families.

2.1 observe

observe reads the current state of Docker entities and optionally enriches containers with current metrics.

Execution mode: batch snapshot by default, hybrid when metric windows are requested.

Simple queries:

observe containers
observe containers where status = running
observe containers | where image contains "postgres" | select name, status, ports

Cross-target JOIN:

A JOIN clause merges rows from two targets on a matching key:

observe containers join images on image = repository
observe containers join networks on name = name
observe containers join volumes on scope = scope

Syntax:

observe <left-target> join <right-target> on <left-key> = <right-key> [ | <pipeline> ]

Execution: nested-loop join over all matching right-target rows for each left-target row; equality comparison via =.

Examples:

observe containers join images on image = repository + ":" + tag
observe containers join networks on id = id
observe containers join images on id = id | select c.name, i.repository
observe containers join images on id = id | where c.image = "nginx:latest"
observe containers join volumes on state = scope

2.2 events

events subscribes to Docker event streams or reads historical events when a time range is present.

Execution mode: stream by default, historical when last, from, or to is present and a telemetry store is configured.

Examples:

events containers
events containers where action = "die"
events containers | where action = "restart" | select time, container, image

2.3 inspect

inspect reads a detailed entity snapshot. With at, it reads a historical snapshot from storage.

Execution mode: batch for current inspection, historical for at.

Examples:

inspect container api-service
inspect image postgres:16
inspect container api-service at "2026-01-01 12:00:00"

2.4 logs

logs retrieves log output from a running container. Returns lines with line numbers, message content, and container name.

Execution mode: batch by default; streaming when a pipeline is present. In streaming mode (follow: true), logs are delivered in real-time until Ctrl+C or --timeout is reached. Pipeline nodes (where, select, limit) apply to the live stream.

Examples:

logs container my-app
logs container my-app tail 50
logs container my-app | where message contains "error" | select line, message

2.5 ping

ping tests connectivity to the Docker daemon. Returns a status field (ok or error) and a human-readable message.

Execution mode: batch.

Examples:

ping

2.6 analyze

analyze runs deterministic analysis over current or stored Docker telemetry.

Execution mode: batch or historical depending on time qualifiers.

Examples:

analyze containers find anomalies
analyze containers find restart_loops last 10m
analyze containers find dependencies
analyze containers find density
analyze containers find leaks
analyze containers find drift
analyze container api-service correlate events last 1h

Analysis types: - anomalies (default) — detects high CPU, memory pressure, restart loops, and unhealthy states. - correlate — finds containers sharing images and labels. - explain — produces a diagnostic signal summary for one or all containers. - dependencies — maps compose project, network, and volume relationships. - density — container distribution across images, states, and compose projects. - leaks — detects memory usage growth trends from historical metrics (requires --store). - drift — compares two telemetry snapshots to detect image, state, label, or restart count changes (requires --store).

2.7 fields

fields returns the schema (field names and types) for a given entity type. This is useful for discovery and tooling integration.

Execution mode: batch.

Examples:

fields containers
fields images
fields networks
fields volumes

2.8 compose

compose queries containers, networks, volumes, and other resources grouped under a Docker Compose project. It filters resources by the com.docker.compose.project label.

Execution mode: batch for most targets; streaming for logs and networks (real-time log and network event streams).

Syntax:

compose ls
compose <project>
compose <project> containers
compose <project> services
compose <project> networks
compose <project> volumes
compose <project> health
compose <project> images
compose <project> stats
compose <project> ps
compose <project> logs <service> [tail <n>]
compose <project> port <service> <port>
compose <project> config [services|networks|volumes]
compose <project> events
observe compose <project>

Examples:

compose ls
compose ls | where containers > 5 | sort by project asc
compose myapp
compose myapp services
compose myapp networks
compose myapp volumes
compose myapp health
compose myapp images
compose myapp stats | where cpu > 80% | select name, service, cpu
compose myapp ps | where state = "running" | select name, service, health
compose myapp logs api-service tail 50
compose myapp logs api-service tail 100 | where message contains "error"
compose myapp port api-service 8080
compose myapp config services
compose myapp config networks
compose myapp config volumes
compose myapp | where cpu > 80% | select name, cpu
compose myapp health | where health = "unhealthy" | select name, service, health
alert when compose_project = 'myapp' and cpu > 85% for 2m then print "High CPU"

2.9 alert

alert defines a continuously evaluated condition and an action.

Execution mode: stream or scheduled evaluation loop.

Alert actions are executed in real time: - print: Outputs a formatted message to stdout. - webhook: Sends an HTTP POST to the specified URL. Requires network access. - restart: Restarts the target container via the bollard native Docker API (restart_container()), without shelling out to the docker CLI.

When --store is active, all fired alerts are persisted to the alert_history table for audit and review.

Examples:

alert when cpu > 85% for 2m then print "High CPU"
alert when restart_count > 3 for 5m then print "Restart loop detected"
alert when memory > 90% for 1m then webhook "http://localhost:9000/hooks/docker"
alert when restart_count > 5 for 3m then restart container api-service

3. Targets

Top-level collection targets:

Singular inspection targets:

Target names are case-sensitive when they refer to Docker names or IDs. DOL keywords are lowercase in v0.2.

4. Fields

4.1 Container Fields

Common container fields:

Individual labels can be accessed with dot notation. For example, given a container with label com.docker.compose.project=myapp:

observe containers where label.com.docker.compose.project = "myapp"
observe containers | where label.env = "production" | select name, label.version

4.2 Image Fields

Common image fields:

4.3 Network Fields

Common network fields:

4.4 Volume Fields

Common volume fields:

4.5 Event Fields

Common event fields:

4.6 Dynamic Fields via set

The set pipeline node adds or overrides a field on each row. The value can be:

Examples:

observe containers | set tier = "prod"
observe containers | set mem_gb = memory / 1073741824 | select name, mem_gb
observe containers | set label = upper(name) | select name, label
observe containers | set health = if state = running then "up" else "down"
observe containers | set severity = case
    when cpu > 80% then "critical"
    when cpu > 50% then "warning"
    else "ok"
  end | select name, severity

5. Literals and Types

DOL v0.2 supports these literal types:

Bare identifiers are accepted for simple values in filters, but strings are recommended when a value contains punctuation, spaces, or mixed case.

5.1 Static Semantic Analysis & Type Safety

DOL implements a static semantic analyzer and type safety validation pass before executing any query. This phase runs immediately after parsing and prevents execution of queries that contain structural or type errors:

6. Operators

6.1 Comparison Operators

6.2 String and Pattern Operators

6.3 Range and Null Operators

6.4 Arithmetic Operators

Arithmetic expressions can be used in set assignments, where filters, having filters, and anywhere an expression is expected.

6.5 Boolean Operators

6.6 Function Calls

String functions: - upper(s) — returns uppercase - lower(s) — returns lowercase - length(s) — returns Integer string length - trim(s) — returns trimmed whitespace - concat(a, b, ...) — returns string concatenation of all arguments - substring(s, start, len) — returns substring extraction - coalesce(a, b, ...) — returns first non-null, non-empty value - starts_with(s, prefix) — returns Boolean, true if s starts with prefix - ends_with(s, suffix) — returns Boolean, true if s ends with suffix - replace(s, from, to) — returns string with all occurrences of from replaced with to - reverse(s) — returns reversed string - repeat(s, n) — returns string repeated n times - position(s, substr) — returns Integer, 0-based index of first substr occurrence (0 if not found) - split_part(s, delim, n) — returns split by delim, the n-th part (1-indexed)

Type conversion functions: - to_int(value) — converts to Integer (parses strings, truncates floats, maps true/false to 1/0) - to_float(value) — converts to Float (parses strings, casts integers) - to_string(value) — converts any value to its string representation

Date/time functions: - now() — returns current UTC timestamp as RFC 3339 string - date_format(ts, fmt) — format a timestamp string according to fmt (strftime syntax, e.g., %Y-%m-%d) - date_diff(a, b, unit) — returns Integer difference between two timestamps in the given unit (seconds, minutes, hours, days) - extract(ts, part) — returns Integer part of a timestamp: year, month, day, hour, minute, second

6.7 Coalesce Function

The coalesce() function returns the first non-null, non-empty value from its arguments:

observe containers | set name = coalesce(label.name, name, "unknown") | select name

If all arguments are null or empty strings, coalesce() returns null.

6.8 $var Field References

Field names can be prefixed with $ for explicit field access. This is useful when a field name might otherwise be parsed as a literal value:

observe containers where $state = running
observe containers | set $cpu = cpu

$state is equivalent to state — the $ prefix is stripped during parsing and the result is treated as a field reference.

6.9 Operator Precedence

Precedence, highest to lowest:

  1. Parentheses: ( ... )
  2. Function calls, unary -
  3. Arithmetic: *, /, %
  4. Arithmetic: +, -
  5. Comparison, contains, matches, starts_with, ends_with, between, is null, is not null
  6. not
  7. and
  8. or

Example:

observe containers where status = running and (cpu > 80% or memory > 90%)
observe containers where image in ("postgres", "mysql", "redis")
observe containers where name matches "^api-"
observe containers where cpu between 50 and 80
observe containers | where finished_at is not null | select name
observe containers | set mem_gb = memory / 1073741824 | select name, mem_gb
observe containers | where upper(name) contains "API" | select name

7. Time Syntax

DOL v0.2 supports relative windows, point-in-time inspection, and full historical ranges.

Relative windows:

last 5m
last 1h
last 2d

Point-in-time:

at "2026-01-01 12:00:00"

Historical ranges are fully supported via the telemetry store:

from "2026-01-01 12:00:00" to "2026-01-01 13:00:00"

Time semantics:

8. Pipeline Syntax

Pipelines transform query output from left to right.

Supported pipeline nodes:

Pipeline rules:

Examples:

observe containers | where cpu > 80% | select name, cpu, memory
observe images | sort by size desc | limit 10
events containers | where action = "die" | group by image
observe containers | set health = if state = running then "healthy" else "unhealthy"
observe containers | if cpu > 90% then alert "Critical CPU" else alert "OK"
observe containers | set severity = case when cpu > 80% then "critical" else "ok" end
observe containers | set mem_gb = memory / 1073741824 | select name, mem_gb
observe containers | sort by state desc, cpu desc | select name, state, cpu
observe containers | distinct | select image
observe containers | sort by name asc | offset 10 | limit 5
observe containers | group by image with count(id) as cnt | having cnt > 3
observe containers | group by image with avg(cpu) as avg_cpu | sort by avg_cpu desc
observe containers | group by image with median(cpu) as med_cpu
observe containers | group by image with percentile(cpu, 95) as p95_cpu
observe containers | fill memory with 0 | where memory > 500
observe containers | where name starts_with "api-"
observe containers | where image ends_with ":latest"
observe containers | set mem_gb = memory / 1073741824
observe containers | set today = now()
observe containers | set day = extract(created_at, 'day')
observe containers where $state = running
observe containers | let $threshold = 80 | where cpu > $threshold
observe containers | let $app = "myapp" | where compose_project = $app

9. Execution Semantics

9.1 Batch Queries

Batch queries consume finite input and return finite output.

Batch examples:

observe containers
observe images | sort by size desc | limit 5
inspect container api-service

Batch execution steps:

  1. Parse query into AST.
  2. Build logical plan.
  3. Read Docker snapshot or stored snapshot.
  4. Apply filters and pipeline stages.
  5. Render output as table or JSON.

9.2 Stream Queries

Stream queries consume unbounded input and keep running until cancelled.

Stream examples:

events containers
events containers | where action = "die"
logs container my-app
logs container my-app tail 50 | where message contains "error"
compose myapp events (batch)
compose myapp events | where action = "die" (batch)
compose myapp networks
compose myapp networks | where action = "connect"
compose myapp logs service api
compose myapp logs service api | where message contains "error"
alert when cpu > 85% for 2m then print "High CPU"

Stream execution rules:

9.3 Hybrid Queries

Hybrid queries combine a current Docker snapshot with recent metric samples.

Examples:

observe containers last 5m
observe containers | where cpu > 80%
analyze containers find anomalies last 1h

Hybrid execution rules:

9.4 Historical Queries

Historical queries require a telemetry store.

Examples:

inspect container api-service at "2026-01-01 12:00:00"
events containers last 1h
analyze container api-service correlate events last 1h

Historical execution rules:

10. Consistency Model

DOL v0.2 uses best-effort consistency:

The implementation should preserve timestamps and raw Docker IDs so users can reconcile output with Docker itself.

11. Error Semantics

Parser errors should include:

Runtime errors should distinguish:

Examples:

parse error at column 28: expected field
  --> observe containers | where | sort by cpu
                            ^

parse error at column 20: expected expression after `where`
  --> observe containers | where | sort by cpu
                               ^

runtime error: `sort by` requires finite input; add `last 5m` or `limit`
runtime error: historical query requires a telemetry store

Parser errors show:

In the CLI and REPL, error messages are displayed in ANSI red with bold formatting for visual prominence. Source pointers appear in cyan (-->), caret in green (^), and suggestions in yellow (help:).

12. Reserved Keywords

Reserved keywords:

alert analyze and asc at between by case compose contains correlate count dashboard
assert debug config desc distinct else end events explain extract false fields fill find for from
group having health if in inspect is join last let limit logs matches max min networks
not null observe of offset or ping repl repeat restart select service services set sort
split_part starts_with ends_with sum then to top true upper lower length trim concat
substring coalesce replace reverse position now date_format date_diff volumes
webhook when where

Docker names that conflict with reserved keywords must be quoted as strings when used as values.

13. Example Query Set

The parser and executor should prioritize these queries first:

observe containers
observe containers where status = running
observe containers | where image contains "postgres" | select name, status
observe images | sort by size desc | limit 10
events containers where action = "die"
events containers | where action = "restart" | select time, container, image
inspect container api-service
inspect container api-service at "2026-01-01 12:00:00"
analyze containers find anomalies
analyze containers find restart_loops last 10m
alert when cpu > 85% for 2m then print "High CPU"  observe containers | where cpu > 80% | alert "High CPU detected"

13.1 Aggregate Functions

Supported aggregate functions for group by ... with ...:

Function Syntax Description Return Type
count count(field) Count of rows in each group Integer
sum sum(field) Sum of numeric values Float
avg avg(field) Arithmetic mean Float
min min(field) Minimum value Float
max max(field) Maximum value Float
median median(field) Median (50th percentile) Float
percentile percentile(field, p) The p-th percentile (0–100) using linear interpolation Float

Examples:

observe containers | group by image with median(cpu) as med_cpu
observe containers | group by image with percentile(memory, 95) as p95_mem
observe containers | group by image with avg(cpu) as avg_cpu, percentile(cpu, 99) as p99_cpu

14. Out of Scope

These features are intentionally deferred:

15. Implementation Notes

The parser was implemented incrementally, starting with the core query families and gradually adding pipeline nodes. The current implementation supports all nine query families and all pipeline nodes described in this specification.

AST types (defined in src/ast.rs):

16. CLI Reference

The DOL CLI supports the following flags:

Flag Description
--store <path> Path to SQLite telemetry store
--collect Start background data collection daemon
--metrics-interval <s> Metrics polling interval in seconds (default: 30)
--snapshot-interval <s> Snapshot interval in seconds (default: 300)
--store-stats Display telemetry store statistics
--apply-retention Apply retention policies to clean old data
--output <fmt> Output format: table, json, json-compact, csv, jsonl (default: table)
--export <path> Write output to file (format inferred from extension: .csv, .json, .jsonl, .table)
--export-format <fmt> Export file format: influx, loki, prometheus (used with --export)
--file <path> / -f <path> Read the DOL query from a .dol file
--host <addr> Docker daemon address (e.g., tcp://192.168.1.100:2375)
--watch <s> Re-run query every N seconds (batch and alert queries)
--timeout <s> Query execution timeout in seconds — if a query takes longer than this, it is aborted (applies to watch, alert, events, store, and single queries)
--explain Show the logical query plan without executing
--diff Compare query results with the last store snapshot (requires --store)
--theme <dark\|light> Color theme for table output (dark or light); can also be set permanently in config via theme: dark\|light
--completion <shell> Generate shell completion script (bash, zsh, fish, powershell, elvish)
--export-influx <url> Push results to InfluxDB v1/v2 HTTP write API
--export-grafana-loki <url> Push results to Grafana Loki HTTP push API
--export-prometheus <url> Push results to Prometheus Pushgateway
repl Start interactive REPL shell with tab completion, history, and REPL commands
top Live-updating TUI container monitor (top-like) with auto-refresh, keyboard controls, CPU/MEM gauge bars, filter mode, and event-driven refresh
dashboard Multi-panel TUI dashboard with container list, state distribution stats (histogram bars), top images, and live Docker events stream
config init Create a default config file at the standard config path
config set <key> <value> Update a configuration value (store, output, host, metrics-interval, snapshot-interval, theme, api-timeout, api-quick-timeout, stats-timeout, events-timeout, webhook-timeout, restart-timeout)
config view Display the current merged configuration (from CLI flags + config file + defaults)

Additional REPL commands (within dol repl):

REPL Command Description
.help Show available REPL commands
.exit / .quit Exit the REPL
.history Show command history
.watch <secs> Re-run the last query every N seconds
.export <path> Write subsequent results to a file
.output <fmt> Set output format (table, json, json-compact, csv, jsonl)
.host [<addr>] Show or set Docker host address within the REPL session

Config file support (YAML/TOML):

17. Docker API Timeout Configuration

DOL uses configurable timeouts for all Docker API calls. These can be set in the config file or via dol config set:

Config Key Default Description
api-timeout 30 Timeout (seconds) for standard Docker API calls (list containers, images, networks, volumes, inspect, logs).
api-quick-timeout 10 Timeout (seconds) for lightweight calls (ping).
stats-timeout 10 Timeout (seconds) for per-container stats collection.
events-timeout 30 Max seconds to wait for a single event from the Docker events stream (used by dol top, dol dashboard).
webhook-timeout 10 HTTP request timeout for alert webhook POST actions.
restart-timeout 30 Timeout (seconds) for alert container restart actions.

Example config:

store: /path/to/telemetry.db
output: table
theme: light
api_timeout: 60
stats_timeout: 15
events_timeout: 60
webhook_timeout: 5
restart_timeout: 45

All timeout values are optional — defaults are used when not set.

The grammar defined in this specification is the source of truth for parser tests.