No description
  • Rust 99.4%
  • Makefile 0.3%
  • Dockerfile 0.3%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
2026-11-05 00:14:47 +01:00
config feat 2026-11-05 00:14:47 +01:00
docs feat 2026-11-05 00:14:47 +01:00
src feat 2026-11-05 00:14:47 +01:00
tests feat 2026-11-05 00:14:47 +01:00
.dockerignore feat 2026-11-05 00:14:47 +01:00
.env.example feat: Build and publish using makefile 2026-10-30 17:10:08 +01:00
.gitignore feat 2026-11-05 00:14:47 +01:00
Cargo.lock feat 2026-11-05 00:14:47 +01:00
Cargo.toml feat 2026-11-05 00:14:47 +01:00
compose.beszel.yaml.example feat: Beszel with email and password 2026-10-30 16:30:38 +01:00
compose.build.yaml feat: Build and publish using makefile 2026-10-30 17:10:08 +01:00
compose.yaml feat 2026-11-05 00:14:47 +01:00
Dockerfile feat 2026-11-05 00:14:47 +01:00
Makefile feat 2026-11-05 00:14:47 +01:00
README.md feat 2026-11-05 00:14:47 +01:00
rust-toolchain.toml init 2026-10-27 20:28:20 +01:00

Homelab Analyzer

A deterministic, self-hosted, read-only daily diagnostics CLI written in Rust 2024 (Rust 1.96+). Prometheus supplies numeric time series; optional Beszel integration supplies system inventory context. The operator runs the CLI and can review its reports in ChatGPT. No LLM, daemon, web server, remediation, infrastructure credentials, SSH client, or Docker socket is required by the analyzer.

Architecture

Operator -> CLI -> YAML rules
                       -> GET Prometheus range API -> pure statistical analysis
                       -> GET Beszel systems (optional context)
                       -> SQLite run + anomaly history
                       -> JSON + Markdown reports

analysis.rs contains deterministic analysis independent of HTTP and persistence; timeline.rs (slots, intervals, counter increases), quality.rs (data-quality causes), alerts.rs (Prometheus alerts) and incidents.rs (incident lifecycle) are pure helpers it uses. insights.rs builds the executive summary and recommendations. source.rs implements bounded HTTP timeouts, read-only data requests, a restricted Beszel authentication POST, and synthetic signals. storage.rs commits runs and deduplicated observations transactionally. report.rs writes each report atomically with an immutable run filename.

Local quick start

cd ~/Code/Rust/homelab-analyzer
cargo build --release --locked
./target/release/homelab-analyzer analyze --demo
./target/release/homelab-analyzer report --demo
./target/release/homelab-analyzer report --demo --format json
./target/release/homelab-analyzer healthcheck --demo

The synthetic scenario intentionally returns exit code 2: it includes a spike, 50% coverage, and a missing expected host. The spike is reported as an incident and as two statistical anomalies. No HTTP requests or service secrets are read in demo mode. Its default end time is fixed for reproducibility. Synthetic and live databases and report directories are separate. The complete demo can be run without LAN access once dependencies are installed.

The application does not auto-load .env locally. Export environment variables in your shell; Compose loads .env. For example, in Bash:

cp .env.example .env
# Edit .env with verified LAN URLs; review config/queries.yaml for your inventory.
# Review the file before sourcing it into your shell.
set -a
source .env
set +a
./target/release/homelab-analyzer healthcheck
./target/release/homelab-analyzer analyze --format json
./target/release/homelab-analyzer report --format markdown

Commands: analyze, report, healthcheck and export (see "Reviewing reports in ChatGPT"). All commands accept --config PATH, --data-dir PATH, --demo, and --format (markdown, markdown-full or json). markdown is a compact summary for people and ChatGPT: incidents, alerts, deviations, grouped data-quality causes and recommendations. markdown-full adds per-series comparison and baseline tables. The saved .md file is compact; the saved .json file has every detail. The executable also provides --help, per-command help, and --version. analyze --end 2026-10-27T00:00:00Z allows reproducible backfills. Explicit end times require a timezone and are rounded down to the configured evaluation step. report reads the most recent saved report without accessing any remote service. healthcheck verifies query API access and optional Beszel inventory access; it does not claim that monitored hosts or backups are healthy. It does not create or check the SQLite database. analyze verifies local write access as it saves.

Exit codes: 0 = completed with adequate data (incidents may still be present), 1 = configuration/local storage/fatal source failure, 2 = degraded data or optional source failure. With exit code 2 the JSON and Markdown reports are still saved; stderr prints the report path and every reason from data_quality.degraded_reasons. report returns 0 when an existing report is rendered, even when that report describes degraded data. Consumers must inspect status, executive_summary and active_incidents, not just the exit code. Status ok describes data quality, not infrastructure health. Reports include schema_version: 2; version 1 reports saved earlier remain readable.

Before v2 every live run exited 2: Prometheus keeps 15 days, so the 30-day baseline never reached 80% coverage, and any unavailable baseline marked the run degraded. Insufficient history is now a non-degrading insufficient_history notice.

Verify metrics before adding infrastructure rules

The default configuration is the verified rimonome profile: 27 verified rules for scrape availability/freshness, manwe/rimonome resources, Proxmox storage, yane ZFS, SMART, Restic and firing Prometheus alerts, plus three v2 counter rules for ZFS vdev and SMART error increases and Prometheus alert collection. Expressions and expected labels were checked against the running API; the profile is specific to this inventory. config/canary.yaml retains the generic vector(1) query API check. Review the profile before using it in another environment. Static thresholds are initial operator policy choices and are documented separately from observed facts.

  1. Set the actual Prometheus LAN URL and verify healthcheck.
  2. Inspect GET /api/v1/label/__name__/values and the Prometheus query UI to find metrics actually exported by manwe, yane, Restic, and Alertmanager.
  3. Inspect label sets and units using GET /api/v1/query?query=..., then check history and gaps in the UI before committing each query.
  4. Update the profile with verified gauge expressions, recording rules, or rate/increase expressions for counters. Avoid raw cumulative counters.
  5. Run a first analysis and inspect coverage, baseline availability, and report cardinality before using the report for daily review.

Example schema using a deliberately non-executable placeholder:

step_seconds: 900
rules:
  - id: verified_capacity
    query: REPLACE_WITH_VERIFIED_PROMQL
    unit: percent
    lower: 0
    upper: 90
    direction: high
    mad_multiplier: 6
    min_delta: 5
    min_coverage: 0.8
    min_baseline_samples: 24
    breach_fraction: 0.05
    stale_after_seconds: 1800
    expected_labels: []
    severity: warning        # critical | warning | info for its incidents
    counter: false           # true: evaluate per-step increases of a cumulative counter
    optional: false          # true: absent series are notices, not degraded data
    recommendation: null     # operator text shown when the rule raises an incident

alerts: {enabled: true} at the top level reads GET /api/v1/rules?type=alert and the ALERTS{alertstate="firing"} range for the current window. Label and annotation values whose names contain password, passwd, secret, token, apikey, api_key, authorization, cookie, credential, private_key or session are replaced by [redacted], and user:password@ is removed from URLs. alerts.exclude_labels: [name] drops further names entirely. Redaction happens before analysis, so secrets reach neither reports nor SQLite.

Counter rules (counter: true) replace each value by its increase since the previous slot. A decrease is a reset (reboot, zpool clear): it adds nothing, its time is recorded in reset_at, and the post-reset value becomes the new reference, so a reset is never reported as new errors. Errors counted between a reset and the next sample are therefore not included; use node_boot_time_seconds if that matters. The current day continues from the last 7d sample, so an increase at midnight is not lost. An incident is raised when the window's total increase exceeds upper (default 0); lower is rejected. MAD comparisons are skipped for counters; series[].counters reports start value, end value, increase and resets for each window. Absolute values remain visible there. Each breach interval carries its measured values (observation.values: value before the interval, first, last, min, max and counter increase), and each incident has its own values aggregated from its intervals, so two incidents on one counter never show the same window total.

Counter fields and their scope

Field Scope
incident.counter.incident_increase Increase inside this incident's breach intervals across all saved reports. Each evaluation slot is counted once: a later report adds only slots after the previous report's end, so overlapping windows are never summed twice. null for incidents saved before this field existed and starting before the current window.
incident.counter.incident_increase_in_window The part of the incident's increase inside the current 24h window.
incident.counter.window_total_increase Increase of the whole series in the current window, shared by every incident on that series.
incident.counter.counter_resets, reset_at Real decreases of the series in the current window (reboot, zpool clear). They add nothing to any increase and never raise an incident.
finding.details.window_total_increase, counter_resets, reset_at Same window totals on the series-level finding. increase and resets remain as deprecated aliases for existing consumers.
incident.details Unchanged copy of the finding details, so for counters it holds window totals, not the incident's own values. Use incident.counter and incident.values.
series.windows.current (counter rules) Statistics of per-step increases, used for coverage and baselines.
series.raw_current (counter rules) Statistics of the raw cumulative counter values in the current window.
series.counters.current Window start and end value, total increase, resets and reset times.

In the export, metrics.json (schema homelab-analyzer-metrics/2) names the series behind every statistics block: sample_statistics.series is value for gauges and counter_value (raw cumulative values) for counters, per_step_increase holds the per-step statistics of counters, and the 7d/30d blocks state series too. A counter that rose from 2 to 4 in two steps therefore shows sample_statistics.max = 4, per_step_increase.max = 1 and counter.window_total_increase = 2.

Unknown YAML fields, duplicate IDs, reversed bounds and nonfinite thresholds are rejected. Label identity uses the complete returned label set, including __name__ when present. expected_labels must contain exact complete label sets; it is not a selector. Use it to detect a host absent from all retained history. Series present in either baseline are checked even if absent from the current day. New labels with no baseline remain explicitly unavailable for historical comparison.

Statistical assumptions and missing data

For analysis end T and step S, the current window is [T-24h, T). Baselines are [T-8d, T-24h) and [T-31d, T-24h): 7 and 30 full preceding days, excluding the current day. Range queries use the inclusive endpoint window_end-S to avoid overlapping samples. Default S is 15 minutes; supported steps are 300–3600 seconds and must divide 86400 exactly. Queries execute sequentially, with a 30-second server query timeout, a 35-second client timeout, a 10-second connection timeout, and a 16 MiB response limit. The application uses blocking HTTP, without an application async runtime; reqwest uses Tokio internally as an implementation detail. Keep query cardinality and configured rule count small.

Coverage is the number of unique finite evaluation slots divided by expected slots. NaN, infinity, off-grid and out-of-window samples do not count. No values are imputed or interpreted as zero. Native histograms are unsupported: convert them to scalar PromQL; responses containing them are marked degraded.

For each label set, the analyzer reports count, coverage, median, MAD, min, max, first and last finite sample, and the window range. A baseline is usable only with sufficient coverage and samples and no API errors or annotations. Short retention is not extrapolated. An unavailable baseline is never an anomaly: its comparison is skipped and unavailable_reason says why (source_issue, no_history, insufficient_coverage, insufficient_samples). data_quality.baselines lists the range and sample span used for each rule's 7d/30d baseline.

Data quality

data_quality.status is degraded only when degraded_reasons is non-empty; each reason names its code, rule, labels and window:

Code Meaning Degrades
source_error / source_warning Prometheus query failed or returned annotations yes
exporter_unavailable series missing while its job/instance target was down or absent in up yes
no_samples target was up (or unknown), but the series has no samples yes
incomplete_data / stale_data coverage below min_coverage or last sample too old yes
alerts_unavailable / beszel_unavailable optional source failed yes
insufficient_history baseline lacks history (for example 30d with 15d retention) no
optional_metric_absent rule marked optional has no series no
target_lookup_failed up lookup failed; missing-data causes unknown no

up is queried once per run, only when some series lacks data.

data_quality.groups merges issues with the same code and window into one entry with the affected rules and the gaps they share. Baseline gaps of at least one hour are detected from the samples (gaps on each 7d/30d summary and on data_quality.baselines) and classified as before_first_sample (retention or a new series) or collection_gap (samples stopped mid-window, for example while Prometheus was down). The per-rule issues remain in degraded_reasons and notices.

Incidents, anomalies and alerts

Static threshold breaches, counter increases and Prometheus alerts are incidents: each has a stable id, severity, status (active/resolved), and two separate parts. history is the lifecycle across saved reports: started_at, ended_at, duration_seconds, total_breach_seconds, number of reports. current_window is the evidence measured in this report only: breach intervals, breach seconds, and whether the breach is ongoing at the window end.

Breach intervals come from evaluation samples: an interval starts at the first breaching sample and ends at the next slot that holds a non-breaching sample or no sample, so times have the resolution of step_seconds. Availability rules use min_over_time(...[15m]), so a failure may have begun up to 15 minutes before the reported start.

Breaches separated by at most incident_gap_seconds (top-level config, default 3600) form one incident; a longer recovery ends it and the next breach opens a new incident with a new id. 0 splits at every recovery. history.span_seconds runs from the first breach to the last breach end (or the window end while active); history.total_breach_seconds counts breach intervals only, breach_intervals counts them, and pattern is continuous or intermittent. A flapping target therefore reads as "intermittent: 7h 45m breach in 26 intervals over a 22h 30m span", never as one uninterrupted outage. current_window contains only the intervals that belong to that incident.

An incident continues into the next report when its next breach starts within the gap, or exactly where an unobserved period between reports ended; a resolved incident reopens with the same id under the same rule. Overlapping windows are not counted twice. It resolves when a later report evaluates the series without a breach. A series without samples keeps the incident active, marked observed_in_current_window: false. Incident identity is the source, rule, labels and kind, without configuration, so threshold tuning keeps the id. Backfills of older windows do not rewrite lifecycles (tracking: window_only).

The JSON report contains executive_summary (measured facts only), active_incidents, resolved_incidents (resolved by this run), historical_incidents (resolved earlier, last 30 days, up to 20), anomalies (MAD deviations, severity info), alerts (firing, pending and resolved within the window, with labels, severity and annotations), data_quality and recommendations. Recommendations are marked basis: operator_rule or basis: heuristic; they are never measurements. findings and rules keep the complete per-series detail. Alert annotations come from the active alert or, for alerts that ended, from the rule template (annotations_source); values are truncated to 1000 characters.

A historical outlier satisfies:

abs(sample - baseline_median) > max(min_delta, mad_multiplier * 1.4826 * baseline_MAD)

direction restricts the sign of the deviation. A finding requires at least breach_fraction of observed current values to exceed the distance. Zero MAD uses min_delta, with strict inequality; configure a meaningful floor in the query's units. Historical checks require sufficient current coverage. Static upper/lower checks use the same breach fraction and still flag observed breaches on partial data, explicitly reporting partial confidence. Missing or stale data is reported independently. Thresholds are strict (> / <).

This is an aggregate distribution baseline, not a forecasting model. It does not adjust for weekday, time-of-day, seasonality, counter resets in raw counters, or correlated samples. Median shifts and isolated spike fractions appear in the report, but brief events below the configured breach fraction can be missed. Prometheus range-query coverage represents evaluation availability, not raw scrape completeness: lookback can reuse old samples. Add verified timestamp, absent_over_time, or scrape-gap expressions for exporter freshness checks.

Prometheus warnings/info annotations make a source degraded and disable the corresponding historical baseline. Their counts are retained rather than raw messages, to avoid leaking sensitive upstream content. Transport/API errors are sanitized and included per window. One failed rule does not discard other results.

Beszel integration boundary

Official Beszel documentation confirms PocketBase APIs and the systems collection, and warns that schemas may change in minor releases. This MVP uses only GET /api/collections/systems/records, requesting id,name,status, with pagination capped at 100 pages of 100 records. It validates the returned fields. Unknown schema, authentication failures, and pagination overflow degrade the report rather than inventing a healthy result.

Set BESZEL_URL and choose exactly one authentication mode:

  1. Account login: set BESZEL_EMAIL and BESZEL_PASSWORD, or use BESZEL_PASSWORD_FILE for a mounted secret. Use a dedicated regular Beszel user with the Read only role and share the monitored systems with that account.
  2. Existing user token: set BESZEL_TOKEN or BESZEL_TOKEN_FILE. Supply the raw PocketBase user token without a Bearer prefix. Its expiry is still the operator's responsibility in this mode.

Example account configuration in the untracked .env:

BESZEL_URL=http://10.0.20.11:12900
BESZEL_EMAIL=analyzer@example.com
BESZEL_PASSWORD='replace-with-account-password'

Remove BESZEL_TOKEN and BESZEL_TOKEN_FILE when using an account. Do not set both BESZEL_PASSWORD and BESZEL_PASSWORD_FILE. Conflicting modes or partial account credentials are configuration errors, with no login attempt. Password contents are not included in error messages. Password files preserve significant spaces and remove only one final LF or CRLF line ending.

Account mode logs in once on every analyze or healthcheck invocation through POST /api/collections/users/auth-with-password, then uses GET requests for system inventory. This is the only permitted POST endpoint; no systems, alerts, metrics, users, or infrastructure are modified. The resulting token stays in memory for that invocation and is never written to the report or database. There is no need to copy or renew a browser token between daily runs. Authentication failures, including MFA challenges requiring interaction, produce a degraded Beszel result without discarding the Prometheus report. No automatic retry is made for failed credentials, and redirects are never followed. A token rejected during inventory is reported as a source failure rather than triggering repeated login attempts. report reads saved output without logging in; --demo never reads credentials or makes authentication requests.

For a file-based password, copy compose.beszel.yaml.example to compose.override.yaml and use this .env configuration instead:

BESZEL_URL=http://10.0.20.11:12900
BESZEL_EMAIL=analyzer@example.com
BESZEL_PASSWORD_FILE=/run/secrets/beszel_password

Create secrets/beszel_password locally with the account password. Keep it out of Git and readable by runtime UID 1001; the optional Compose override mounts it as a secret. Setting _FILE alone does not mount a host file into the container. Do not set BESZEL_PASSWORD in file mode. The override is loaded automatically by Docker Compose and is ignored by Git.

After publishing the updated image from the PC and updating configuration:

make pull
make healthcheck
make

Check that beszel.status is ok and systems contains the expected hosts. An empty list can indicate that no systems were shared with the user. Creating a user does not automatically grant access to existing systems. In the PocketBase admin panel (/_/), open the systems collection, edit each intended system and add the analyzer account to its users field, preserving existing users. See the Beszel system-sharing instructions.

After changing access, run make healthcheck and then make (or make analyze) to collect a fresh report. make report only reads the last saved result; rebuilding the image or running a healthcheck does not update that saved report. Do not use superuser credentials or the agent registration/universal token for this adapter. Beszel inventory remains context only: its numeric historical collection names and fields have deliberately not been guessed. Prometheus remains the numeric analysis source.

Sources checked during implementation:

Persistence and deduplication

Local SQLite stores immutable JSON runs, anomaly fingerprints, and unique run/fingerprint observations. Fingerprints include source identity, normalized configuration, rule ID, complete labels, and finding kind. The first/last seen analysis timestamps and occurrence count are attached to findings.

A run ID hashes the report before persistence metadata. Identical retries do not increment counts; corrected or late data yields a new run. Backfills preserve minimum first-seen and maximum last-seen timestamps. Counts represent distinct observed runs, not days. Changing configuration deliberately starts a new anomaly identity. Finding history is observation history; incident lifecycle is stored separately in the incidents table (see above). No notifications or Alertmanager writes occur.

Reports live at data/reports/{live,synthetic}/<run_id>.{json,md}. Each file is atomically replaced; readers should use the SQLite-backed report command if both report files have not yet been written after a crash. Rerunning an identical analysis repairs missing files. No automatic retention/deletion is performed; back up and rotate the data volume externally.

Publishing on the PC and running on rimonome

make publish builds the Dockerfile on the PC and pushes the image to the internal Forgejo registry. Rimonome pulls and runs the finished image without compilation. Both hosts were verified as x86_64; build and runtime use linux/amd64.

One-time publisher login (Docker prompts for the password or access token):

docker login git.manwe.xyz

On the PC:

make publish
# Optional named release instead of latest:
make publish IMAGE_TAG=v0.1.1

The default image is git.manwe.xyz/akasiek/homelab-analyzer:latest. Override IMAGE_REPOSITORY or IMAGE_TAG through the environment, .env, or Make command-line variables. Publishing a named tag does not also update latest. Build and push use the same Compose configuration and selected image reference. A failed build stops the target before push. Publishing builds the current working tree, so commit changes first when you want the published contents traceable to Git.

compose.build.yaml is a standalone build/publish configuration and does not load runtime environment files or secret mounts. .env is still read by Compose for image-variable interpolation when present. Registry credentials are managed by Docker login and are never supplied as build arguments, copied into the image, or added to the application .env. Only make publish pushes to the registry; make build builds and tags the image locally.

Forgejo documents its image naming, login and push workflow in the container registry guide. Private packages also require a Docker login on rimonome using credentials with pull access. The repository-linked image source label is included in the image.

Docker Compose and LAN-only operation

On rimonome after committing/pushing the source changes and publishing the image:

cd ~/docker/homelab-analyzer
git pull --ff-only
# First installation only: copy .env.example to .env and configure LAN endpoints.
# For a private package, first run: docker login git.manwe.xyz
mkdir -p data
make pull
make

make pull downloads the image selected by IMAGE_REPOSITORY and IMAGE_TAG. The default Compose file contains no build configuration. Normal commands use --pull never: they run the already selected local image without an implicit network update. Run make pull explicitly when adopting a new image. A first installation therefore needs a pull before analysis. To pin a release, set IMAGE_TAG in the server .env before pulling; use the same tag that was published. Retain older published release tags when you need rollback.

make                           # Analyze using the downloaded image.
make demo                      # Analyze synthetic data using the downloaded image.
make report                    # Read the latest saved Markdown report.
make report ARGS='--format json'
make healthcheck               # Check source connectivity; no build or pull.
make export                    # Analyze once and write exports/homelab-<UTC time>.zip.
make export-json               # Same, JSON files only (no summary.md).
make export-summary            # Same, summary.md and manifest.json only.
make pull                      # Download the selected published image.
make build                     # PC: build and tag locally, without publishing.
make publish                   # PC: build and publish the selected tag.

Additional analyzer options can be passed through ARGS, for example make ARGS='--end 2026-10-27T00:00:00Z'. The Makefile preserves command failures: an analyzer exit code of 2 (degraded data) is also reported by make as Error 2. This still means the analysis generated a report; it is not a build failure.

The data volume remains available after the temporary container is removed. Runtime Compose overrides, including the optional Beszel password-secret mount, are applied automatically. The build/publish Compose file deliberately excludes those runtime overrides. Rule files ship inside the image (/app/config), so make publish on the PC followed by make pull on rimonome also updates the configuration. To test rules without rebuilding, add ./config:/app/config:ro to the service volumes in a local compose.override.yaml; while that mount exists it hides the image's rules. The current ./data:/data bind mount and UID/GID are preserved during this image-workflow migration.

Equivalent direct Compose commands for an already configured server:

docker compose pull analyzer
docker compose run --rm --pull never analyzer healthcheck
docker compose run --rm --pull never analyzer analyze
docker compose run --rm --pull never analyzer report --format json
# Synthetic smoke scenario (expected exit 2):
docker compose run --rm --pull never analyzer analyze --demo

This is a one-shot service; use make for each daily review. Do not use compose up -d as a scheduler. No automatic schedule is installed. The analyzer container has no access to Docker or SSH credentials. Publishing an image does not update or restart anything on rimonome.

Reviewing reports in ChatGPT

cd ~/docker/homelab-analyzer
make export

make export runs one analysis (exactly like make, including saving it to SQLite) and writes a single archive, for example exports/homelab-20261104-234500.zip, to the host's ./exports directory (Compose mounts it at /exports; the Makefile creates it, and it must be writable by UID 1001 like ./data). Upload that ZIP to ChatGPT. The command prints the archive path; the exit code follows analyze (2 = degraded data, archive still written). export accepts --contents full|json|summary and --output-dir (or EXPORT_DIR).

File Content
summary.md Short report: data sources, changes since the previous run, correlated events, incidents, alerts, deviations, data quality, recommendations, and a reading guide. No per-metric tables.
incidents.json Incidents with id, severity, status, target host/service/disk, start, end, span and breach time, start/end/delta values, related Prometheus alerts, history and measurement quality; correlation groups that link events on one disk; changes since the previous run.
metrics.json One line per series: current value, median, min, max, p05, p95 over 24h, 7d/30d comparison or the reason it is unavailable, counter increase and resets, coverage and flags. No raw samples.
diagnostics.json The complete report (schema_version: 2), identical to --format json.
manifest.json Export schema, application version, run id, creation time, window, SHA-256 and size of every file, and incomplete data sources.

All files come from the same run and share its run_id. The archive is written to a hidden .zip.partial file and renamed only when complete. Values of PROMETHEUS_URL, BESZEL_URL (including their host names), tokens, the Beszel e-mail and password are replaced by [redacted] in every file; alert labels and annotations are already redacted during analysis.

Correlation groups join incidents that share an entity label value (device, disk, disk_id, serial_number, pool, mountpoint; by-id names also match their serial) and whose breaches are at most correlation.window_seconds (default 1800) apart. correlation.aliases adds read-only queries whose series link several names of one entity, for example disk="/dev/sdc" with its serial_number:

correlation:
  window_seconds: 1800
  aliases:
  - query: smartmon_device_info{job="node",instance="manwe:9100"}
    labels: [disk, serial_number]

The alias query is read once per run at the last evaluation slot; if it fails, the run is not degraded and incidents are correlated by exact label values only.

Copy the Markdown output into ChatGPT for interpretation. To read the last saved report without collecting new metrics, use make report. Use make report ARGS='--format json' when a structured report is useful. Exit code 2 still means a report was produced with degraded data; inspect its data-quality section.

ChatGPT interpretation is an operator action, outside the analyzer. The application never calls an LLM, uploads reports, or executes suggested infrastructure changes. An external scheduler may run the deterministic CLI later if explicitly configured.

The runtime uses UID/GID 1001, a read-only root filesystem, dropped capabilities, no-new-privileges, PID/memory/CPU limits, and a small temporary filesystem. Only /data persists; configuration is part of the read-only image. There are no published ports, no host network, no Docker socket, and no runtime root privileges. The host ./data directory must exist and be writable by UID 1001 before running the container. On rimonome, create it as that user; arrange matching ownership explicitly on other hosts. The image build requires internet access.

Compose's bridge network allows outbound LAN connections but does not enforce an egress allowlist. Use the server firewall/network policy to restrict egress to the configured LAN services. LAN-only service binding/firewalls are infrastructure responsibility; the analyzer opens no listening socket. For Docker DNS service names, explicitly attach to the existing monitoring network in a local override, or use actual LAN DNS names/IPs. Never assume localhost reaches the host.

Secrets may be supplied via environment variables or _FILE paths. Setting both forms for one token is rejected. For Compose secrets, declare a secret file and mount it through a local override, then set the corresponding _FILE variable to /run/secrets/<name>. Keep secrets out of Git and restrict file permissions. Environment files and data are excluded from Git and the build context. HTTP clients reject redirects, use TLS verification, and ignore proxy environment variables. Prefer HTTPS when the LAN is not fully trusted. Remote labels and inventory remain potentially sensitive report contents; protect the data volume.

Development and verification

cargo fmt --check
cargo clippy --locked --all-targets -- -D warnings
cargo test --locked

Tests use synthetic time series, mocked API responses, and local loopback HTTP servers: outliers, zero MAD, directionality, missing history, incomplete data, nonfinite/duplicate samples, window boundaries, API failures, Beszel pagination/schema, report generation, SQLite retry/backfill deduplication, incident lifecycle across reports, counter increases and resets, alert collection, data-quality causes, and CLI exit codes against a fake Prometheus with 15-day retention. No live infrastructure is required or modified. Live Prometheus metric inventory, the installed Beszel version, and deployment connectivity require separate checks on rimonome. Local tests never contact live services.

Migration from the Python prototype

The implementation lives at ~/Code/Rust/homelab-analyzer. The original project at ~/Code/Python/homelab-analyzer remains unchanged as a reference. Rust is the maintained implementation for this MVP; Python, uv and pytest are not needed to build or run it. The image contains the compiled release executable, system CA certificates and the default configuration, without Cargo or a Python runtime.

Commands, YAML rules, exit codes and the JSON report's analysis fields retain the prototype's meaning. tests/fixtures/python_demo.json is a captured reference from the Python implementation; a Rust test compares the complete synthetic analysis against it with numeric tolerance. The normal Rust tests do not execute Python.

The Rust project starts its own history; existing Python databases and reports are not moved or overwritten. Hash encoding changed, so run IDs and fingerprints should not be compared across implementations. Atomic SQLite writes also serialize concurrent identical runs; report now opens the database strictly read-only.

The copied Compose configuration uses the existing local choice 1001:1001; the Rust image and /data ownership match it. Existing volumes created with another UID need administrator-managed ownership adjustment before reuse. This migration does not change existing volumes or deploy anything to rimonome.

Local migration verification

  • cargo test --locked: 31 integration tests passed, including Python-reference parity, real loopback HTTP checks, concurrent SQLite deduplication and CLI flows.
  • cargo fmt --check and Clippy with -D warnings: passed.
  • docker compose config --no-env-resolution --quiet: passed.
  • The release Docker image built successfully. Its synthetic healthcheck, analysis, saved report and identical retry passed with network disabled, UID 1001, read-only root, no capabilities and no-new-privileges. SQLite remained writable on the isolated data volume. No Python, Cargo, SSH or Docker socket was present.

The original language migration did not contact live services. The subsequent rimonome profile was verified against the live Prometheus API; see profile evidence and limits. Beszel is disabled until an operator supplies its URL and authentication settings.