Enable instrumentation.prometheus and your node serves pax_chain metrics on port 26660 with no extra instrumentation to add. You read BeginBlock, EndBlock, CheckTx, DeliverTx and Commit durations, per-transaction results and gas, and per-block gas utilisation against the MaxGasWanted cap. Bucket edges sit at the stated targets of 500 ms, 1.5 s and 2.5 s, so each one reads as an exact bucket ratio. No node dashboard ships yet.

What it does

Every Paxeer X node records OpenTelemetry histograms and counters for each ABCI phase of a block, for per-transaction results and gas, and for per-block gas utilisation. The histogram bucket boundaries are chosen to line up with the block-processing latency target stated in the code: p50 at or below 500 ms, p95 at or below 1.5 s, p99 at or below 2.5 s. The metrics are exported in Prometheus format under the pax_chain namespace and served on the node's instrumentation port, :26660, when instrumentation.prometheus is enabled. A local Docker Prometheus and Grafana stack ships alongside the four-node local cluster.

Key capabilities

• Histograms for BeginBlock, EndBlock, module EndBlock, CheckTx, DeliverTx, DeliverTxBatch and Commit durations.
• Block-processing histogram labelled by execution type, with bucket edges at the 500 ms, 1.5 s and 2.5 s targets.
• Per-block gas-wanted histogram and a utilisation ratio against the MaxGasWanted consensus parameter.
• Counters for transaction results, execution type, gas used and wanted, and giga-executor fallbacks.
• Light invariance total-supply check duration plus invalid-key and unmarshal-failure counters.
• app_build_info gauge reporting the running node version and git commit on every scrape.

Differentiators

The latency targets are encoded in the histogram bucket boundaries themselves (0.5 s, 1.5 s and 2.5 s are bucket edges), so compliance with each target reads as an exact bucket ratio instead of a quantile interpolated across a wide bucket. Two bucket edges above the p99 line, at 3 s and 4 s, exist to refine quantiles just past it.

Outcomes

• Identify which ABCI phase consumed a block's time budget.
• Read each latency target as an exact bucket ratio.
• Watch how close blocks run to the block gas cap.

Technical notes

• The OpenTelemetry meter provider is bound once at application construction (node/app.go:559) to a Prometheus exporter with namespace pax_chain. It registers on the default Prometheus registry, which the consensus node serves at prometheus-listen-addr (:26660 in the shipped node configs).
• Two separate local stacks exist and they monitor different things. The compose overlay publishes Prometheus on host port 9099 and Grafana on 3000 with credentials admin/admin, and scrapes pax-node-0 through pax-node-3 on :26660 under job pax-localnet. The monitornode scripts publish Prometheus on 9091 and scrape one target, host.docker.internal:9090, under job cryptosim. That port is the default metrics address of the storage benchmarks, not of a node.
• The only shipped Grafana dashboard is cryptosim-dashboard.json. Its 28 panels query the storage benchmark and Pebble engine series; no panel queries the pax_chain_app_* node metrics, so there is no dashboard for the ABCI and block-latency histograms yet.
• Both monitoring overlays pull unpinned :latest Prometheus and Grafana images.
• No alerting rule, recording rule or multi-node service discovery exists in the tree; the Prometheus configs that have a rule_files key leave it empty.
• The hosted gateway and registry each expose a /metrics route, but it returns JSON event-producer counters rather than Prometheus exposition format, and no scrape config targets either service.
• The gas histogram bucket set assumes the current 50M MaxGasWanted cap and would need new boundaries if that cap changes.

More in Network