arcodash
Start free trial
PlatformConnectorsPricingSecurityDocs
Sign inStart free trial

Data observability monitors

arcodash has one monitor mechanism: a threshold check on one column of a saved query’s result. “Data observability monitor” isn’t a separate kind of monitor in the product — it’s four common questions people ask about their data, each answered by writing the right query and pointing a regular monitor at it. This page is the recipe for each.

[!NOTE] There’s no dedicated “check type” picker for these four — you write the query yourself and attach a plain column/operator/value monitor to it, same as any other alert. If you’re used to a tool that has a one-click freshness/row-count/duplicates wizard, the equivalent here is these four query shapes below plus the monitor you’d build for any other threshold.

Freshness

“Has this table been updated recently?” Write a query that returns how long ago the newest row was written, e.g.:

SELECT extract(epoch from now() - max(created_at)) AS seconds_since_last_row FROM events;

Monitor the seconds_since_last_row column with > and a threshold in seconds (e.g. 3600 for “alert if nothing’s landed in the last hour”). Put the query on a schedule so this check runs without anyone opening it — see Scheduled refresh.

Row count

“Did today’s load actually produce roughly the expected number of rows?” — catches a silently truncated or empty load, not just a missing one.

SELECT count(*) AS row_count FROM orders WHERE created_at::date = current_date;

Monitor row_count with < and a floor value tuned to what a normal day looks like. Consider a generous floor rather than an exact expected count — the point is catching “way off,” not flagging ordinary day-to-day variance.

Null rate

“Is a column that should always be populated silently going empty?”

SELECT
  100.0 * count(*) FILTER (WHERE email IS NULL) / nullif(count(*), 0) AS null_pct
FROM users;

Monitor null_pct with > and a small tolerance (e.g. 1 for “alert if over 1% of rows have a null email”), rather than != 0, since a handful of legitimately-null rows is often normal.

Duplicates

“Did something upstream insert the same record twice?”

SELECT count(*) AS dup_count
FROM (SELECT order_id, count(*) FROM orders GROUP BY order_id HAVING count(*) > 1) t;

Monitor dup_count with > and 0 — any duplicate at all is usually worth knowing about, unlike row count and null rate where a small nonzero value is often fine.

New Monitor form with column, operator, and value threshold filled in

Choosing thresholds

Every one of these is the same shape: pick a column that goes wrong in an obvious direction, and set a threshold with enough margin that normal variation doesn’t trigger it constantly. Start looser than you think you need — a monitor nobody trusts because it cries wolf gets muted and ignored, which defeats the point. Tighten the threshold once you’ve watched it run clean for a while.

Where to go next