Alert threshold: static, relative, and baseline

The three kinds

Static. A fixed number you choose. Alert if seconds_since_last_row > 3600, or if a null rate is above 1 percent. It is easy to understand and explain, and right when the limit comes from a real requirement, such as a freshness promise to a team. It breaks when the metric grows, because last year’s floor becomes meaningless.

Relative. Compare today with its own recent past: alert if today is 20 percent below the average of the last 7 days. It follows growth automatically. It is wrong when the history is noisy or has a weekly shape, because the average is not what a normal Monday looks like.

Baseline. The system learns what normal is and flags values that sit far outside it. No number to pick, which also means no number to reason about when it fires.

What arcodash supports

arcodash monitors have two kinds, per the alerts overview.

• Threshold: a column from the first row of a saved query’s result, an operator (>, <, >=, <=, ==, !=) and a value. This covers static thresholds directly. For a relative threshold, put the arithmetic in the query and monitor the resulting percentage.
• Baseline: no operator or value. It learns from the column’s own history, with sensitivity low, medium or high. The mechanics are on metric monitoring.

The docs are direct about one limit: there is no seasonality model, so a strong weekly pattern can cause false positives. If your metric has one, compare against the same weekday in the query and use a threshold monitor.

A relative threshold in SQL

This returns today’s value, the 7-day average before it, and the percentage change. Monitor pct_change with < and a negative value, say -20.

WITH t AS (SELECT v FROM m WHERE day = current_date),
     b AS (SELECT avg(v) AS a FROM m
           WHERE day BETWEEN current_date - 7 AND current_date - 1)
SELECT t.v                                  AS today,
       round(b.a, 1)                        AS avg_7d,
       round(100.0 * (t.v - b.a) / b.a, 1)  AS pct_change
FROM t, b;

Test a threshold against history first

Before you ship a number, count how often it would have fired over the last month. Here a static floor of 960 and a 5 percent drop below the trailing 7-day average are compared on the same 30 days.

SELECT count(*) AS days_checked,
       count(*) FILTER (WHERE revenue < 960)             AS would_fire_static,
       count(*) FILTER (WHERE revenue < 0.95 * avg_7d)   AS would_fire_relative
FROM (SELECT day, revenue,
             avg(revenue) OVER (ORDER BY day
               ROWS BETWEEN 7 PRECEDING AND 1 PRECEDING) AS avg_7d
      FROM daily_revenue) t
WHERE avg_7d IS NOT NULL;

On the sample data, the static floor fires 5 times in 29 days and the relative rule 2. If you did not have 5 real incidents, that is 3 or more false alarms a month. Run this on your own metric before you commit.

Choosing, and the trade-off

• Use static when a requirement defines the limit: freshness SLAs, a null rate that must be zero for a key column.
• Use relative for volume metrics that grow and have no strong weekly shape.
• Use baseline when you have many metrics and no time to tune each, and accept the weekly-pattern caveat.

Tighter thresholds catch smaller problems and fire more often. For a metric someone checks daily, a miss costs a day. For one that gates a payout, it costs more, so tolerate the noise. Notifications fire on state change only, and a rearm period throttles flapping monitors, so repeat noise is bounded, but a bad threshold still opens incidents. See data incidents for what happens next.