Risk axes and tuning

Three risk axes

Every verification response includes scores, also in observe mode alongside wouldDecide and wouldGrant. Each axis is a separate integer from 0 to 100: the sum of its fired signal weights, capped at 100. The existing score is unchanged. These are risk indicators rather than probabilities.

Each reason contributes to one axis. Unmapped reasons contribute nothing. Public responses expose the numbers but no reason ids. Detailed reasons and lift tables are owner diagnostics.

// Additional field on POST /v1/verify responses
"scores": { "automation": 25, "repeat": 50, "abuse": 0 }

Label outcomes

From your backend, report confirmed fraud and legitimate outcomes using the opaque verification reference. Keep the site secret server-side. You can also label events in the dashboard.

POST https://nolgard.com/v1/feedback
Content-Type: application/json

{"secret":"sk_…","ref":"ev_123","outcome":"fraud"}

{"ok":true,"ref":"ev_123","outcome":"fraud","applied":true}

Use outcome: "legit" for a confirmed legitimate request. Labels also feed actor reputation. A Legit device label can remove its repeat limits for 30 days; only submit labels you trust.

Run tuning

Open your site's Tuning tab. Select a false-positive budget of 0.1%, 0.5%, 1% or 2%. The default is 0.5%. You need at least 20 fraud and 20 legit labels on verifications from the last 30 days. Labels on collect events are inherited by their linked verification unless it has its own label. Events without usable stored reasons are excluded and counted separately.

POST /api/sites/:key/tuning/run
{"fpBudget":0.005}

// Not enough labelled history:
{"ok":false,"error":"insufficient-labels","counts":{"fraud":12,"legit":36},"run":{…},"skipped":0}

These portal endpoints use your owner session and same-origin checks rather than your integration secret. Demo writes are forbidden. Tuning requests are limited to 20 per minute per site. The last ten runs are saved; runs expire after 90 days.

The lift table counts each reason once per event: fraud count, legit count, precision (fraud divided by all labelled events containing the reason) and recall (fraud containing the reason divided by all fraud). Precision depends on the reviewed sample rather than the rate of fraud in all traffic.

How thresholds are chosen

Tuning reconstructs the baseline bot score by summing stored scored reasons, capped at 100, excluding owner-rule results. Separately it sums repeat/abuse reasons marked unscored for the repeat/limit gate. This gate is distinct from the public repeat axis: email and referral reasons can also limit a grant.

A deterministic joint search finds the bot and repeat/limit thresholds with the highest labelled-fraud recall while keeping the combined false-positive rate within your budget. False-positive rate is flagged legit events divided by all legit labels. Ties prefer fewer false positives, then a higher bot threshold, then a higher limit threshold. Bot thresholds never fall below your current suspicious threshold. A limit threshold of 101 disables that arm. If no pair satisfies the budget, no suggestion is made.

Before and after confusion matrices show flagged fraud (TP), missed fraud (FN), flagged legit (FP), and unflagged legit (TN). Flagged means either score crosses its threshold. This is a projection of those two risk gates rather than a replay of owner rules, login challenges, verified-agent overrides or grant policy. Reason weights are not trained or changed. There is no logistic-regression model.

Apply explicitly

Running tuning never changes enforcement. Review the matrices, then press Apply thresholds.

POST /api/sites/:key/tuning/apply
{"runId":"run UUID from the response"}

{"ok":true,"thresholds":{"bot":65,"limit":50}}

Apply updates only the bot and repeat/limit score thresholds, leaving the suspicious threshold, account-count threshold, weights, mode and rules unchanged. It writes a tuning.apply audit entry with the run id and old/new thresholds. Already-applied runs, runs older than 30 days, and runs whose current thresholds changed are rejected. GET /api/sites/:key/tuning returns label counts, current thresholds and the last ten runs.

Limits

• Your labels are biased toward what you reviewed. Label legitimate random samples as well as suspicious cases.
• Twenty labels per class is a minimum rather than a level of statistical confidence. At 0.5%, fewer than 200 legit labels permit zero observed false positives. This does not prove the real rate is zero.
• The matrices evaluate the same data used to select thresholds. They are not a holdout evaluation; small samples can overfit.
• Historical labels may already reflect policy and reputation changes. Re-scoring cannot recreate missing signals or counterfactual actor counts.
• The default budget and limit-score gate of 1 are logical defaults rather than measured guarantees. Review the result and re-run monthly as traffic changes.