Tests

What a test is in Vanta, the difference between built-in tests and Custom Tests, and when to reach for a Custom Test instead of a standard one.

In Vanta, a test is an automated configuration check that evaluates the resources in your tenant and produces pass/fail evidence against your controls. Tests are how synced data turns into the audit-ready signal that frameworks like SOC 2, ISO 27001, and HIPAA actually require.

Every test answers the same question for every resource it sees: does this resource meet the rule, or doesn’t it? That per-resource result rolls up into an overall test status (OK, NEEDS_ATTENTION, DEACTIVATED, IN_PROGRESS, INVALID, NOT_APPLICABLE) and feeds the controls the test is mapped to.

This page is about what tests are and when to author your own. To query existing test results and failing entities through the API, see Manage failing tests.

How tests work

Every test follows the same shape, regardless of whether Vanta shipped it or you built it:

1. Pick a resource type — user accounts, devices, vulnerabilities, custom resources, etc.
2. Apply a rule to each record of that type (e.g. “MFA is enabled”, “disk is encrypted”, “active = true AND lastUpdated within 30 days”).
3. Emit a per-resource pass/fail with the resource’s identifier, status, and reason.
4. Roll up to a single test status that shows on the test page and on every control the test is mapped to.
5. Re-run on a schedule as new sync data arrives, automatically flipping status when the underlying data changes.

Tests are read-only with respect to your source systems — they evaluate data Vanta has already received from an integration; they don’t reach back out to the source.

Built-in tests vs Custom Tests

There are two kinds of tests in Vanta, and the choice between them is driven entirely by what data you’re testing.

	Built-in tests	Custom Tests
Authored by	Vanta	You
Available for	Vanta-supported resource types (UserAccount, Computer, Vulnerability, BackgroundCheck, TrainingRecord, etc.)	Any resource type with synced data — including custom resources
Setup	Auto-created when you connect an integration that produces a supported resource	Authored in the Vanta Dashboard under Tests → + Create custom test
Logic	Maintained and versioned by Vanta	A pass/fail rule you define against the resource’s properties
Mutability	Versioned by Vanta; you can deactivate but not edit	Immutable after creation — copy and edit the copy if you need changes
Framework mapping	Pre-mapped to common framework controls	You map each Custom Test to the controls it satisfies
API support	Queryable via the Manage Vanta API	Queryable via the Manage Vanta API; no API to create them

Built-in tests

Built-in tests are the default. The moment you connect an integration that produces a supported resource type, Vanta auto-creates the relevant tests, runs them on the synced data, and pre-maps them to the right controls in your enabled frameworks.

You don’t author them, you don’t maintain them, and you don’t need to know what “compliant” means for UserAccount or Computer — Vanta has already encoded the audit expectations.

Reach for a built-in test when:

• You’re syncing a supported resource type and the prebuilt rule is what your auditor is going to ask about anyway.
• You want test coverage to follow the framework — when a control is added or revised by Vanta, your built-in tests track with it.
• You’d rather not own pass/fail logic, edge cases, or the audit-evidence story for that data.

Custom Tests

A Custom Test is a pass/fail rule you define on top of resource data Vanta is already syncing. You pick the resource type, write the rule (e.g. “all records have active = true AND lastUpdated within the last 30 days”), and Vanta evaluates it per record on every sync.

Custom Test logic is evaluated per record with an implicit “all records must pass” reducer — one failing record fails the whole test.

Custom Tests are immutable after creation. If you need to change the logic, name, or description later, copy the test and edit the copy — Vanta keeps the original for audit-trail purposes. Lock in your rule before mapping the test to controls.

Custom Tests are authored in the Vanta Dashboard. There is no API to create them, and they may require a plan upgrade or add-on. See Vanta Plans and Pricing or contact your Customer Success team if the + Create custom test button isn’t visible on the Tests page.

For the full authoring walkthrough, see Creating Custom Tests.

When to use a Custom Test over a standard test

Built-in tests cover the common case. Reach for a Custom Test when one of the following is true:

You’re testing a custom resource

This is the headline use case. Custom resources ship with no tests at all — Vanta doesn’t know what “compliant” means for a schema you defined yourself. Until you author a Custom Test against them, custom-resource records are passive data and contribute zero evidence to your controls.

If you’ve gone to the trouble of defining a custom resource, a Custom Test is the step that turns it into audit signal.

You need a stricter rule than the built-in test enforces

Built-in tests encode Vanta’s interpretation of the framework requirement, which is usually the audit floor. If your internal policy is stricter — for example, MFA must use a hardware key rather than any second factor, or vulnerabilities of a specific CVSS range must be remediated faster than the built-in SLA — author a Custom Test that enforces your tighter rule and map it to the same control.

Your auditor is asking for evidence on a property Vanta doesn’t already test

Built-in resource types support custom properties — fields you add to a built-in schema like UserAccount or Computer. The built-in tests don’t know about those custom properties; only a Custom Test can read them. If a control hinges on a field you added (e.g. requires_background_check = true, data_classification = "restricted"), a Custom Test is the only way to evaluate it.

You’re modeling something framework-adjacent that isn’t covered out of the box

Internal-only servers, change-management records, vendor-review artifacts, on-prem appliances, custom inventory — anything you sync as a custom resource because Vanta doesn’t natively model it. A Custom Test is what defines the compliance criteria for that data shape inside your tenant.

You want to layer a tenant-specific rule on data Vanta already pulls in

Even when Vanta has a built-in test, you may want an additional rule that’s specific to your environment — for example, “every production database resource must have an owner tag matching an active employee.” Build a Custom Test alongside the built-in one; they both contribute evidence to the controls they’re mapped to.

When not to use a Custom Test

If your data fits a supported resource type and the built-in test asks the same question your auditor will, use the built-in test. Custom Tests are a deliberate extension, not the default — every Custom Test is a rule you now own, version, and explain to an auditor.

How tests fit with integrations and resources

Tests are the third leg of the data model:

1. An integration authenticates and pushes data into Vanta.
2. That data lands as resources of a built-in or custom type.
3. Tests evaluate those resources and emit pass/fail results.
4. Test results are mapped to controls, which roll up into the frameworks you’re tracking.

The implication is clean: you can’t get useful test signal without resources, and resources don’t produce evidence without tests. A custom resource without a Custom Test, or a Custom Test without synced resource data, both produce nothing.

Lifecycle of a Custom Test

A typical Custom Test lifecycle:

1. Sync the data first. The Custom Test builder only shows resource types that have at least one record synced. Push at least one record before you start authoring.
2. Author the test in the Vanta Dashboard under Tests → + Create custom test, picking the resource type and writing the per-record rule.
3. Preview the evaluation — confirm the resource count is non-zero and the right records are flagged passing/failing before you save.
4. Lock in the rule. Once saved, the test is immutable. Changes require copying the test and editing the copy.
5. Map to controls so failing resources surface as gaps in the frameworks you care about.
6. Monitor and triage via the List Tests endpoint or the Tests page; failing entities become the work queue.

Related

Manage failing tests

Query test results and filter for failing resources via the API.

Resources

The data model tests evaluate — built-in resource types and custom resources.

Integrations

How resource data gets into Vanta in the first place.

Creating Custom Tests

Step-by-step authoring guide in the Vanta Help Center.