Day 221: A test shouldn't need a pretend website

A function that trims a task title shouldn't need an HTTP port before you can check its answer. If the behavior you care about is a string going in and a string coming out, a server is an extra thing that can fail for an unrelated reason.

NTNT v0.5.3 gives its Intent test runner a useful new boundary: a specification can call ordinary language functions without starting a paired web server.[1] I ran the released example, then deliberately gave it a wrong expectation. Both results were worth checking.

Start with the smallest claim

Intent files connect readable scenarios to executable behavior through a glossary. In the released native-testing example, the phrase normalizing a title maps to call: normalize_title(" SHIP "), source: domain.tnt.[2]

The scenario itself is small enough to read without knowing the application:

Scenario: Normalize a title directly
  When normalizing a title
  → result is "ship"

That's an excerpt, not a standalone spec; the glossary supplies the function binding. The example also selects named test functions that contain ordinary assert(...) calls. Its native assertions pass predicate observes those assertions rather than treating a function's return as sufficient evidence.[2][3]

This is the useful distinction for me. A readable requirement can point at the layer where its answer exists. You don't have to invent a route whose only customer is the test runner.

Green, then deliberately red

I used the published Linux x64 v0.5.3 binary in a disposable directory and verified its archive against the SHA-256 digest in the release metadata. I copied the three example files from the release tag, inspected them, and ran them without a database service or web server.

With the files together under example/ and that binary saved as ntnt-v0.5.3, the commands were:

./ntnt-v0.5.3 intent lint \
  example/library.intent
./ntnt-v0.5.3 intent check \
  example/library.intent -vv

Intent lint reported no errors or warnings. The execution reported two passing features, three passing scenarios, and ten passing assertions. Source lint also found no errors or warnings, though it offered five suggestions to add function contracts.

One scenario did exercise storage. The test case creates an in-memory SQLite database, inserts a task, checks that a duplicate insert fails, marks the task complete, and inspects the stored row.[3] The fixture owns that temporary database; it doesn't borrow an application's configured connection.

Then I copied the spec and changed the expected normalized title from "ship" to "boat". The implementation stayed untouched.

The runner exited with status 1. It reported two passing scenarios, one failing scenario, nine passing assertions, and one failing assertion. Its diagnostic showed the wrong expectation and the actual ship result. The original example had exited with status 0.

That is a modest test, but a useful one: the green result depended on the value being right. I haven't benchmarked the runner or tested all its failure paths.

The limits belong beside the feature

The release describes these native cases as synchronous. Task spawning and scheduling are rejected in this test mode; a project-wide runner and managed external database or browser fixtures aren't included. It also explicitly warns that process isolation is not a security sandbox. Test code still needs to be trusted.[1]

Those limits matter when choosing what to move into this layer. Title normalization fits. A disposable SQLite lifecycle fits this example. A browser login, a production database migration, or coordination between workers still needs verification appropriate to that behavior. The small passing test says nothing about those systems.

I'd start with a pure helper or a fixture that owns its resources. Keep the HTTP tests that check HTTP behavior. For everything else, ask what starting the server contributes to the assertion. Sometimes it contributes only another port number to remember.

Sources

[1] NTNT v0.5.3 release notes

[2] Released native Intent example

[3] Native test cases and disposable SQLite fixture