Testing a package

A package is tested with the engine that will run it: the same adapter, the same fixed step, the same image. Runs off real time need no plant, no port and no privileges, so they fit a workstation and a CI job.

The interface

taktwerk inspect <package> --kind raw          # dimensions, variables, types, units, ranges
taktwerk new <package> --kind raw -o try.toml  # a project with one instance
taktwerk check try.toml                        # binds, resolves, prints the plan

inspect shows the interface exactly as the engine reads it: a wrong shape, causality or unit shows here before anything runs.

One run

taktwerk run try.toml --speed max --no-connectors --cycles 10000 \
  --set est.dims.n=4 --set est.q='[0.1, 0.1, 0.1, 0.1]' --seed 1

--speed max --no-connectors runs unpaced on engine time; a file recorder loses no row. With no recorder in the project, add one ([[recorder]] with signals = ["@io", "est.*"]).
--set binds dimensions and sets parameters, tunables and initial inputs without editing the project; --seed seeds every instance with a seed variable.
--stop-when 'est.status != 0' ends the run at the first tick that sets a status bit; the stop event names the condition.
A deterministic package gives the same bytes twice: run it again with the same seed and compare the parts’ lines in the two SHA256SUMS files.

Scenarios

A scenario file is a test of the package in a project: inputs driven, faults injected, a stop limit, a seed.

seed = 1

[stop]
time_s = 3600.0

[[event]]                 # a measurement steps
at_s = 600.0
add = "plant.y"
value = 0.5

[[event]]                 # the sensor freezes for 5 minutes
at_s = 1200.0
fault = "plant.y"
hold = true
for_s = 300.0

[[event]]                 # the reading is lost
at_s = 2000.0
fault = "plant.y"
value = nan
for_s = 60.0

[[event]]                 # the sensor drifts from here on
at_s = 2400.0
fault = "plant.y"
offset = 0.2

A fault changes what the package reads, not what the producer wrote: the producer’s output keeps its column. To record what the package received through a wired input, compose it (inputs = { y = ["plant.y"] } records as est.y). drop = true on an external input with a max_age_ms tests the stale path: the engine skips ticks, holds the outputs and publishes taktwerk.status faulted until the input is fresh again.

What to look for: the status bits set and cleared at the right ticks, outputs held or flagged while they are, no fail-stop for a bad measurement, and recovery after the fault ends.

In Python

import taktwerk

run = taktwerk.run("try.toml", scenario="scenarios/frozen-sensor.toml", seed=1)
status = run["est.status"]
frozen = [s for t, s in zip(run.t_engine, status) if 1200 <= t < 1500]
assert all(s & 255 for s in frozen[100:])          # flagged while frozen
assert status[-1] & 255 == 0                       # valid again at the end
print(run.events_of("scenario"))

taktwerk.run starts taktwerk run unpaced without connectors and returns the record of the project’s file recorder; it fits a pytest suite. See Python.

Campaigns

A campaign runs the scenarios over many parameter sets: a grid over tuning values, random samples of plant parameters ([sample]), replicates for other noise draws, [[case]] for the scenarios. Runs that differ only in case or grid point share their seed, so a comparison sees the same noise. taktwerk.read_campaign loads the results into one table.

In the loop

The write gate and the connectors run only in real time. Before a plant: a project with the plant connector and writes = "dry-run" evaluates the gate and logs what it would write (Commissioning).

Other importers

A raw package wrapped with taktwerk fmu-wrap runs in any FMI 3 importer; testing the FMU there checks that the interface is what other tools expect.

Keyboard shortcuts

taktwerk