Testing

Testing

Verify your Livt designs and investigate failures directly in Visual Studio Code.

Livt tests describe how a design should behave. A test component creates the design under test, drives its inputs or calls its public functions, allows time to pass where needed, and checks the result with assertions. You write these checks in Livt and run them against the generated design in simulation, without needing an FPGA board.

In Visual Studio Code, you can discover tests, run a complete project or a single scenario, and inspect failures alongside the source. This page starts with a small counter, then extends the same workflow to state, timing, and interactions between components. A passing simulation verifies the scenarios you exercise; physical timing and device resource usage still need checking in the hardware integration workflow.

Write your first test

A useful first test has a small design and a result that is easy to observe. In this example, PacketCounter records how many packets have been accepted. Create src/PacketCounter.lvt with the following contents:

livt
component PacketCounter
{
    public count: int

    public fn Accept()
    {
        count++
    }
}

The stored field count starts at zero, and each completed call to Accept() increments it. We can check these two behaviours independently: one test checks the initial state, and another checks the result of two calls. Create tests/PacketCounterTest.lvt:

livt
@Test
component PacketCounterTest
{
    counter: PacketCounter

    new()
    {
        counter = new PacketCounter()
    }

    @Test
    fn StartsEmpty()
    {
        assert counter.count == 0
    }

    @Test
    fn CountsAcceptedPackets()
    {
        counter.Accept()
        counter.Accept()

        assert counter.count == 2
    }
}

The @Test annotation identifies the component as a test suite and each annotated function as a test case. Its field and constructor form the fixture: they create the component that the tests exercise. Keep the actions that establish a scenario inside its test function, so the intended sequence is visible beside the assertions.

For a new example project, use this livt.toml. In an existing project, adapt the source and test settings rather than replacing unrelated configuration:

toml
[project]
name = "TestingDemo"
component = "PacketCounter"
path = "src"
outdir = "out"

[tests]
path = "tests"
components = ["PacketCounterTest"]

The tests.path setting locates the test sources, and tests.components selects the test components. Save the manifest and source files before running. See Project Configuration for arranging tests in a larger project.

Run tests in Visual Studio Code

Open the Livt Tests view and expand your project, test file, and test component to see the individual cases. VS Code’s Test Explorer presents the same discovered tests; use whichever view fits your workflow. The screenshots include TickCounterTest, the additional timing example introduced later on this page.

Livt Tests showing PacketCounterTest with StartsEmpty and CountsAcceptedPackets beside their source in VS Code.
The test tree connects the project, test component, and individual checks to their source.

Hover over a project, component, or test and use its run control. Running the project gives broader feedback after a change; running one test keeps an investigation focused. The extension builds the required design and starts simulation, then updates the results in the test tree and editor. Use Show Livt Test Output to follow compilation and simulation progress.

Run Livt Tests control beside CountsAcceptedPackets in the Livt Tests view.
Run an individual test from its row, or select the project or component to run a broader set.

If a test is missing

Check that the file is saved, lies under the configured test path, and contains both the component and function annotations. Confirm that the suite is included in tests.components. Use Refresh Livt Tests after correcting the configuration, and inspect discovery errors or source diagnostics if the test still does not appear. In a workspace with several projects, make sure you are looking at the intended project.

Understand test results

A passing test means its executed checks completed successfully. Review which tests actually ran: a selected test passing does not establish that the rest of the project passed, and a skipped test provides no result for that scenario. After a focused fix, run the relevant component or project again.

Passing PacketCounterTest cases and the Livt Tests output showing successful execution of the timing test, with three tests passed.
The example project passes all three tests: initial state, accepted packets, and the timing check introduced below.

Investigate an assertion failure

An assertion makes the expected result executable. In CountsAcceptedPackets, two calls should leave count at two. To see how failures are reported, temporarily change its final assertion to assert counter.count == 3 and run that test. The resulting mismatch reports an expected value of three and an actual value of two.

Failed CountsAcceptedPackets test with its assertion and VS Code failure details showing expected 3 and actual 2.
Open the failed result to compare the expectation with the observed value and inspect the associated source.

Start with the first failing check, its source location when available, and any relevant simulation output. Decide whether the implementation violates its contract, the test sets up the wrong scenario, or the expectation is incorrect. In this deliberate example the expectation is wrong: restore assert counter.count == 2, rerun the test, and then run the project.

For expectations that need context, an assertion can include a constant message, for example assert counter.count == 2, "Expected two accepted packets". Prefer checks of observable behaviour over checking every internal detail; the test should explain what consumers can rely on.

Use diagnostics and simulation output

Compilation errors and simulator setup failures prevent the test from reaching its checks. Inspect the Problems view for compiler and language-server diagnostics, and the test output for tool failures. Fix those problems before interpreting the run as a behavioural result. If the simulator is unavailable, check the installation using the Livt installation manager.

Simulation.Report can add context to a simulation, such as the operation being attempted or a value being transferred. It does not replace an assertion: printing an unexpected value does not itself fail a test. Keep simulation-only reporting in verification or debugging code outside the synthesis path. The Livt Tests output view shows progress and results; when you need complete raw simulator output, use the verbose command below.

Test state and timing

The packet counter tests an operation-level contract: after two completed calls, two packets have been counted. The calls are scheduled operations, so this does not prove that the component accepts a packet on every clock edge. For functions with value-based results, start with representative inputs and boundary values; for stored state, also check the initial state and the effect of a sequence of operations.

Check a precise number of cycles

A timing test needs an explicit model of when the design updates and when the test observes it. A short process body alone does not guarantee a one-cycle repetition rate. The following component uses a single recurring state to increment on every active clock cycle. Add it as src/TickCounter.lvt:

livt
component TickCounter
{
    public count: int

    process Count()
    {
        state Tick {
            count++
            goto Tick
        }
    }
}

Create tests/TickCounterTest.lvt to sample the count and check it again two clock edges later:

livt
@Test
component TickCounterTest
{
    counter: TickCounter

    new()
    {
        counter = new TickCounter()
    }

    @Test
    fn AdvancesTwoCycles()
    {
        var before: int
        state {
            before = counter.count
        }
        state {}
        state {
            assert counter.count == before + 2
        }
    }
}

The first state records the current count, the empty state occupies the intervening cycle, and the final state checks the count two edges after the sample. Both components use the same simulation context. Comparing against a sampled value avoids assuming that the process still has its startup value when the test function begins. Keeping the sample and assertion inside explicit states also makes their observation points clear.

Add TickCounterTest to the manifest’s tests.components list, alongside PacketCounterTest, and run the project again. All three tests should pass. See state blocks and Execution Models for how scheduled calls and suspensions affect sequencing.

When completion time can vary, give the scenario a bounded waiting interval and assert that it completed within that interval. An unlimited wait can leave a broken design simulating indefinitely. When the contract promises an exact latency, check that latency directly; adding extra waits until a test passes can hide a regression.

Test interfaces and interactions

A component often depends on a source of data, a consumer, or a shared resource. Replace a complex dependency with a small test component when that makes the interaction predictable. For example, a byte-source test double can provide a known sequence and control when data is available, allowing the consuming component to be tested without a complete UART or external device model.

Keep the same public interface as the real dependency and model the behaviours relevant to the scenario. A source that always has data can verify a normal transfer, but it cannot establish correct behaviour when data is delayed. Add focused scenarios for stalls, empty inputs, or backpressure where those are part of the contract. See Interfaces for connecting components through shared contracts.

Test register interfaces through the bus

For an HxS register interface, use Livt.Bus transactions to exercise the public bus contract. A write followed by a read can check address decoding, data layout, the invoked operation, and the returned result together. Calling the bound function directly checks its behaviour but leaves those bus connections untested.

Set a bounded transaction timeout, then cover reset values, consecutive writes, repeated reads, and relevant address boundaries. Add byte-enable, reserved-bit, selector, and delegated-window scenarios when the interface uses them. The Register Interfaces guide provides complete component and bus-test examples, including SetTimeoutTicks, Write32, and Verify32.

Check acceptance, completion, and reset

An operation can be accepted before its externally visible work is complete. For example, a successful transmit request may mean that a byte entered a buffer; it does not necessarily mean that the final serial bit has left the device. Test the acceptance result and the documented completion indication as separate behaviours.

For a cycle-sensitive interface, drive requests before the sampling edge and observe acceptance together with the payload at that edge. A clocked monitor can retain those observations for later assertions. Avoid inserting scheduled getter calls into this measurement path, because the calls can advance time and change what is being observed.

For example, suppose a one-entry slot has no empty-slot bypass, accepts a replacement when a full slot is popped, and gives clear priority over transfers. Its contract can be checked with this edge sequence. These decisions belong to this example contract; a different queue may intentionally specify different behaviour.

EdgeRequestExpected decision
0None after startupThe slot is empty.
1Push AAccept A; do not bypass it to the output.
2Push B while fullReject B and retain A.
3Pop and push BAccept both transfers; return A and store B.
4PopReturn B and leave the slot empty.
5Push A and pop while emptyAccept the push and reject the pop.
6Clear with push and popClear the slot and accept neither transfer.
7NoneThe slot remains empty.

A monitor for this scenario should check each edge, retain any mismatch, and record when the final edge has been observed. The test can then wait for a bounded interval and assert both completion and the absence of mismatches. That later assertion remains meaningful because the monitor already checked the edge-specific decisions.

Check reset as its own contract. An initial-state test and a logical-clear test do not establish correct hardware reset during an active transaction. Exercise reset assertion and release under the intended clock policy, then verify that traffic resumes correctly. For cross-clock designs, check coordinated reset and synchronised acknowledgements rather than treating one domain’s local status as a global view. See Contexts for clock and reset relationships.

Keep tests reliable

Each test should establish the state it needs and remain independent of execution order. Share construction through the fixture, but keep scenario-specific actions in the test that checks their result. Small fixtures and focused test doubles make failures easier to explain and reduce the chance that an unrelated subsystem changes the outcome.

Name tests after the behaviour they promise, and separate ordinary operation, boundary cases, reset, and error paths where that makes failures more informative. Preserve ordering, acceptance, completion, and backpressure checks when refining an implementation. If the public contract changes, update its tests and consumers together; otherwise, the existing checks should continue to describe the required behaviour.

Use the quick feedback of an individual test while developing, then run the broader suite before sharing a change. Keep assertions as the pass/fail criteria and reports as supporting context. Once the tests work locally, run the same checks automatically and retain their results with the command-line test runner.