The test function in the Assertion module only tests the last assert.

This is because only the last command output is returned and received by Assertion.Test.run

For example:

defmodule MathTest do
  use Assertion
  test "integers can be added and subtracted" do
    assert 2 + 1 == 5
    assert 5 + 5 == 10
  end
end

MathTest.run()

Output:

.

Even though assert 2 +1 == 5 fails, it doesn’t get returned so its never caught.

One solution is to have assert macro raise error to be caught by Application.Test.run.