Passing tests reported as failures when using godog junit format

[Samfucious]

The junit xml appears to be malformed.

✓ . (196ms)

=== Failed
=== FAIL: . TestMain (unknown)

--- PASS: TestMain (0.00s)

DONE 4 tests, 1 failure in 0.633s
testlib_test.go.txt

testlib.go.txt

[dnephin]

Thank you for the bug report! I suspect this is actually a bug in test2json failing to parse the test output. This has happened a few times. See #141, #105, and golang/go#38063

When there is a missing pass or fail line gotestsum has to assume that the test failed so it can print the output, as assuming the test passed would be worse. The exit status of the process is always correct, so it should be used to determine real pass or fail.

I took a quick look at the attached files. Maybe that godog library is using parallelism, or printing extra stdout which is breaking the test2json parser? If you are using an older version of Go it may not have the fix yet.

If you can reproduce the issue with a more minimal example we can open an issue on the Go issue tracker.

[Samfucious]

Thanks for the quick response. Here's a more minimal example.
My go version is 1.17, and godog version 0.12.0.
testlib_test.go.txt

[Samfucious]

It's only when using the "junit" formatting from godog that this is an issue.

Further experimentation has shown that either using "pretty" godog formatting, or not using godog at all, while running "gotestsum --junitfile " produces expected behavior.

[dnephin]

It sounds like this is expected (because the output from godog is not json), or a bug in godog not producing the correct json. I'm going to close this issue since it seems there is nothing to fix in gotestsum. Please comment if I've misunderstood and there is a problem to fix here.