Error when running build on QA: Stack empty.

[pmachapman]

When I attempt to run a build on QA for a translation engine with USFM corpora, I get an error stating "Stack empty."

I have recreated this for a couple of projects, both a back translation and MT translation, with the same error. This doesn't occur when the corpora are text files (not USFM).

Build details

{
    "id": "662574068d3131ee8e61ff35",
    "url": "/api/v1/translation/engines/661dcae2624b51027a9e8561/builds/662574068d3131ee8e61ff35",
    "revision": 2,
    "engine": {
      "id": "661dcae2624b51027a9e8561",
      "url": "/api/v1/translation/engines/661dcae2624b51027a9e8561"
    },
    "trainOn": [
      {
        "corpus": {
          "id": "661dcae5624b51027a9e8564",
          "url": "/api/v1/translation/engines/661dcae2624b51027a9e8561/corpora/661dcae5624b51027a9e8564"
        },
        "textIds": [
          "PHM",
          "3JN",
          "PS2"
        ]
      }
    ],
    "pretranslate": [
      {
        "corpus": {
          "id": "661dcae5624b51027a9e8564",
          "url": "/api/v1/translation/engines/661dcae2624b51027a9e8561/corpora/661dcae5624b51027a9e8564"
        },
        "textIds": [
          "MAT",
          "2JN"
        ]
      }
    ],
    "step": 0,
    "message": "Stack empty.",
    "state": "Faulted",
    "dateFinished": "2024-04-21T20:16:09.434Z",
    "options": {
      "max_steps": 20
    }
  }

[johnml1135]

machine-job <fail> fail: SIL.Machine.AspNetCore.Services.NmtPreprocessBuildJob[0]
      Build faulted (66256fbb8d3131ee8e61ff1a)
      System.InvalidOperationException: Stack empty.
         at System.Collections.Generic.Stack`1.ThrowForEmptyStack()
         at System.Collections.Generic.Stack`1.Pop()
         at SIL.Machine.Corpora.UsfmTextBase.TextRowCollector.EndNonVerseText(UsfmParserState state, ScriptureRef scriptureRef) in /app/src/SIL.Machine/Corpora/UsfmTextBase.cs:line 243
         at SIL.Machine.Corpora

[ddaspit]

Did the stack trace get cut off?

[johnml1135]

Here is the core of the issue:

\Marker d
\Name d - Label - Descriptive Title - Hebrew Subtitle
\Description A Hebrew text heading, to provide description (e.g. Psalms)
\OccursUnder c
\Rank 4
\TextType VerseText
\TextProperties paragraph publishable vernacular level_1
\StyleType Paragraph
\FontSize 12
\Italic
\Justification Center
\SpaceBefore 4
\SpaceAfter 4

Note that the descriptors are called VerseText. When determining to push a token, UsfmParserState asks IsVerseText which returns "yes!" but adds no tokens, as it assumes that the verse has already begun. Then at the end, when the paragraph closes, it asks if the CurrentTextType is NonVerse, which it is, so it tries to pop the segment, which fails.