[Bug] Twig language is not tokenized correctly following arbitrary text

[claylibrarymarket]

Reproducible in vscode.dev or in VS Code Desktop?

• Not reproducible in vscode.dev or VS Code Desktop

Reproducible in the monaco editor playground?

• Not reproducible in the monaco editor playground

Monaco Editor Playground Link

https://microsoft.github.io/monaco-editor/playground.html?source=v0.39.0#XQAAAAK1AQAAAAAAAABBqQkHQ5NjdNsB--jY7SIQ9S7DNlzs5W-mwj0fe1ZCDRFc9ws-BJqVKAuo8RmGmaXErfyRZJ0vWOYpCTsNDP_s_DNT3H_TlPKlr4B7-RZlcXRvkRWqcGq2SFRad0lLGMGd_Vs60GQVX2i4lRn1gwe1SnaKylvbf1QywH_5HdOyjP2qZEUMliQnTEW7eRqRI_NplD0MHXxa454CIsLuP3UPVxK3TaJD362c5uHjGVsg4MEmWiWJEhInvTR636LjpSIip9NpN1jjBCpzwlnD7Vcwlio10h7r_KC8y8AHQH9cP-wvFGzg2SJomSs4JSAT7bujjzjcPENGcsLFVgUp1U-ohL-zb75ArHP9K_rupXANH8TDOnMWmFivOEvh7SvU_6ejs3M

Monaco Editor Playground Code

monaco.editor.colorize('{# initializing ... #}', 'twig', {}).finally(() => {
	const value = JSON.stringify({
		incorrect: monaco.editor.tokenize('test {{ var }}', 'twig'),
		correct: monaco.editor.tokenize('{{ var }}', 'twig'),
	}, null, 4);

	const editor = monaco.editor.create(document.getElementById('container'), {
		language: 'json',
		value
	});
});

Reproduction Steps

Attempt to tokenize Twig code with some arbitrary leading text.

Actual (Problematic) Behavior

The Twig code is not tokenized properly.

Expected Behavior

The Twig code should be tokenized properly.

Additional Context

I believe a small change to the last regular expression in tokenizer.root in this file would resolve this issue:

• /[^<]+/ => /[^<{]+/.