More completions for environments

[clason]

I noticed that when asking for completions on \begin{, typical suspects such as theorem, lemma, proof, etc. are missing from the list of candidates.

(I'm spoiled; vimtex has a huge list of completions generated via convert-cwl, see https://github.com/lervag/vimtex/tree/master/autoload/vimtex/complete. Also, one neat related feature is that when completing for \end{, the first match is always the environment from the last unclosed \begin{...} above.)

EDIT: the corpus also contains information about the package the environment (or command) is defined in, which can (and is by vimtex) shown as a preview (rather than "user-defined").

[pfoerster]

Which package that defines lemma and theorem are you using? As far as I know, you have to use \newtheorem{lemma}{Lemma} in order to use \begin{lemma}. TexLab should already show the package for imported commands (like \lipsum from the lipsum package). Which package name should TexLab show in the case of \newtheorem?

[clason]

As far as I know, lemma and theorem environments, as well as the \newtheorem command, are provided by the amsthm package.

[pfoerster]

The following document does not compile on my machine (TeX 3.14159265 (TeX Live 2019/Arch Linux)):

\documentclass{article}

\usepackage{amsthm}

\begin{document}

\begin{lemma}
    Foo Bar Baz
\end{lemma}

\end{document}

The log says: ! LaTeX Error: Environment lemma undefined..
One has to explicitly define the environment using \newtheorem, which is actually built-in command that gets redefined by the amsthm package.
The following document does compile on my machine:

\documentclass{article}

\newtheorem{lemma}{Lemma}

\begin{document}

\begin{lemma}
    Foo Bar Baz
\end{lemma}

\end{document}

As one has to explicitly define the environment using \newtheorem, TexLab is saying that the environment is defined by the user.

[clason]

You are of course completely right, I confused that with the proof environment. (I'm too used to working with document classes that define these environment as a standard...)

(I checked again; vimtex provides theorem as defined by default -- which in fact, like lemma, is not defined by default -- and proof as defined by amsthm, but not any others. Sorry for the sloppy issue!)

[clason]

Regarding the preselected environment on \end{: this works nicely now, thank you -- with the one niggle that it actually only works on \end{} (i.e., if the braces are already closed). This is not an issue with VS Code which auto-closes braces, but vim doesn't (by default, and many people prefer this).

To be precise, completion is triggered correctly on \end{, but the preselected property is only added when completing inside \end{}.

[pfoerster]

@clason Can you try the latest commit of the master branch? I think I have fixed the issue.

[clason]

Yes, that works now, thank you!

(One minor thing that I just noticed with a minimal example: vimtex always takes the -- only! -- completion candidate from whatever is in the \begin{...} -- so it works even if it doesn't know about the environment for completing the \begin{ itself. Not sure how relevant this is in an actual document, though, since texlab seems pretty smart about finding the possible environments to actually use.

In fact, it might make sense to not only preselect the matching environment, but return it as the only completion candidate -- I have a hard time coming up with an example where you don't want to end the last opened environment. What do you think?)

[clason]

I'm also still missing some (less common) environments such as alignat (provided by amsmath).

Is the list still being expanded, or did you decide on only providing a common subset for performance or simplicity, or is this a parsing bug? Did I prematurely close the issue?

[pfoerster]

I cannot reproduce this issue with my machine.
Can you please

• test it with the VSCode version?
• attach the file ~/.texlab/components.json (this file keeps track of all classes and packages including their defined commands and environments)?

Is the list still being expanded, or did you decide on only providing a common subset for performance or simplicity, or is this a parsing bug? Did I prematurely close the issue?

The list is always "incomplete" in the sense that it gets recomputed after every keystroke.
This is necessary because Visual Studio Code experiences severe lags when we send all possible matches at once (this is probably due to missing virtualization in the completion list).

[clason]

I cannot reproduce this on another machine, either; the other machine's components.json doesn't contain alignat, though (it's way too large to attach).

So do I understand correctly and the completion candidates are crawled from actually the included packages and stored (persistently) in ~/.texlab/components.json?

In this case, could it be that this file had been already created by an earlier version of texlab, which contained a -- for some reason, for example crashing server or prematurely closing the file, incomplete -- entry for amsmath so that package is skipped now?

I'll try removing the file and restarting and see if that helps.

[clason]

Yes, something is wrong with texlab on that machine; opening a tex file and letting texlab idle for a few minutes produces the attached (nearly empty) file. This is latest master compiled with rust nightly on Linux.
components.json.txt

Curiously, what's missing from this file is amssymb and amsthm, which are (the only ones) included via \usepackage{amssymb,amsthm}. Also missing are packages included by other packages.

The other machine is a macOS 1.0.0 binary downloaded from the releases page; here I get a 50k json, and everything works correctly (for the same tex file!)

[clason]

But even on the other machine, it doesn't give me (all) environment candidates for included packages for a more complicated project. (Commands seem to work, curiously.)

[pfoerster]

Yes, something is wrong with texlab on that machine; opening a tex file and letting texlab idle for a few minutes produces the attached (nearly empty) file. This is latest master compiled with rust nightly on Linux.

@efoerster needs to have a look at this. We will probably add some logging statements to debug the problem.

Curiously, what's missing from this file is amssymb and amsthm, which are (the only ones) included via \usepackage{amssymb,amsthm}. Also missing are packages included by other packages.

Until the latest commit, TexLab did not support the syntax \usepackage{a, b}. However, this should be fixed now.

But even on the other machine, it doesn't give me (all) environment candidates for included packages for a more complicated project. (Commands seem to work, curiously.)

Are these packages included with the syntax from above? If not, can you please give an example of an environment that is missing?

[clason]

Yes, it's (again) alignat (and (x)alignat*, even if (x)alignat is present.)

[clason]

I see the following in the stderr log (with the latest commit, which correctly picks up the full list of packages, but not their contents):

[2019-07-06 12:01:11] TRACE - Sent message:
[2019-07-06 12:01:11] Content-Length: 138
[2019-07-06 12:01:11] 
[2019-07-06 12:01:11] {"jsonrpc":"2.0","method":"window/progress/start","params":{"cancellable":false,"id":"index","message":"scrartcl.cls","title":"Indexing"}}
[2019-07-06 12:01:11] 
[2019-07-06 12:01:11] TRACE - Sent message:
[2019-07-06 12:01:11] Content-Length: 100
[2019-07-06 12:01:11] 
[2019-07-06 12:01:11] {"jsonrpc":"2.0","method":"window/progress/report","params":{"id":"index","message":"scrartcl.cls"}}
[2019-07-06 12:01:11] 
[2019-07-06 12:01:12] WARN - Component `scrartcl.cls` could not be analyzed
[2019-07-06 12:01:12] TRACE - Sent message:
[2019-07-06 12:01:12] Content-Length: 73
[2019-07-06 12:01:12] 
[2019-07-06 12:01:12] {"jsonrpc":"2.0","method":"window/progress/done","params":{"id":"index"}}

[efoerster]

The problem seems to be related to the compilation of the document. Can you please try to the compile the following document

\documentclass{scrartcl}
\listfiles
\begin{document}
\end{document}

with latex --interaction=batchmode and provide the logfile? Additionally, which TeX distribution do you use on this machine?

[pfoerster]

Regarding the issue with alignat: the starred versions of (x)alignat are indeed missing. However, the environments x(alignat) are found on my machine. Can you please check if the file ~/.texlab/components.json contains the package amsmath.sty with the environment alignat?

EDIT: If i understand you correctly, then alignat can be found inside ~/.texlab/components.json.

[clason]

@efoerster Good call! With this minimal file (and TeXlive 2019, updated daily), I get

This is pdfTeX, Version 3.14159265-2.6-1.40.20 (TeX Live 2019) (preloaded format=latex)
 restricted \write18 enabled.
---! /home/clason/.texlive2019/texmf-var/web2c/pdftex/latex.fmt made by different executable version
(Fatal format file error; I'm stymied)

Deleting the offending latex.fmt file gives the attached log, and fixes the issue ((x)alignat is a candidate) for a simple file, but still not for a multi-file project.

@pfoerster Yes, (x)alignat is in the components.json, but not (x)alignat*. For the bigger project, amsmath.sty isn't in there at all (possibly because it hasn't gotten there (yet) -- is there any way the process will stall or terminate if there are too many packages?

test.log

[pfoerster]

@clason The problem should be fixed now. (x)alignat* is now part of the completion list. In addition to that, we eliminated the need of scanning the packages at runtime. Now, a script is scanning all packages and classes in advance.

[clason]

Nice! And much more performant, too.

I still have the issue that it doesn't appear in a multi-file project -- I suspect that something prevents texlab from finding the included file where all the \usepackages are. Do you have some idea how I could debug this?

[clason]

I think I see what the problem is: I'm using my own documentclass, which itself \RequirePackages crucial packages like amsmath. These are not picked up by texlab.

To be specific, texlab doesn't consider \RequirePackage (the correct command for class files, as it doesn't require a \documentclass before it) as an alternative for \usepackage.

[pfoerster]

Thanks for the report. This was indeed the problem. TexLab should now support \RequirePackage with the latest commit on the master branch. In addition to that, \RequirePackage now triggers the completion for packages just like \usepackage.

[clason]

Indeed, that fixed it! Thanks for the quick help!

I feel really bad about this, but could it be that preselect for closing environments also needs to be adapted to \RequirePackage? It works in a single file document, but apparently not in the big one I keep on about...

[clason]

Hmm, the preselect sometimes works and sometimes doesn't. I don't see a pattern, so I'm closing this for now. If I see something that allows pinpointing the problem, I'll reopen (or open a new issue if it's not related to this).

Thank you for your patience with me!