Ark debugger support

[kevinushey]

Basic implementation in posit-dev/ark#79 and #1135.

Main missing features:

• ark: Debugger: Breakpoint support #1766 (internal preview)
• ark: Debugger: Browse dummy documents when source references are missing #1729 (MVP)
• ark: Debugger: Variable pane support #1764
• Ark: Can't use Cmd+Enter with fallback srcrefs because the virtual doc is a text file #2689

Generalised source availability:

• Ark: Debugger: Provide virtual documents for dangling source references #2282
• Ark: Debugger: Source missing srcrefs from deparsed code #2285
• Ark: Debugger: Source missing srcrefs from downloaded tarballs #2286

Fallback srcref issues:

• Ark: Duplicate virtual docs when stepping inside a function without srcrefs multiple times #2691

Reported issues:

• R: problems with auto-complete and focus when debugging #1440
• R debugger: starting debugger prompts file saving #1728
• R: Running a command in the console causes cursor to jump back to the debugged line #2786
• ark: Debugger: Watch pane support #1765

[jmcphers]

We will want to use the Debug Adapter Protocol for this since doing so will let us integrate with all of VS Code's extensive tooling and UI for debugger support.

https://microsoft.github.io/debug-adapter-protocol/

At first glance, however, I don't see a way for the debug adapter to notify the host that the debugger is active or otherwise initiate a debug session, so that might be something we need to reconcile. Perhaps we can treat it as always attached?

Debugger support is going to be quite a lot of work and may not fit for public beta.

[lionel-]

After posit-dev/ark#79 and #1135 we now have basic debugger support.

Current features:

• When a browser prompt is detected, a debugger session is automatically started on the positron side. This brings up the debugger toolbar.

• All navigation commands (step in, step out, next) are supported.

• Positron automatically jumps to the corresponding file location if any srcref is attached to the visited frame call.

• The call stack pane is supported and populated with file location support if srcrefs are attached.

• The reload button currently restarts R. We might want to tweak this later on. Perhaps remove the button entirely since it doesn't make sense to reload a debugger when the debugger is also the debugged program?

• The quit button sends Q and causes R to return to top-level. The debugger session automatically ends when a non-debugger prompt is encountered.

Missing features in order of importance (at least my understanding) for alpha users:

• Variable and Watch panes.

• Automatic srcref instrumentation in dummy files in case references are missing. For script users this might be most important. Since we are only sending complete expressions to R, we could also detect simple function definitions and use source() from a temp file to evaluate those, then adjust the srcrefs to reflect the actual file location. Or perhaps use the dummy file mechanism with a redirect to an actual file, or similar.

• Breakpoints. I haven't investigated how to support breakpoints yet. It's possible that we'll have to keep the debug session on at all times in order to add hooks that let us insert the relevant browser() calls. I noticed there is an option we might be able to use to hide the debug toolbar while R is not paused in the debugger, but I haven't explored that yet. Alternatively, there might exist a more direct VS Code API for hooking into the breakpoint UI. This would be ideal.

  browser() insertions will need to happen behind the scene, using setBreakpoint() or following RStudio's approach.

[jennybc]

I think #1440 contains some more candidates for things to work on (completion, focus).

[lionel-]

I've added a "Generalised source availability" section for creating source references from various sources that point to virtual documents. With these implemented, we'll provide R users with the same sort of full transparency experience that users of environments like Emacs enjoy. This makes it a breeze to discover and understand how dependencies are implemented and makes for a very comfortable debugging experience.

I'm currently leaning towards using virtual documents instead of unpacking as temp files, so we don't have to manage the resources or deal with users editing the files. Unfortunately this will only be available via jump to definitions / references actions, or via debugger stepping. That said the file explorer might gain access to these virtual documents in the future if this LSP extension gets implemented: https://github.com/NTaylorMullen/LSPVirtualDocuments/blob/master/Documents/FileSystemSpec.md.

Edit: oh but it looks like VS Code already supports providing a file system, so we could look at that in the future: https://code.visualstudio.com/api/references/vscode-api#FileSystemProvider

Of course it'd be even better if R packages bundled actual source files to be installed on disk. Alas for now we're stuck with source references that we have to create most of the time since repos do not actually bundle them.