Helix unable to parse stackTrace response from debugpy (Python DAP)

[rkshthrmsh]

While trying to integrate debugpy, it seems that Helix is unable to parse the stackTrace response. This is possibly because Helix is attempting to interpret Module.id and / or StackFrame.moduleId as a string instead of a number, which is what debugpy returns. The DAP spec recommends either of these types for those fields. Here is a log from Helix:

2023-03-05T18:42:12.778 mio::poll [TRACE] registering event source with poller: token=Token(5), interests=READABLE | WRITABLE
2023-03-05T18:42:12.778 mio::poll [TRACE] registering event source with poller: token=Token(6), interests=READABLE | WRITABLE
2023-03-05T18:42:12.778 helix_dap::transport [INFO] -> DAP {"type":"request","seq":0,"command":"initialize","arguments":{"adapterID":"debugpy","clientID":"hx","clientName":"helix","columnsStartAt1":true,"linesStartAt1":true,"locale":"en-us","pathFormat":"path","supportsInvalidatedEvent":false,"supportsMemoryReferences":false,"supportsProgressReporting":false,"supportsRunInTerminalRequest":true,"supportsVariablePaging":false,"supportsVariableType":true}}
2023-03-05T18:42:12.850 helix_dap::transport [INFO] <- DAP {"seq": 1, "type": "event", "event": "output", "body": {"category": "telemetry", "output": "ptvsd", "data": {"packageVersion": "1.6.6"}}}
2023-03-05T18:42:12.851 helix_dap::transport [INFO] <- DAP event Output(Output { output: "ptvsd", category: Some("telemetry"), group: None, line: None, column: None, variables_reference: None, source: None, data: Some(Object {"packageVersion": String("1.6.6")}) })
2023-03-05T18:42:12.851 helix_dap::transport [INFO] <- DAP {"seq": 2, "type": "event", "event": "output", "body": {"category": "telemetry", "output": "debugpy", "data": {"packageVersion": "1.6.6"}}}
2023-03-05T18:42:12.851 helix_dap::transport [INFO] <- DAP event Output(Output { output: "debugpy", category: Some("telemetry"), group: None, line: None, column: None, variables_reference: None, source: None, data: Some(Object {"packageVersion": String("1.6.6")}) })
2023-03-05T18:42:12.851 helix_dap::transport [INFO] <- DAP {"seq": 3, "type": "response", "request_seq": 0, "success": true, "command": "initialize", "body": {"supportsCompletionsRequest": true, "supportsConditionalBreakpoints": true, "supportsConfigurationDoneRequest": true, "supportsDebuggerProperties": true, "supportsDelayedStackTraceLoading": true, "supportsEvaluateForHovers": true, "supportsExceptionInfoRequest": true, "supportsExceptionOptions": true, "supportsFunctionBreakpoints": true, "supportsHitConditionalBreakpoints": true, "supportsLogPoints": true, "supportsModulesRequest": true, "supportsSetExpression": true, "supportsSetVariable": true, "supportsValueFormattingOptions": true, "supportsTerminateRequest": true, "supportsGotoTargetsRequest": true, "supportsClipboardContext": true, "exceptionBreakpointFilters": [{"filter": "raised", "label": "Raised Exceptions", "default": false, "description": "Break whenever any exception is raised."}, {"filter": "uncaught", "label": "Uncaught Exceptions", "default": true, "description": "Break when the process is exiting due to unhandled exception."}, {"filter": "userUnhandled", "label": "User Uncaught Exceptions", "default": false, "description": "Break when exception escapes into library code."}], "supportsStepInTargetsRequest": true}}
2023-03-05T18:42:12.851 helix_dap::transport [INFO] <- DAP success in response to 0
2023-03-05T18:42:12.851 helix_view::document [DEBUG] id 1 modified - last saved: 0, current: 0
2023-03-05T18:42:12.851 helix_term::application [DEBUG] received editor event: DebuggerEvent(Event(Output(Output { output: "ptvsd", category: Some("telemetry"), group: None, line: None, column: None, variables_reference: None, source: None, data: Some(Object {"packageVersion": String("1.6.6")}) })))
2023-03-05T18:42:12.851 helix_dap::transport [INFO] -> DAP {"type":"request","seq":1,"command":"launch","arguments":{"cwd":"/Users/rkshthrmsh/ws/ipr/ccsds_compression/proto","mode":"debug","program":"/Users/rkshthrmsh/ws/ipr/ccsds_compression/proto/test.py"}}
2023-03-05T18:42:12.851 helix_term::application [DEBUG] received editor event: DebuggerEvent(Event(Output(Output { output: "debugpy", category: Some("telemetry"), group: None, line: None, column: None, variables_reference: None, source: None, data: Some(Object {"packageVersion": String("1.6.6")}) })))
2023-03-05T18:42:12.998 helix_dap::transport [INFO] <- DAP {"seq": 4, "type": "event", "event": "initialized"}
2023-03-05T18:42:12.998 helix_dap::transport [INFO] <- DAP event Initialized(None)
2023-03-05T18:42:12.998 helix_term::application [DEBUG] received editor event: DebuggerEvent(Event(Initialized(None)))
2023-03-05T18:42:12.998 helix_dap::transport [INFO] -> DAP {"type":"request","seq":2,"command":"setBreakpoints","arguments":{"breakpoints":[{"line":2}],"source":{"path":"/Users/rkshthrmsh/ws/ipr/ccsds_compression/proto/test.py"},"sourceModified":false}}
2023-03-05T18:42:12.999 helix_dap::transport [INFO] <- DAP {"seq": 5, "type": "response", "request_seq": 2, "success": true, "command": "setBreakpoints", "body": {"breakpoints": [{"verified": true, "id": 0, "source": {"path": "/Users/rkshthrmsh/ws/ipr/ccsds_compression/proto/test.py"}, "line": 2}]}}
2023-03-05T18:42:12.999 helix_dap::transport [INFO] <- DAP success in response to 2
2023-03-05T18:42:12.999 helix_dap::transport [INFO] -> DAP {"type":"request","seq":3,"command":"configurationDone","arguments":null}
2023-03-05T18:42:12.999 helix_dap::transport [INFO] <- DAP {"seq": 6, "type": "response", "request_seq": 3, "success": true, "command": "configurationDone"}
2023-03-05T18:42:12.999 helix_dap::transport [INFO] <- DAP success in response to 3
2023-03-05T18:42:12.999 helix_dap::transport [INFO] <- DAP {"seq": 7, "type": "response", "request_seq": 1, "success": true, "command": "launch"}
2023-03-05T18:42:12.999 helix_dap::transport [INFO] <- DAP success in response to 1
2023-03-05T18:42:12.999 helix_view::editor [DEBUG] editor status: Debugged application started
2023-03-05T18:42:12.999 helix_dap::transport [INFO] <- DAP {"seq": 8, "type": "event", "event": "process", "body": {"startMethod": "launch", "isLocalProcess": true, "systemProcessId": 27185, "name": "/Users/rkshthrmsh/ws/ipr/ccsds_compression/proto/test.py", "pointerSize": 64}}
2023-03-05T18:42:12.999 helix_dap::transport [INFO] <- DAP event Process(Process { name: "/Users/rkshthrmsh/ws/ipr/ccsds_compression/proto/test.py", system_process_id: Some(27185), is_local_process: Some(true), start_method: Some("launch"), pointer_size: Some(64) })
2023-03-05T18:42:12.999 helix_view::document [DEBUG] id 1 modified - last saved: 0, current: 0
2023-03-05T18:42:12.999 helix_view::document [DEBUG] id 1 modified - last saved: 0, current: 0
2023-03-05T18:42:13.000 helix_term::application [DEBUG] received editor event: DebuggerEvent(Event(Process(Process { name: "/Users/rkshthrmsh/ws/ipr/ccsds_compression/proto/test.py", system_process_id: Some(27185), is_local_process: Some(true), start_method: Some("launch"), pointer_size: Some(64) })))
2023-03-05T18:42:13.000 helix_view::handlers::dap [WARN] Unhandled event Process(Process { name: "/Users/rkshthrmsh/ws/ipr/ccsds_compression/proto/test.py", system_process_id: Some(27185), is_local_process: Some(true), start_method: Some("launch"), pointer_size: Some(64) })
2023-03-05T18:42:13.103 helix_dap::transport [INFO] <- DAP {"seq": 9, "type": "event", "event": "thread", "body": {"reason": "started", "threadId": 1}}
2023-03-05T18:42:13.103 helix_dap::transport [INFO] <- DAP event Thread(Thread { reason: "started", thread_id: ThreadId(1) })
2023-03-05T18:42:13.103 helix_term::application [DEBUG] received editor event: DebuggerEvent(Event(Thread(Thread { reason: "started", thread_id: ThreadId(1) })))
2023-03-05T18:42:13.103 helix_view::document [DEBUG] id 1 modified - last saved: 0, current: 0
2023-03-05T18:42:13.110 helix_dap::transport [INFO] <- DAP {"seq": 10, "type": "event", "event": "stopped", "body": {"reason": "breakpoint", "threadId": 1, "preserveFocusHint": false, "allThreadsStopped": true}}
2023-03-05T18:42:13.110 helix_dap::transport [INFO] <- DAP event Stopped(Stopped { reason: "breakpoint", description: None, thread_id: Some(ThreadId(1)), preserve_focus_hint: Some(false), text: None, all_threads_stopped: Some(true), hit_breakpoint_ids: None })
2023-03-05T18:42:13.110 helix_term::application [DEBUG] received editor event: DebuggerEvent(Event(Stopped(Stopped { reason: "breakpoint", description: None, thread_id: Some(ThreadId(1)), preserve_focus_hint: Some(false), text: None, all_threads_stopped: Some(true), hit_breakpoint_ids: None })))
2023-03-05T18:42:13.110 helix_dap::transport [INFO] -> DAP {"type":"request","seq":4,"command":"threads","arguments":null}
2023-03-05T18:42:13.111 helix_dap::transport [INFO] <- DAP {"seq": 11, "type": "response", "request_seq": 4, "success": true, "command": "threads", "body": {"threads": [{"id": 1, "name": "MainThread"}]}}
2023-03-05T18:42:13.111 helix_dap::transport [INFO] <- DAP success in response to 4
2023-03-05T18:42:13.111 helix_dap::transport [INFO] -> DAP {"type":"request","seq":5,"command":"stackTrace","arguments":{"threadId":1}}
2023-03-05T18:42:13.112 helix_dap::transport [INFO] <- DAP {"seq": 12, "type": "response", "request_seq": 5, "success": true, "command": "stackTrace", "body": {"stackFrames": [{"id": 2, "name": "<module>", "line": 2, "column": 1, "source": {"path": "/Users/rkshthrmsh/ws/ipr/ccsds_compression/proto/test.py", "sourceReference": 0}}], "totalFrames": 1}}
2023-03-05T18:42:13.112 helix_dap::transport [INFO] <- DAP success in response to 5
2023-03-05T18:42:13.112 helix_dap::transport [INFO] <- DAP {"seq": 13, "type": "event", "event": "module", "body": {"reason": "new", "module": {"id": 0, "name": "__main__", "path": "/Users/rkshthrmsh/ws/ipr/ccsds_compression/proto/test.py"}}}
2023-03-05T18:42:13.112 helix_dap::transport [INFO] -> DAP {"type":"request","seq":6,"command":"stackTrace","arguments":{"threadId":1}}
2023-03-05T18:42:13.112 helix_dap::transport [ERROR] err: <- Parse(Error("invalid type: integer `0`, expected a string", line: 0, column: 0))
2023-03-05T18:42:13.112 mio::poll [TRACE] deregistering event source from poller
2023-03-05T18:42:33.129 helix_view::document [DEBUG] id 1 modified - last saved: 0, current: 0
2023-03-05T18:42:33.130 helix_view::editor [DEBUG] editor status: Thread 1 stopped because of breakpoint (all threads stopped)
2023-03-05T18:42:33.130 helix_view::document [DEBUG] id 1 modified - last saved: 0, current: 0
2023-03-05T18:42:33.130 helix_view::document [DEBUG] id 1 modified - last saved: 0, current: 0

Related issue in debugpy repo: microsoft/debugpy#1227

[the-mikedavis]

There's a TODO for supporting numbers for Module.id here:

helix/helix-dap/src/types.rs

Line 314 in 3905243

pub id: String, // TODO: || number

[thomasaarholt]

I believe I have a fix (will PR) for the "String || Number" problem, but I seem to have a different bug. Could you, @rkshthrmsh, provide your languages.toml config for the debugger? Do you still get to the above error when you try to run :debug-start, or do you now get an error even sooner than that one?

Using the debug config provided by @RmStorm, I get the error "No such file (...) "<directory-open-in-helix>/{0}". I think the {0} is supposed to be converted into the path of the open file, but instead its being interpreted literally.

In @RmStorm's config, if I replace args = { mode = "debug", program = "{0}" } with args = { mode = "debug", program = "run.py" }, so that I am now hardcoding the filename run.py, I can successfully run the debug.

[rkshthrmsh]

Hi @thomasaarholt, here's my languages.toml:

[[language]]
name = "python"

[language.debugger]
name = "debugpy"
transport = "stdio"
command = "python3"
args = ["-m",  "debugpy.adapter"]

[[language.debugger.templates]]
name = "source"
request = "launch"
completion = [ { name = "entrypoint", completion = "filename", default = "." } ]
args = { mode = "debug", program = "{0}"}

Hardcoding the file name, as you mentioned, did not have any effect for me. The program was still suspended by my terminal, and the terminal still got messed up. I tried providing the full path to the file as well. Was there something else you tried?

[thomasaarholt]

I've opened #7637, since I at least experience the {0} in multiple debuggers (rust and python). I think its either a regression, or something wrong with my system. It would be nice for me to get confirmation if its the latter.

I still get a messed-up terminal when things crash, but if I replace {0} with run.py (my script's name), and run hx -v run.py followed by :debug-start, I get one step further in the debugging process, and the error messages are different in the log (:log-open). I'm able to fix this second error in my bugfix PR (not submitted yet), but I still do have more errors that are similar.

The first error (using {0}) is:

FileNotFoundError: [Errno 2] No such file or directory: '/Users/thomas/codes/test-fabrica/{0}'

The second error, using run.py is:

2023-07-15T17:49:22.828 helix_dap::transport [ERROR] err: <- Parse(Error("invalid type: integer `0`, expected a string", line: 0, column: 0))

Make sure you are on a new-ish build of helix. You can see the logs using :log-open and scroll to bottom.

[Desdaemon]

I still get a messed-up terminal when things crash

Can confirm, though it's only at the beginning. It seems as though Helix was being forcefully suspended with Ctrl+Z, causing some terminals like Kitty to freak out. I then used fg to return to Helix, and debugging kind of works. I did apply a patch for correctly parsing Module.id as well.

[carschandler]

Any update on this? Would love to spend some more time in Helix, but as a Python developer the lack of support for debugpy is holding me back slightly.

[mcfetz]

I have the same problem. In the meantime I use helix in combination with the pudb debugger in a tmux session. Works for me quite well.

[slawomirlech]

Since there was no progress on this issue I proposed my solution for problem with parsing. Helix still breaks terminal on hitting any breakpoint, but I believe this might be one step forward solving underlying issue

[andersfylling]

I currently use Helix for my main work, but as I need to debug a lot of python code I have to have Jetbrains in addition. Now I'm struggling with poetry issues due to a Jetbrain bug so I have to do side-IDE hopping to compensate 😄

I'm very interested in this, but I don't know rust. Does anyone know the specific places that has to be changed to support this? Anyone more experienced that knows how we could make it easier for us nuubs to jump in and contribute to this part? Could we instead of supporting ints, just transform integers into strings beforehand somehow?