Extra characters in keepassxc open mode attribute retrieval

[halostatue]

@twpayne unfortunately i have to come back to this issue as the fix from #3471 has side effects in open mode.
After updating and configure open mode, the password was correctly asked for but my ssh public keys are changed:

chezmoi apply
diff --git a/.ssh/id_ed25519_underverse.pub b/.ssh/id_ed25519_underverse.pub
index c4c248ca9831d6f9d8cb7a910bdb06884e4b68b2..e4418f91ecd026655301c19a84c0d2947b3976a5 100644
--- a/.ssh/id_ed25519_underverse.pub
+++ b/.ssh/id_ed25519_underverse.pub
@@ -1 +1 @@
-ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAII3ZrFoBYwruxZLLBo6OWDjppsR1On/vuC6cgWMc0j/P underverse
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAII3ZrFoBYwruxZLLBo6OWDjppsR1On/vuC6cgWMc0j/P underverse

First i thought it is a kind of whitespace stuff...but then my keychain was not happy about the files anymore:

.ssh/id_ed25519_underverse.pub is not a public key file

So i take a closer look and the generated files do have some control sequences (seems to be ending of Bracketed Paste Mode) in front of the line:

cat -A .ssh/id_ed25519_underverse.pub
^[[?2004l^Mssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAII3ZrFoBYwruxZLLBo6OWDjppsR1On/vuC6cgWMc0j/P underverse$

The used template for this is:

cat private_dot_ssh/id_ed25519_underverse.pub.tmpl
{{- keepassxcAttribute "SSH / GPG/Underverse - ed25519" "public-key" }}

The keepassxcAttachment and the keepassxc functions are working and the keepassxcAttribute is also working again after disabling the open mode.

Originally posted by @MorphBonehunter in #3468 (comment)

[halostatue]

@MorphBonehunter, could you make a "fake" keepassxc database that exhibits this behaviour and attach it? I don't use keepassxc and don't feel comfortable that I would be able to correctly replicate the issue you are seeing for debugging.

[twpayne]

I observed extra backspace characters in the output on my machine. It sounds like other terminal control characters are being added by something somewhere too. I have no idea what's causing this: they don't seem to be generated by keepassxc-cli, and chezmoi obviously doesn't add them or request them.

Setting the TERM environment variable to empty seems to help.

@MorphBonehunter do you still get the control characters in the generated file when you run

$ TERM= chezmoi apply

?

[MorphBonehunter]

Just to be sure, I also checked the output from the keepassxc-cli and as @twpayne mentioned, there is no additional chars in it:

keepassxc-cli show --attributes "public-key" --quiet --show-protected Vault.kdbx "SSH / GPG/Underverse - ed25519" | \cat -A
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAII3ZrFoBYwruxZLLBo6OWDjppsR1On/vuC6cgWMc0j/P underverse$

Indeed if i unset the TERM variable, the output is as i expected it.

But then i wonder why the keepassxc function isn't also affected as it uses the same keepassxcOutputOpen function?
I could not find any hint why the SetBracketedPasteMode could be enabled, not in the inittemplatefuncs.go nor in the expect library.

@halostatue are you still interested in an dummy database for further testing?

[halostatue]

@halostatue are you still interested in an dummy database for further testing?

I think that it might be useful to have something around for future testing, because setting TERM= feels like a hack that may not always work, but the immediate need for it has been eliminated because of the hack.

[MorphBonehunter]

Ok, i adopt the structure of my database into the attached ziped dummy database.
The password (all passwords) are chezmoi, the key is an newly generated one which isn't used anywhere.
My used template for the test is:

cat private_dot_ssh/id_ed25519_chezmoi.pub.tmpl
{{- keepassxcAttribute "SSH / GPG/Testkey - unused" "public-key" }}

And yes, the same behavior also with this database (just to be sure):

cat -A .ssh/id_ed25519_chezmoi.pub
^[[?2004l^Mssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAfuqgYoqBmhA2dKBGzfXVldOaImVuoNcJ1XFhUyU254 chezmoi$

[twpayne]

@MorphBonehunter what is the output of chezmoi doctor on your system?

[MorphBonehunter]

The output is:

chezmoi doctor
RESULT    CHECK                       MESSAGE
warning   version                     v2.45.0, built at 2024-01-13T20:55:45Z
ok        latest-version              v2.45.0
ok        os-arch                     linux/amd64 (Arch Linux)
ok        uname                       Linux cassandra 5.15.133.1-microsoft-standard-WSL2 #1 SMP Thu Oct 5 21:02:42 UTC 2023 x86_64 GNU/Linux
ok        go-version                  go1.21.6 (gc)
ok        executable                  /usr/bin/chezmoi
ok        config-file                 ~/.config/chezmoi/chezmoi.yaml, last modified 2024-01-15T16:13:11+01:00
warning   source-dir                  ~/.local/share/chezmoi is a git working tree (dirty)
ok        suspicious-entries          no suspicious entries
warning   working-tree                ~/.local/share/chezmoi is a git working tree (dirty)
ok        dest-dir                    ~ is a directory
ok        umask                       022
ok        cd-command                  found /bin/bash
ok        cd-args                     /bin/bash
info      diff-command                not set
ok        edit-command                found /usr/bin/vim
ok        edit-args                   /usr/bin/vim
ok        git-command                 found /usr/sbin/git, version 2.43.0
ok        merge-command               found /usr/sbin/vimdiff
ok        shell-command               found /bin/bash
ok        shell-args                  /bin/bash
info      age-command                 age not found in $PATH
ok        gpg-command                 found /usr/sbin/gpg, version 2.4.3
info      pinentry-command            not set
info      1password-command           op not found in $PATH
info      bitwarden-command           bw not found in $PATH
info      bitwarden-secrets-command   bws not found in $PATH
info      dashlane-command            dcli not found in $PATH
info      doppler-command             doppler not found in $PATH
info      gopass-command              gopass not found in $PATH
ok        keepassxc-command           found /usr/sbin/keepassxc-cli, version 2.7.6
ok        keepassxc-db                /mnt/c/Users/dwendler/Nextcloud/KeePassXC_dwe/Vault.kdbx is a file
info      keeper-command              keeper not found in $PATH
info      lastpass-command            lpass not found in $PATH
info      pass-command                pass not found in $PATH
info      passhole-command            ph not found in $PATH
info      rbw-command                 rbw not found in $PATH
ok        vault-command               found /usr/sbin/vault, version 1.15.4
info      vlt-command                 vlt not found in $PATH
info      secret-command              not set

[twpayne]

But then i wonder why the keepassxc function isn't also affected as it uses the same keepassxcOutputOpen function?
I could not find any hint why the SetBracketedPasteMode could be enabled, not in the inittemplatefuncs.go nor in the expect library.

I have absolutely no idea where these characters are coming from. chezmoi certainly doesn't add them, and from reading keepassxc-cli's source code, I don't see them there either.

Maybe something like ncurses is somehow hooking into the terminal control and adding these "helpful" markers, but honestly I have no idea how it would actually do this. Also, I don't know if WSL2 is doing something funny with the terminal to maybe improve integration with Windows.

@MorphBonehunter could you try a binary from #3483 and see if this fixes the problem even when you unset the TERM variable? You can download a binary at the bottom of this page.

[MorphBonehunter]

First...I think i can rule out the WSL stuff 😄
For testing i did install chezmoi on one of my other systems. This is an physical server and i have the same result, the ^[[?2004l^M stuff is also present.

Second...the downloaded binary does work.

chezmoi doctor
RESULT    CHECK                       MESSAGE
ok        version                     v2.45.0-SNAPSHOT-654516ad5, commit 654516ad58677ea10af0211f4df5a96d443f623a, built at 2024-01-15T17:39:43Z, built by goreleaser
...

It's also working after unset TERM, the files are generated without the control sequences.
For crosscheck, it is also still working without open mode with and without TERM set.

[MorphBonehunter]

Just an addition, the keepassxcAttachment and the keepassxc functions are also working in open and non open mode with the downloaded binary.
I want to make sure that the removal of the TERM env did not have any other effects of the keepass stuff.