Newline is not removed with value coming from 1password

[alexsanduk]

Describe the bug

When value is rendered from 1password newline is not removed. It is important if you try to construct url based on username, password etc

To reproduce

$ chezmoi execute-template '{{- onepasswordRead "op://Personal/Postgres/username" -}}:{{- onepasswordRead "op://Personal/Postgres/password" -}}'
test_user
:test_password

Expected behavior

Newline is removed since {{- and -}} are used.

$ chezmoi execute-template '{{- onepasswordRead "op://Personal/Postgres/username" -}}:{{- onepasswordRead "op://Personal/Postgres/password" -}}'
test_user:test_password

Output of command with the --verbose flag

$ chezmoi execute-template --verbose '{{- onepasswordRead "op://Personal/Postgres/username" -}}:{{- onepasswordRead "op://Personal/Postgres/password" -}}'
test_user
:test_password

Output of chezmoi doctor

$ chezmoi doctor
RESULT    CHECK                MESSAGE
ok        version              v2.24.0, commit 72d9846a7ae51fd3398727d48815fc2f13a681f9, built at 2022-09-25T19:19:53Z, built by goreleaser
warning   latest-version       v2.25.0
ok        os-arch              darwin/amd64
ok        uname                Darwin MyMac 21.6.0 Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64 x86_64
ok        go-version           go1.19.1 (gc)
ok        executable           ~/.local/user/bin/chezmoi
ok        upgrade-method       replace-executable
ok        config-file          no config file found
ok        source-dir           ~/.local/share/chezmoi is a git working tree (clean)
ok        suspicious-entries   no suspicious entries
ok        working-tree         ~/.local/share/chezmoi is a git working tree (clean)
ok        dest-dir             ~ is a directory
ok        umask                022
ok        cd-command           found /bin/zsh
ok        cd-args              /bin/zsh
info      diff-command         not set
ok        edit-command         found ~/.local/user/nvim-macos/bin/nvim
ok        edit-args            ~/.local/user/nvim-macos/bin/nvim
ok        git-command          found /usr/bin/git, version 2.37.0
ok        merge-command        found /usr/bin/vimdiff
ok        shell-command        found /bin/zsh
ok        shell-args           /bin/zsh
info      age-command          age not found in $PATH
info      gpg-command          gpg not found in $PATH
info      pinentry-command     not set
ok        1password-command    found /usr/local/bin/op, version 2.7.1
info      bitwarden-command    bw not found in $PATH
info      gopass-command       gopass not found in $PATH
info      keepassxc-command    keepassxc-cli not found in $PATH
info      keepassxc-db         not set
info      keeper-command       keeper not found in $PATH
info      lastpass-command     lpass not found in $PATH
info      pass-command         pass not found in $PATH
info      passhole-command     ph not found in $PATH
info      vault-command        vault not found in $PATH
info      secret-command       not set

Additional context

[halostatue]

This looks like a bug in onePasswordRead to me, as echo .$(op read op://Personal/item/username). works as expected (outputting .halostatue. and not .halostatue\n.

The whitespace removal features {{- -}} only affects whitespace surrounding the template tags, not whitespace returned by the template tags.

$ chezmoi execute-template ':{{- "\n" }}:'
:
:⏎

You can work around this with | trim:

$ chezmoi execute-template ':{{- "\n" | trim }}:'
::⏎

So for you, that would be:

$ chezmoi execute-template --verbose '{{- onepasswordRead "op://Personal/Postgres/username" | trim -}}:{{- onepasswordRead "op://Personal/Postgres/password" | trim -}}'
test_user:test_password

We should still fix onepasswordRead; it looks like we are adding an extra \n. I think this would be easy, because adding --format json to the arguments in onepasswordtemplatefuncs.go:370 would force a JSON output, which seems to work:

$ echo .$(op read op://Personal/item/username --format json).
.halostatue.

[twpayne]

We should still fix onepasswordRead; it looks like we are adding an extra \n. I think this would be easy, because adding --format json to the arguments in onepasswordtemplatefuncs.go:370 would force a JSON output, which seems to work:

I don't think this will work in the case where the data being read from 1Password is a complete file (personal example retrieving SSH private key from 1Password). In these cases, whitespace might be significant or the file's contents might already be in JSON format.

chezmoi returns the output of op read unchanged. The newline is added by 1Password. @halostatue's solution of using trim where whitespace needs to be stripped is the right solution here I think.

[halostatue]

@twpayne is correct. op read … | xxd shows the 0x0a at the end. Sorry for the confusion. This should probably be reported as a bug to 1Password, as it means that reading a URL is less useful than it might otherwise be.

[twpayne]

Thank you for all the fixes @halostatue, both the suggestion to use trim and the code in #2499.

Hopefully this is now resolved. Please re-open if needed,

[halostatue]

I was just looking over some additional details and found this for op read:

  -n, --no-newline           Do not print a new line after the secret.

Wondering if we should look at adding that to our call of op read.

[twpayne]

Yes, I think that should be the default. Can you check which version of op added this option? Would you like to make a PR?

[halostatue]

From 2.0, when op read was introduced. It isn’t the most obvious in the help, as I only noticed it and I’ve been using this for a while (but I hadn’t been using op read).

[twpayne]

Great! So we can use --no-newline unconditionally, as you've already done in #2532. Fantastic!