invalid byte sequence in UTF-8 on encrypted binary

[djtecha]

After encrypting a binary file and placing the blob into the eyaml file using yum, all eyaml edits to the file fail with the following:

[hiera-eyaml-core] !!! invalid byte sequence in UTF-8
[hiera-eyaml-core] ["/usr/local/rvm/gems/ruby-2.0.0-p247/gems/hiera-eyaml-2.0.3/lib/hiera/backend/eyaml/parser/parser.rb:45:in match?'", "/usr/local/rvm/gems/ruby-2.0.0-p247/gems/hiera-eyaml-2.0.3/lib/hiera/backend/eyaml/parser/parser.rb:45:inblock in parse_scanner'", "/usr/local/rvm/gems/ruby-2.0.0-p247/gems/hiera-eyaml-2.0.3/lib/hiera/backend/eyaml/parser/parser.rb:44:in each'", "/usr/local/rvm/gems/ruby-2.0.0-p247/gems/hiera-eyaml-2.0.3/lib/hiera/backend/eyaml/parser/parser.rb:44:infind'", "/usr/local/rvm/gems/ruby-2.0.0-p247/gems/hiera-eyaml-2.0.3/lib/hiera/backend/eyaml/parser/parser.rb:44:in parse_scanner'", "/usr/local/rvm/gems/ruby-2.0.0-p247/gems/hiera-eyaml-2.0.3/lib/hiera/backend/eyaml/parser/parser.rb:36:inparse'", "/usr/local/rvm/gems/ruby-2.0.0-p247/gems/hiera-eyaml-2.0.3/lib/hiera/backend/eyaml/subcommands/edit.rb:57:in execute'", "/usr/local/rvm/gems/ruby-2.0.0-p247/gems/hiera-eyaml-2.0.3/lib/hiera/backend/eyaml/CLI.rb:45:inexecute'", "/usr/local/rvm/gems/ruby-2.0.0-p247/gems/hiera-eyaml-2.0.3/bin/eyaml:13:in <top (required)>'", "/usr/local/rvm/gems/ruby-2.0.0-p247/bin/eyaml:23:inload'", "/usr/local/rvm/gems/ruby-2.0.0-p247/bin/eyaml:23:in <main>'", "/usr/local/rvm/gems/ruby-2.0.0-p247/bin/ruby_noexec_wrapper:14:ineval'", "/usr/local/rvm/gems/ruby-2.0.0-p247/bin/ruby_noexec_wrapper:14:in `

'"]

[djtecha]

Follow up: This occurs on centos 6.4/7.0 and ruby-2.0.0.353-20.el7.x86_64.
Steps to reproduce:

1. eyaml encrypt -n gpg -f ~/file.bin --gpg-recipients=test --gpg-always-trust -o string
2. copy ENC[GPG,*******] to file
3. eyaml edit file.eyaml and exit

[sihil]

Hi @djtecha.

Do you have a suggestion for how we should handle this case? Perhaps catch the exception and not decrypt that particular block? It would be quite dangerous to spit out binary data into an editor IMHO.

I'm curious as to what you are using this for? A keystore or similar?

Simon

[djtecha]

So, my use case was we needed to encrypt a binary file from google so that we could pass it via puppet and not worry about someone intercepting the code as they wouldn't have the proper keys to decrypt it. I don't really have any suggestions for how this should work, but am curious as to the proper way for encrypting binary files? As I used the described method found here: https://github.com/TomPoulton/hiera-eyaml/blob/master/features/encrypts.feature
And then tried to add the output block to a file and it broke said file.

[proofpoint-mlariz]

Has anyone found a workaround for this? Just ran into this myself.

[djtecha]

Well, you can output the binary block then drop that in a file. The decryption should match your original md5sum, but you can no longer edit that file. I just included them in my hiera tree.

[sblatnick]

I am running into this problem. I tried this in my eyaml:

    project::truststore: !!binary |
      ENC[...

Is that what djtecha was suggesting? Anyone see any security risk involved?

I checked the result, and it does seem to be in binary, but the md5sums don't match.

[sblatnick]

I found a work around that can be implemented two ways, for those that need a way around this:
The problem is with loading binary in hiera, so instead, let's convert the binary to text:

1 - base64 encode the binary manually:

base64 /path/to/binary > file.txt

2 - store the base64 text in your yaml and convert it to eyaml so it's encrypted
3 - load it in puppet from hiera:

$base64 = hiera('key::name::here')
exec { 'create binary':
    command => "/usr/bin/base64 -d <<< \"${base64}\" > /path/to/binary",
}

Alternatively, if you want to control permissions and other things, you can use (untested):

  $base64 = hiera('key::name::here')
  $binary = inline_template("<%= `/usr/bin/base64 -d <<< \"${base64}\"` %>")
  file { '/path/to/binary':
    ensure  => 'present',
    path    => '/path/to/binary',
    owner   => 'root',
    group   => 'root',
    mode    => 0644,
    content  => "$binary",
    force => true,
    replace => true,
  }

[lawrenceong]

For decoding a base64 encoded string, you can also use the base64 function from puppet's stdlib. See: https://forge.puppet.com/puppetlabs/stdlib

file { '/path/to/binary':
  ...
  content => base64('decode', $base64encoded_binary_data),
  ...
}

[sblatnick]

Thanks! The worked great.