Not decrypting/working with puppetserver 2.7.2 (Function lookup() did not find a value for the name)

[FlorianSW]

Today I wanted to upgrade my puppetmaster from 2.6.0-1puppetlabs1 to 2.7.2-1puppetlabs1 (puppet-agent from 1.7.0-1trusty to 1.9.2-1trusty) using apt-get on an Ubuntu 14.04 host. The installation worked properly and the server was working, too. However, the encrypted strings and blocks of the hiera yaml data wasn't decrypted by the puppetmaster, which resulted in replaced passwords and other private data with their encrypted versions of hiera (I had configured yaml as a fallback for eyaml).

After configuring eyaml as the only backend for hiera I got he following error:
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Function lookup() did not find a value for the name 'automysqlbackup::backup::mysql_dump_username' on node eclair.dwnet

and this log on the puppetmaster:

2017-03-03 23:04:18,809 INFO  [qtp417635516-72] [puppetserver] Puppet Caching node for eclair.dwnet
2017-03-03 23:04:27,324 INFO  [qtp417635516-73] [puppetserver] Puppet 'replace_facts' command for eclair.dwnet submitted to PuppetDB with UUID bd521661-9e19-4f06-9967-3dcfd27d29a3
2017-03-03 23:04:27,418 INFO  [qtp417635516-73] [puppetserver] Puppet Caching node for eclair.dwnet
2017-03-03 23:04:28,455 WARN  [qtp417635516-73] [puppetserver] Puppet Unknown variable: '::realm'. at /etc/puppetlabs/code/environments/production/modules/ssh/manifests/client.pp:8:8
2017-03-03 23:04:28,709 ERROR [qtp417635516-73] [puppetserver] Puppet Function lookup() did not find a value for the name 'automysqlbackup::backup::mysql_dump_username'
2017-03-03 23:04:28,710 ERROR [qtp417635516-73] [puppetserver] Puppet Function lookup() did not find a value for the name 'automysqlbackup::backup::mysql_dump_username' on node eclair.dwnet
2017-03-03 23:04:28,711 ERROR [qtp417635516-73] [puppetserver] Puppet Server Error: Function lookup() did not find a value for the name 'automysqlbackup::backup::mysql_dump_username' on node eclair.dwnet
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/lookup.rb:86:in `fail_lookup'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/lookup.rb:57:in `lookup'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/hiera/puppet_function.rb:71:in `lookup'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/hiera/puppet_function.rb:46:in `hiera_no_default'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/dispatch.rb:47:in `invoke'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/dispatcher.rb:36:in `dispatch'
org/jruby/RubyKernel.java:1242:in `catch'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/dispatcher.rb:35:in `dispatch'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/function.rb:46:in `call'
org/jruby/RubyKernel.java:1242:in `catch'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/function.rb:45:in `call'
/etc/puppetlabs/code/environments/production/modules/role/manifests/mariadb.pp:22:in `call_function'
org/jruby/RubyKernel.java:1079:in `eval'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/runtime3_support.rb:284:in `call_function'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/runtime3_support.rb:282:in `call_function'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:941:in `call_function_with_block'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:910:in `eval_CallNamedFunctionExpression'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/visitor.rb:69:in `visit_this_1'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:82:in `evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:855:in `eval_AttributeOperation'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/visitor.rb:69:in `visit_this_1'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:82:in `evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:822:in `eval_ResourceExpression'
org/jruby/RubyArray.java:1613:in `each'
org/jruby/RubyEnumerable.java:852:in `inject'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:821:in `eval_ResourceExpression'
org/jruby/RubyArray.java:2414:in `map'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:782:in `eval_ResourceExpression'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/visitor.rb:69:in `visit_this_1'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:82:in `evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:638:in `eval_BlockExpression'
org/jruby/RubyArray.java:1613:in `each'
org/jruby/RubyEnumerable.java:852:in `inject'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:638:in `eval_BlockExpression'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/visitor.rb:69:in `visit_this_1'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:82:in `evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/parser/evaluating_parser.rb:63:in `evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:32:in `evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:72:in `evaluate'
org/jruby/RubyKernel.java:1242:in `catch'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:71:in `evaluate'
org/jruby/RubyKernel.java:1242:in `catch'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:70:in `evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast.rb:31:in `safeevaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/type.rb:184:in `evaluate_code'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/resource.rb:81:in `evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/resource.rb:73:in `evaluate'
org/jruby/RubyArray.java:1613:in `each'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:395:in `evaluate_classes'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loader/../../../puppet/functions/include.rb:16:in `include'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/dispatch.rb:47:in `invoke'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/dispatcher.rb:36:in `dispatch'
org/jruby/RubyKernel.java:1242:in `catch'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/dispatcher.rb:35:in `dispatch'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/function.rb:46:in `call'
org/jruby/RubyKernel.java:1242:in `catch'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/function.rb:45:in `call'
/etc/puppetlabs/code/environments/production/manifests/site.pp:5:in `call_function'
org/jruby/RubyKernel.java:1079:in `eval'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/runtime3_support.rb:284:in `call_function'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/runtime3_support.rb:282:in `call_function'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:941:in `call_function_with_block'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:910:in `eval_CallNamedFunctionExpression'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/visitor.rb:69:in `visit_this_1'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:82:in `evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:638:in `eval_BlockExpression'
org/jruby/RubyArray.java:1613:in `each'
org/jruby/RubyEnumerable.java:852:in `inject'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:638:in `eval_BlockExpression'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/visitor.rb:69:in `visit_this_1'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:82:in `evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/parser/evaluating_parser.rb:63:in `evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:32:in `evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast.rb:31:in `safeevaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/type.rb:184:in `evaluate_code'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/resource.rb:81:in `evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/resource.rb:73:in `evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:555:in `evaluate_ast_node'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:178:in `compile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:178:in `compile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:293:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:162:in `compile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:33:in `compile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:266:in `compile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:264:in `compile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:223:in `benchmark'
/opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/1.9/benchmark.rb:295:in `realtime'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:222:in `benchmark'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:262:in `compile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:53:in `find'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:194:in `find'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:121:in `do_find'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:48:in `call'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:293:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:47:in `call'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:82:in `process'
org/jruby/RubyArray.java:1613:in `each'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:81:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:87:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:87:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:60:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:58:in `process'
file:/opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar!/puppetserver-lib/puppet/server/master.rb:42:in `handleRequest'
Puppet$$Server$$Master_1195964549.gen:13:in `handleRequest'
request_handler_core.clj:273:in `invoke'
jruby_request.clj:46:in `invoke'
jruby_request.clj:31:in `invoke'
request_handler_service.clj:34:in `handle_request'
request_handler.clj:3:in `invoke'
request_handler.clj:3:in `invoke'
core.clj:2515:in `invoke'
core.clj:211:in `invoke'
core.clj:45:in `invoke'
core.clj:343:in `invoke'
core.clj:51:in `invoke'
ringutils.clj:83:in `invoke'
master_core.clj:428:in `invoke'
ring.clj:21:in `invoke'
ring.clj:12:in `invoke'
comidi.clj:249:in `invoke'
jetty9_core.clj:424:in `invoke'
normalized_uri_helpers.clj:80:in `invoke'
2017-03-03 23:04:51,386 INFO  [qtp417635516-69] [puppetserver] Puppet Could not find filesystem info for file 'modules/admin/home/florian' in environment production
2017-03-03 23:04:51,388 INFO  [qtp417635516-69] [puppetserver] Puppet Not Found: Could not find instances in file_metadata with 'modules/admin/home/florian'
2017-03-03 23:04:54,026 INFO  [qtp417635516-71] [puppetserver] Puppet 'store_report' command for eclair.dwnet submitted to PuppetDB with UUID 711d01f9-a410-4202-a35b-dccf9dca33fb
2017-03-03 23:04:54,027 INFO  [qtp417635516-71] [puppetserver] Puppet Not sending tagmail report; no changes

I use eyaml 2.1.0 and tried already to install the gem on the puppetserver again, without any success (the installation worked, but I got the same errors). My hiera.yaml configuration:

---
:backends:
  - eyaml
#  - yaml

:hierarchy:
  - "nodes/%{::trusted.certname}"
  - common

#:yaml:
# datadir is empty here, so hiera uses its defaults:
# - /etc/puppetlabs/code/environments/%{environment}/hieradata on *nix
# - %CommonAppData%\PuppetLabs\code\environments\%{environment}\hieradata on Windows
# When specifying a datadir, make sure the directory exists.
#  :datadir:

:eyaml:
  :datadir:
  :extension: 'yaml'
  # If using the pkcs7 encryptor (default)
  :pkcs7_private_key: /etc/puppetlabs/puppet/secure/keys/private_key.pkcs7.pem
  :pkcs7_public_key: /etc/puppetlabs/puppet/secure/keys/public_key.pkcs7.pem

Decrypting the file with the eyaml command directly returns the decrypted content of the file.

This could be related to https://tickets.puppetlabs.com/browse/PUP-7215 (?)

After rolling back puppetserver and puppet-agent, anything is working now again.

[prozach]

Perhaps related to: https://docs.puppet.com/puppet/4.9/hiera_config_yaml_5.html#configuring-a-hierarchy-level-hiera-eyaml

[bazzie]

Any progess on this issue?

[rnelson0]

@FlorianSW this was tentatively linked to an issue fixed in puppet 4.9.3. Can you confirm if the problem was resolved or if it still exists?

@bazzie are you having the same issue, and if so are you at puppet >=4.9.3 or below?

[bazzie]

@rnelson0 Same issue. puppet 4.10.0 and puppetserver 2.7.2
I'm upgrading from puppet-agent 1.5.2 and puppetserver 2.3.2

[FlorianSW]

@rnelson0 Thanks for your answer. I'm still running puppet 4.7.0 (server 2.6.0). I would need to update again to see, if the problem is resolved. However, @bazzie seems to did that already?

[bazzie]

2017-07-17 08:47:47,728 ERROR [qtp1737653640-66] [puppetserver] Puppet Evaluation Error: Error while evaluating a Function Call, Class[Custom_sssd_ldap]: expects a value for parameter 'authbindpw' at /etc/puppetlabs/code/environments/vbox/modules/profiles/manifests/base_os.pp:158:11 on node box-krn0-vbx-v01
2017-07-17 08:47:47,728 ERROR [qtp1737653640-66] [puppetserver] Puppet Server Error: Evaluation Error: Error while evaluating a Function Call, Class[Custom_sssd_ldap]: expects a value for parameter 'authbindpw' at /etc/puppetlabs/code/environments/vbox/modules/profiles/manifests/base_os.pp:158:11 on node box-krn0-vbx-v01
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/types/type_mismatch_describer.rb:570:in validate_parameters' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/types/type_mismatch_describer.rb:536:in validate_parameters'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/type.rb:425:in validate_resource_hash' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/type.rb:383:in set_resource_parameters'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/type.rb:171:in evaluate_code' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/resource.rb:81:in evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in profile' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/resource.rb:73:in evaluate' org/jruby/RubyArray.java:1613:in each'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:395:in evaluate_classes' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loader/../../../puppet/functions/contain.rb:25:in contain'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/dispatch.rb:60:in invoke' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/dispatcher.rb:43:in dispatch'
org/jruby/RubyKernel.java:1242:in catch' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/dispatcher.rb:42:in dispatch'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/function.rb:46:in call' org/jruby/RubyKernel.java:1242:in catch'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/function.rb:45:in call' /etc/puppetlabs/code/environments/vbox/modules/schufa_profiles/manifests/base_os.pp:158:in call_function'
org/jruby/RubyKernel.java:1079:in eval' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/runtime3_support.rb:284:in call_function'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in profile' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/runtime3_support.rb:282:in call_function' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:941:in call_function_with_block'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:910:in eval_CallNamedFunctionExpression' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/visitor.rb:69:in visit_this_1'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:82:in evaluate' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:638:in eval_BlockExpression'
org/jruby/RubyArray.java:1613:in each' org/jruby/RubyEnumerable.java:852:in inject'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:638:in eval_BlockExpression' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/visitor.rb:69:in visit_this_1'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:82:in evaluate' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:672:in eval_CaseExpression'
org/jruby/RubyEnumerable.java:594:in find' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:656:in eval_CaseExpression'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/scope.rb:980:in with_guarded_scope' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:651:in eval_CaseExpression'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/visitor.rb:69:in visit_this_1' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:82:in evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:638:in eval_BlockExpression' org/jruby/RubyArray.java:1613:in each'
org/jruby/RubyEnumerable.java:852:in inject' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:638:in eval_BlockExpression'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/visitor.rb:69:in visit_this_1' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:82:in evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:672:in eval_CaseExpression' org/jruby/RubyEnumerable.java:594:in find'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:656:in eval_CaseExpression' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/scope.rb:980:in with_guarded_scope'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:651:in eval_CaseExpression' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/visitor.rb:69:in visit_this_1'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:82:in evaluate' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:638:in eval_BlockExpression'
org/jruby/RubyArray.java:1613:in each' org/jruby/RubyEnumerable.java:852:in inject'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:638:in eval_BlockExpression' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/visitor.rb:69:in visit_this_1'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:82:in evaluate' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/parser/evaluating_parser.rb:63:in evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:32:in evaluate' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:72:in evaluate'
org/jruby/RubyKernel.java:1242:in catch' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:71:in evaluate'
org/jruby/RubyKernel.java:1242:in catch' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:70:in evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast.rb:31:in safeevaluate' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/type.rb:184:in evaluate_code'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/resource.rb:81:in evaluate' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in profile' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/resource.rb:73:in evaluate'
org/jruby/RubyArray.java:1613:in each' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:395:in evaluate_classes'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loader/../../../puppet/functions/contain.rb:25:in contain' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/dispatch.rb:60:in invoke'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/dispatcher.rb:43:in dispatch' org/jruby/RubyKernel.java:1242:in catch'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/dispatcher.rb:42:in dispatch' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/function.rb:46:in call'
org/jruby/RubyKernel.java:1242:in catch' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/function.rb:45:in call'
/etc/puppetlabs/code/environments/vbox/modules/schufa_roles/manifests/vbox_app.pp:19:in call_function' org/jruby/RubyKernel.java:1079:in eval'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/runtime3_support.rb:284:in call_function' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in profile' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/runtime3_support.rb:282:in call_function'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:941:in call_function_with_block' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:910:in eval_CallNamedFunctionExpression'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/visitor.rb:69:in visit_this_1' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:82:in evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:638:in eval_BlockExpression' org/jruby/RubyArray.java:1613:in each'
org/jruby/RubyEnumerable.java:852:in inject' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:638:in eval_BlockExpression'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/visitor.rb:69:in visit_this_1' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/evaluator_impl.rb:82:in evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/parser/evaluating_parser.rb:63:in evaluate' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:32:in evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:72:in evaluate' org/jruby/RubyKernel.java:1242:in catch'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:71:in evaluate' org/jruby/RubyKernel.java:1242:in catch'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast/pops_bridge.rb:70:in evaluate' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/ast.rb:31:in safeevaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/type.rb:184:in evaluate_code' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/resource.rb:81:in evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in profile' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/resource.rb:73:in evaluate' org/jruby/RubyArray.java:1613:in each'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:395:in evaluate_classes' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loader/../../../puppet/functions/include.rb:16:in include'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/dispatch.rb:60:in invoke' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/dispatcher.rb:43:in dispatch'
org/jruby/RubyKernel.java:1242:in catch' /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/dispatcher.rb:42:in dispatch'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/functions/function.rb:46:in `call'
2017-07-17 08:47:47,838 INFO [qtp1737653640-70] [puppetserver] Puppet 'store_report' command for box-krn0-vbx-v01 submitted to PuppetDB with UUID 9aa75739-a05b-42dc-80b2-0393a7a36e32

[bazzie]

puppet lookup works without problems..

[root@box-krn0-vbx-v01 ~]# puppet lookup custom_sssd_ldap::authbindpw
Warning: /etc/puppetlabs/puppet/hiera.yaml: Use of 'hiera.yaml' version 3 is deprecated. It should be converted to version 5
(in /etc/puppetlabs/puppet/hiera.yaml)
Warning: /etc/puppetlabs/code/environments/vbox/modules/hiera/defaults.yaml: file does not contain a valid yaml hash
Warning: /etc/puppetlabs/code/environments/vbox/modules/hiera/box/defaults.yaml: file does not contain a valid yaml hash
--- secretpw
...

[bazzie]

My issue seems to be with custom fact interpolation.

[rnelson0]

@bazzie Sorry for the delayed response. It looks like you have resolved your issue, and it had to do with a fact not interpolating correctly leading to a cascading issue with hiera. Is that correct? If so and you think it has anything to do with changes in puppet/hiera (even indirectly) over the various versions, could you detail it to see if @FlorianSW's issue can be resolved the same way?

[FlorianSW]

I upgraded again, got the following versions, and it seems to work now, it's a bit strange, I did nothing different to the last time :]

# puppet --version
4.10.6
# puppetserver --version
puppetserver version: 2.7.2