Missing addresses on the command line cause unfriendly error messages in the new engine #3912

baroquebobcat · 2016-09-28T15:39:22Z

$ ./pants --enable-v2-engine filter tests/python/pants_test/:not_real
Exception caught: (<class 'pants.build_graph.address_lookup_error.AddressLookupError'>)
  File "/...../pants/src/python/pants/bin/pants_exe.py", line 50, in <module>
    main()
  File "/...../pants/src/python/pants/bin/pants_exe.py", line 44, in main
    PantsRunner(exiter).run()
  File "/...../pants/src/python/pants/bin/pants_runner.py", line 57, in run
    options_bootstrapper=options_bootstrapper)
  File "/...../pants/src/python/pants/bin/pants_runner.py", line 46, in _run
    return LocalPantsRunner(exiter, args, env, options_bootstrapper=options_bootstrapper).run()
  File "/...../pants/src/python/pants/bin/local_pants_runner.py", line 53, in run
    self._maybe_profiled(self._run)
  File "/...../pants/src/python/pants/bin/local_pants_runner.py", line 50, in _maybe_profiled
    runner()
  File "/...../pants/src/python/pants/bin/local_pants_runner.py", line 93, in _run
    self._exiter).setup()
  File "/...../pants/src/python/pants/bin/goal_runner.py", line 80, in __init__
    daemon_graph_helper
  File "/...../pants/src/python/pants/bin/goal_runner.py", line 126, in _init_graph
    return graph_helper.create_build_graph(target_roots, self._root_dir)
  File "/...../pants/src/python/pants/bin/engine_initializer.py", line 88, in create_build_graph
    for _ in graph.inject_specs_closure(target_roots.as_specs()):
  File "/...../pants/src/python/pants/engine/legacy/graph.py", line 207, in inject_specs_closure
    for address in self._inject(specs):
  File "/...../pants/src/python/pants/engine/legacy/graph.py", line 224, in _inject
    self._index(request.roots)
  File "/...../pants/src/python/pants/engine/legacy/graph.py", line 83, in _index
    'Build graph construction failed for {}:\n{}'.format(node.subject, trace))

Exception message: Build graph construction failed for SingleAddress(directory=u'tests/python/pants_test', name=u'not_real'):
  Computing LegacyTarget for SingleAddress(directory=u'tests/python/pants_test', name=u'not_real')
    Computing Addresses for SingleAddress(directory=u'tests/python/pants_test', name=u'not_real')
      Throw(exc=ResolveError(u'A Struct was not found in namespace tests/python/pants_test for name "not_real". Did you mean one of?:\n  tests/python/pants_test:int-test\n  tests/python/pants_test:base_test\n  tests/python/pants_test:test_infra\n  tests/python/pants_test:test_maven_layout',))

In v1, the error looks like

$ ./pants  filter tests/python/pants_test/:not_real
Exception caught: (<class 'pants.build_graph.address_mapper.InvalidBuildFileReference'>)
  File "/...../pants/src/python/pants/bin/pants_exe.py", line 50, in <module>
    main()
  File "/...../pants/src/python/pants/bin/pants_exe.py", line 44, in main
    PantsRunner(exiter).run()
  File "/...../pants/src/python/pants/bin/pants_runner.py", line 57, in run
    options_bootstrapper=options_bootstrapper)
  File "/...../pants/src/python/pants/bin/pants_runner.py", line 46, in _run
    return LocalPantsRunner(exiter, args, env, options_bootstrapper=options_bootstrapper).run()
  File "/...../pants/src/python/pants/bin/local_pants_runner.py", line 53, in run
    self._maybe_profiled(self._run)
  File "/...../pants/src/python/pants/bin/local_pants_runner.py", line 50, in _maybe_profiled
    runner()
  File "/...../pants/src/python/pants/bin/local_pants_runner.py", line 93, in _run
    self._exiter).setup()
  File "/...../pants/src/python/pants/bin/goal_runner.py", line 208, in setup
    context, invalidation_report = self._setup_context()
  File "/...../pants/src/python/pants/bin/goal_runner.py", line 186, in _setup_context
    self._expand_specs(self._target_specs, self._fail_fast)
  File "/...../pants/src/python/pants/bin/goal_runner.py", line 165, in _expand_specs
    for address in self._build_graph.inject_specs_closure(specs, fail_fast):
  File "/...../pants/src/python/pants/build_graph/mutable_build_graph.py", line 99, in inject_specs_closure
    fail_fast=fail_fast):
  File "/...../pants/src/python/pants/build_graph/build_file_address_mapper.py", line 186, in scan_specs
    for address in self._scan_spec(spec, fail_fast):
  File "/...../pants/src/python/pants/build_graph/build_file_address_mapper.py", line 240, in _scan_spec
    return {self.spec_to_address(spec.to_spec_string())}
  File "/...../pants/src/python/pants/build_graph/build_file_address_mapper.py", line 122, in spec_to_address
    .format(message=e, spec=spec))

Exception message: not_real was not found in BUILD files from tests/python/pants_test. Perhaps you meant one of: 
  :base_test
  :int-test
  :test_infra
  :test_maven_layout
  when translating spec tests/python/pants_test:not_real

Which I think is more clear.

The text was updated successfully, but these errors were encountered:

stuhood · 2016-11-03T19:28:09Z

Related to #4007

stuhood · 2016-11-19T20:49:21Z

Related to #3695

stuhood · 2017-04-10T19:42:01Z

Related to #4446: should probably overhaul trace a bit to render flat-and-indented like python exceptions do, and then have --print-exception-stacktrace do something intelligent in this mode as well.

baroquebobcat · 2017-04-12T23:09:27Z

pants list failure message types.

Mostly v2 says Spec {repr of spec} does not match any targets..

missing dir, implicit target name

v1

$ ./pants --no-enable-v2-engine --no-print-exception-stacktrace list missing-dir
Unknown goals: missing-dir
Use `pants goals` to list goals.
Use `pants help` to get help.

v2

$  ./pants  --no-print-exception-stacktrace list missing-dir
Unknown goals: missing-dir
Use `pants goals` to list goals.
Use `pants help` to get help.

missing dir, recursive wildcard

v1

$ ./pants --no-enable-v2-engine --no-print-exception-stacktrace list missing-dir::
Exception caught: (<class 'pants.build_graph.address_lookup_error.AddressLookupError'>)

Exception message: Can only scan directories and missing-dir is not a valid dir.

v2

$  ./pants  --no-print-exception-stacktrace list missing-dir::
Exception caught: (<class 'pants.engine.legacy.graph.InvalidCommandLineSpecError'>)

Exception message: Spec DescendantAddresses(directory='missing-dir') does not match any targets.

missing dir, single level wildcard

v1

$  ./pants --no-enable-v2-engine --no-print-exception-stacktrace list missing-dir:
Exception caught: (<class 'pants.build_graph.address_mapper.BuildFileScanError'>)

Exception message: /Users/nhoward/Workspace/pants/missing-dir does not contain any BUILD files.

v2

$ ./pants  --no-print-exception-stacktrace list missing-dir:
Exception caught: (<class 'pants.engine.legacy.graph.InvalidCommandLineSpecError'>)

Exception message: Spec SiblingAddresses(directory='missing-dir') does not match any targets.

missing dir, explicit name

v1

$  ./pants --no-enable-v2-engine --no-print-exception-stacktrace list missing-dir:name
Exception caught: (<class 'pants.build_graph.address_mapper.InvalidBuildFileReference'>)

Exception message: /Users/nhoward/Workspace/pants/missing-dir does not contain any BUILD files.
  when translating spec missing-dir:name

v2

$ ./pants --no-print-exception-stacktrace list missing-dir:name
Exception caught: (<class 'pants.engine.legacy.graph.InvalidCommandLineSpecError'>)

Exception message: Spec SingleAddress(directory=u'missing-dir', name=u'name') does not match any targets.

dir with no BUILD files at all, implicit name

v1

$ ./pants --no-enable-v2-engine --no-print-exception-stacktrace list build-support/bin
Exception caught: (<class 'pants.build_graph.address_mapper.InvalidBuildFileReference'>)

Exception message: /Users/nhoward/Workspace/pants/build-support/bin does not contain any BUILD files.
  when translating spec build-support/bin:bin

v2

$ ./pants --no-print-exception-stacktrace list build-support/bin
Exception caught: (<class 'pants.engine.legacy.graph.InvalidCommandLineSpecError'>)

Exception message: Spec SingleAddress(directory=u'build-support/bin', name=u'bin') does not match any targets.

dir with no BUILD files at all, explicit name

v1

$ ./pants --no-enable-v2-engine --no-print-exception-stacktrace list build-support/bin:name
Exception caught: (<class 'pants.build_graph.address_mapper.InvalidBuildFileReference'>)

Exception message: /Users/nhoward/Workspace/pants/build-support/bin does not contain any BUILD files.
  when translating spec build-support/bin:name

v2

$ ./pants --no-print-exception-stacktrace list build-support/bin:name
Exception caught: (<class 'pants.engine.legacy.graph.InvalidCommandLineSpecError'>)

Exception message: Spec SingleAddress(directory=u'build-support/bin', name=u'name') does not match any targets.

dir with no BUILD files at all, recursive wildcard

v1

$ ./pants --no-enable-v2-engine --no-print-exception-stacktrace list build-support/bin::
<works like ./pants list ::>

v2

$ ./pants --no-print-exception-stacktrace list build-support/bin::
Exception caught: (<class 'pants.engine.legacy.graph.InvalidCommandLineSpecError'>)

Exception message: Spec DescendantAddresses(directory='build-support/bin') does not match any targets.

nh: I think this being an error is the right call.

dir with no BUILD files at all, wildcard

v1

$ ./pants --no-enable-v2-engine --no-print-exception-stacktrace list build-support/bin:
Exception caught: (<class 'pants.build_graph.address_mapper.BuildFileScanError'>)

Exception message: /Users/nhoward/Workspace/pants/build-support/bin does not contain any BUILD files.

v2

$ ./pants --no-print-exception-stacktrace list build-support/bin:
Exception caught: (<class 'pants.engine.legacy.graph.InvalidCommandLineSpecError'>)

Exception message: Spec SiblingAddresses(directory='build-support/bin') does not match any targets.

dir with no direct BUILD files, but with ones in subdir, wildcard

v1

Exception message: /Users/nhoward/Workspace/pants/src does not contain any BUILD files.
  when translating...

v2

Spec ... does not match any targets.

dir with no direct BUILD files, but with ones in subdir, recursive wildcard

v1

Prints all recursive entries under the dir.

v2

Same.

dir with BUILD file, implicit name -- missing target

v1

$ ./pants --no-enable-v2-engine --no-print-exception-stacktrace list src/java/org/pantsbuild/tools/runner/
Exception caught: (<class 'pants.build_graph.address_mapper.InvalidBuildFileReference'>)

Exception message: runner was not found in BUILD files from src/java/org/pantsbuild/tools/runner. Perhaps you meant one of: 
  :runner-binary
  :runner-library
  when translating spec src/java/org/pantsbuild/tools/runner:runner

v2

$ ./pants --no-print-exception-stacktrace list src/java/org/pantsbuild/tools/runner
Exception caught: (<class 'pants.engine.legacy.graph.InvalidCommandLineSpecError'>)

Exception message: Spec SingleAddress(directory=u'src/java/org/pantsbuild/tools/runner', name=u'runner') does not match any targets.

dir with BUILD file, explicit name -- missing target

v1

$ ./pants --no-enable-v2-engine --no-print-exception-stacktrace list src/java/org/pantsbuild/tools/runner/:something
Exception caught: (<class 'pants.build_graph.address_mapper.InvalidBuildFileReference'>)

Exception message: something was not found in BUILD files from src/java/org/pantsbuild/tools/runner. Perhaps you meant one of: 
  :runner-binary
  :runner-library
  when translating spec src/java/org/pantsbuild/tools/runner:something

v2

$ ./pants --no-print-exception-stacktrace list src/java/org/pantsbuild/tools/runner:something
Exception caught: (<class 'pants.engine.legacy.graph.InvalidCommandLineSpecError'>)

Exception message: Spec SingleAddress(directory=u'src/java/org/pantsbuild/tools/runner', name=u'something') does not match any targets.

baroquebobcat · 2017-04-12T23:12:00Z

Missing target referenced in a build file.

v1

$ ./pants --no-enable-v2-engine --no-print-exception-stacktrace list :
Exception caught: (<class 'pants.build_graph.build_graph.TransitiveLookupError'>)

Exception message: /Users/nhoward/Workspace/pants/bar does not contain any BUILD files.
  when translating spec bar
  referenced from //:foo

v2

$ ./pants --no-print-exception-stacktrace list :
Exception caught: (<class 'pants.build_graph.address_lookup_error.AddressLookupError'>)

Exception message: Build graph construction failed for (SiblingAddresses(directory=u''), Select(Collection.of(HydratedTarget))):
Computing Select(SiblingAddresses(directory=u''), =Collection.of(HydratedTarget))
  Computing Task(<function transitive_hydrated_targets at 0x10dc1f938>, SiblingAddresses(directory=u''), =Collection.of(HydratedTarget))
    Computing TransitiveDependencies( =HydratedTarget, =BuildFileAddresses)
      Computing Task(<function hydrate_target at 0x10dc1f9b0>, bar:bar, =HydratedTarget)
        Computing Task(<function hydrate_struct at 0x10da7bed8>, bar:bar, =(symbol table types))
          Computing Task(<function resolve_unhydrated_struct at 0x10da7bf50>, bar:bar, =UnhydratedStruct)
            Computing Task(<function parse_address_family at 0x10da7b668>, Dir(path='bar'), =AddressFamily)
              Throw(Directory "Dir(path='bar')" does not contain build files.)

Currently we always print out a trace when a BUILD file has missing targets. This brings the v2 behavior more in line with v1 by adding special cases for resolve errors to ensure we produce an actionable error message. It doesn't cover transitive references of missing targets yet. See pantsbuild#3912 for some of the possible cases

### Problem There's two problems. 1. For some classes of missing targets, we always print a trace. V1 doesn't, it displays a short error message that's pretty clear. 2. For other error cases, we just say that the spec was missing without further information. ### Solution This patch brings v2's behavior more in line with v1's for the above cases. It adds special handling for `ResolveError`s in those cases. It does not attempt to improve trace generation generally, or to communicate where a non-existent address is referenced from if it has references to it. Those are follow one bits of work. See #3912 for a number of related cases.

### Problem There's two problems. 1. For some classes of missing targets, we always print a trace. V1 doesn't, it displays a short error message that's pretty clear. 2. For other error cases, we just say that the spec was missing without further information. ### Solution This patch brings v2's behavior more in line with v1's for the above cases. It adds special handling for `ResolveError`s in those cases. It does not attempt to improve trace generation generally, or to communicate where a non-existent address is referenced from if it has references to it. Those are follow one bits of work. See pantsbuild#3912 for a number of related cases.

baroquebobcat · 2017-04-25T19:06:01Z

#4509 covered the existing BUILD file cases, but not the missing BUILD file / directory ones. I'm going to put together a patch for that next.

### Problem When the engine has an issue finding BUILD files to fulfill a request for targets, it should have a good message even if the requested spec has no associated BUILD files. ### Solution This patch adds another error raising path to addresses_from_address_families so that missing directories and empty directories have improved error messages. It also * adds an error case for address mapper for unrelated Throws so that their associated exception's text will be reported if one is encountered instead of a missing field error. * removes a needless call to scheduler.root_entries, which calls into the native engine. ### Result The cases in #3912 for missing directories or directories without BUILD files have ok errors.

### Problem. Injected addresses that were bad were not being caught by the error handling introduced in the other patches for pantsbuild#3912 ### Solution. Add the checks that raise the right exceptions to inject method ### Result. Now injections will also get the more helpful error. I also removed the additional unnecessary root_entries call in inject

### Problem. Injected addresses that were bad were not being caught by the error handling introduced in the other patches for #3912 ### Solution. Add the checks that raise the right exceptions to the `_inject` method. ### Result. Now injections will also get the more helpful error. I also removed the additional unnecessary root_entries call in `_inject`.

baroquebobcat · 2017-04-27T20:01:37Z

With #4523, all of the cases mentioned here are much better. I extracted #4515 to cover the remaining tricky part. I'm closing this issue because I don't feel that's a blocker for 1.3.0.

### Problem There's two problems. 1. For some classes of missing targets, we always print a trace. V1 doesn't, it displays a short error message that's pretty clear. 2. For other error cases, we just say that the spec was missing without further information. ### Solution This patch brings v2's behavior more in line with v1's for the above cases. It adds special handling for `ResolveError`s in those cases. It does not attempt to improve trace generation generally, or to communicate where a non-existent address is referenced from if it has references to it. Those are follow one bits of work. See pantsbuild#3912 for a number of related cases.

### Problem When the engine has an issue finding BUILD files to fulfill a request for targets, it should have a good message even if the requested spec has no associated BUILD files. ### Solution This patch adds another error raising path to addresses_from_address_families so that missing directories and empty directories have improved error messages. It also * adds an error case for address mapper for unrelated Throws so that their associated exception's text will be reported if one is encountered instead of a missing field error. * removes a needless call to scheduler.root_entries, which calls into the native engine. ### Result The cases in pantsbuild#3912 for missing directories or directories without BUILD files have ok errors.

) ### Problem. Injected addresses that were bad were not being caught by the error handling introduced in the other patches for pantsbuild#3912 ### Solution. Add the checks that raise the right exceptions to the `_inject` method. ### Result. Now injections will also get the more helpful error. I also removed the additional unnecessary root_entries call in `_inject`.

baroquebobcat added bug engine labels Sep 28, 2016

kwlzn added this to the v2 engine/daemon adoption milestone Oct 7, 2016

baroquebobcat self-assigned this Nov 29, 2016

stuhood removed the bug label Mar 2, 2017

stuhood added the UX label Mar 22, 2017

stuhood modified the milestones: v2 engine: daemon-by-default beta, 1.3.0 Apr 7, 2017

baroquebobcat mentioned this issue Apr 24, 2017

[engine] Better error messages for missing targets #4509

Merged

baroquebobcat mentioned this issue Apr 25, 2017

[engine] include referring target and referred spec in missing target error message #4515

Closed

baroquebobcat mentioned this issue Apr 25, 2017

[engine] Improve error messages for missing/empty dirs #4517

Merged

baroquebobcat mentioned this issue Apr 26, 2017

[engine] pass on ResolveErrors during address injection #4523

Merged

baroquebobcat closed this as completed Apr 27, 2017

baroquebobcat mentioned this issue Feb 1, 2018

[v2-engine errors] Sort suggestions for typo'd targets. dedup errors w/o trace #5413

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Missing addresses on the command line cause unfriendly error messages in the new engine #3912

Missing addresses on the command line cause unfriendly error messages in the new engine #3912

baroquebobcat commented Sep 28, 2016

stuhood commented Nov 3, 2016

stuhood commented Nov 19, 2016

stuhood commented Apr 10, 2017

baroquebobcat commented Apr 12, 2017

baroquebobcat commented Apr 12, 2017

baroquebobcat commented Apr 25, 2017

baroquebobcat commented Apr 27, 2017

Missing addresses on the command line cause unfriendly error messages in the new engine #3912

Missing addresses on the command line cause unfriendly error messages in the new engine #3912

Comments

baroquebobcat commented Sep 28, 2016

stuhood commented Nov 3, 2016

stuhood commented Nov 19, 2016

stuhood commented Apr 10, 2017

baroquebobcat commented Apr 12, 2017

missing dir, implicit target name

v1

v2

missing dir, recursive wildcard

v1

v2

missing dir, single level wildcard

v1

v2

missing dir, explicit name

v1

v2

dir with no BUILD files at all, implicit name

v1

v2

dir with no BUILD files at all, explicit name

v1

v2

dir with no BUILD files at all, recursive wildcard

v1

v2

dir with no BUILD files at all, wildcard

v1

v2

dir with no direct BUILD files, but with ones in subdir, wildcard

v1

v2

dir with no direct BUILD files, but with ones in subdir, recursive wildcard

v1

v2

dir with BUILD file, implicit name -- missing target

v1

v2

dir with BUILD file, explicit name -- missing target

v1

v2

baroquebobcat commented Apr 12, 2017

Missing target referenced in a build file.

v1

v2

baroquebobcat commented Apr 25, 2017

baroquebobcat commented Apr 27, 2017