Sserebryakov/my build graph #1
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Replace deprecated `AggressiveCompile.compile1` usage with `IC.incrementalCompile` - Add workaround class to expose an `Analysis.empty` method that takes the nameHashing parameter Unfortunately, I found out after these changes that neither Fedor's nor my fix made the 0.13.9 release. I started working to backport them here (and may post that later) but ran into other issues that I'm looking into. Meanwhile, it seemed worth it to get this in at least. Testing Done: pantsbuild#1985 a local publish looked sane, and compiled/tested all examples cleanly Reviewed at https://rbcommons.com/s/twitter/r/2629/
Junit tests now run using a java Distribution found to match the test_platform.target_level of the test target. By default any jvm whose version is *at least* the target_level is used. Using jvms which are of the exact same version of the junit_tests can be forced via the --strict-jvm-version flag. If the test_platform isn't specified, it is assumed to be the same as the platform (inherited from jvm_target). Testing Done: Added tests under testprojects/tests/org/pantsbuild/testproject/testjvms, run by integration tests in tests/python/pants_test/backend/jvm/tasks/test_junit_run_integration.py. CI went green here: https://travis-ci.org/pantsbuild/pants/builds/76115240 CI went green here: https://travis-ci.org/pantsbuild/pants/builds/76163217 Bugs closed: 2008 Reviewed at https://rbcommons.com/s/twitter/r/2651/
The existing impl did not include the severity in its match, so it's possible that errors could be filtered.
- Only filter non-errors
- Filter zinc compilation at the Reporter level, rather than in the logger
- Split between message filters and filename filters
Testing Done:
Used:
./pants --config-override=pants.ini.isolated publish.jar --no-dryrun --local=~/.m2/repository --named-snapshot=$USER-`date +%s` src/scala/org/pantsbuild/zinc::
...to do a local publish, then tested that:
./pants --config-override=pants.ini.isolated compile testprojects/src/java/org/pantsbuild/testproject/dummies::
properly applied both `-msg-filter` and `-file-filter`.
----
Can add a pants integration test once this release goes out.
Reviewed at https://rbcommons.com/s/twitter/r/2656/
…ogging;0.0.2 # Prevent Travis-CI from running for this automated jar publish commit: # http://docs.travis-ci.com/user/how-to-skip-a-build/ [ci skip]
…bt-inc;0.0.1 # Prevent Travis-CI from running for this automated jar publish commit: # http://docs.travis-ci.com/user/how-to-skip-a-build/ [ci skip]
….0.8 # Prevent Travis-CI from running for this automated jar publish commit: # http://docs.travis-ci.com/user/how-to-skip-a-build/ [ci skip]
…s to a remote repo that requires authentication, pants will duplicate the username/password on the CLI, one for each artifact published. Reviewed at https://rbcommons.com/s/twitter/r/2560/
Introduces the concept of a subsystem client, which Task and Subsystem mix in (this is not in Optionable because the GlobalOptionsRegistrar must not depend on any subsystems). All subsystem dependencies are expressed by the client by implementing subsystem_dependencies(). That dependency consists of a subsystem type and a scope. This generalizes the previous concept of global and task-specific subsystem dependencies. Contains a temporary hack so that the old declarations still work. Those will be mopped up in a followup change, just to keep this one from spiraling out too much. Issue: pantsbuild#1957 Testing Done: CI passes: https://travis-ci.org/pantsbuild/pants/builds/76136929. Bugs closed: 1957 Reviewed at https://rbcommons.com/s/twitter/r/2653/
To delay various classpath-length related issues, use short unique jar names.
After:
$ ls -1 .pants.d/compile/jvm/zinc/jars/10ba29e1edb1.jar | wc -c
48
Testing Done:
pantsbuild#2009
Reviewed at https://rbcommons.com/s/twitter/r/2647/
GoFetch now supports remote libraries with nested packages as well as
implicit targets for transitive dependencies that fall within the same
remote import path root as an explicitly declared remote dependency.
The awkward fit of zip_url to the more generically named GoRemoteLibrary
is fixed by moving the task of fetching fully out to a Fetchers
subsystem that works by delegating based on matches against the remote
import path. To support this, both GoRemoteLibrary and GoLocalSource
targets gain a consistent notion of `import_path` that matches the same
concept in go.
The existing zip download functionality for remote libraries on github
is extracted to a generic ArchiveFetcher implementation with a default
configuration that knows how to construct and github archive URLs and
unpack them.
Additionally, a 'go_remote_libraries' macro is introduced to allow for
depending on a suite of packages at a single revision without repeating
the version in the BUILD file.
Testing Done:
Tested this in Medium's mono-repo in a manual test case that's now
replicated by the `GoFetchIntegrationTest.test_issues_1998` test.
Also some ad-hoc testing of the fetcher options like so (flip default
github `.tar.gz` fetch to `.zip`):
```console
$ ./pants \
--fetchers-mapping="{r'github.com/':'ArchiveFetcher'}" \
--archive-fetcher-matchers="{r'github.com/(?P<user>[^/]+)/(?P<repo>[^/]+)':('https://github.com/\g<user>/\g<repo>/archive/{rev}.zip', 'master', 1)}" \
compile contrib/go/examples/3rdparty/go/github.com/AdRoll/goamz:dynamodb
...
13:24:38 00:00 [resolve]
13:24:38 00:00 [ivy]
13:24:38 00:00 [cache].
13:24:38 00:00 [go]
Invalidated 1 target.INFO] Downloading https://github.com/AdRoll/goamz/archive/51b70b0035cf28c4ca6d719156347ac56a41d922.zip...
Invalidated 3 targets.INFO] Downloading https://github.com/bitly/go-simplejson/archive/v0.4.3.zip...
Invalidated 1 target.INFO] Downloading https://github.com/cbroglie/mapstructure/archive/25325b46b67d1c3eb7d58bad37d34d89a31cf9ec.zip...
...
$ $ tree .pants.d/compile/go/contrib.go.examples.3rdparty.go.github.com.AdRoll.goamz.dynamodb/pkg/linux_amd64/github.com/AdRoll/goamz/
.pants.d/compile/go/contrib.go.examples.3rdparty.go.github.com.AdRoll.goamz.dynamodb/pkg/linux_amd64/github.com/AdRoll/goamz/
├── aws.a -> /home/jsirois/dev/3rdparty/jsirois-pants4/.pants.d/compile/go/contrib.go.examples.3rdparty.go.github.com.AdRoll.goamz.aws/pkg/linux_amd64/github.com/AdRoll/goamz/aws.a
├── dynamodb
│ └── dynamizer.a -> /home/jsirois/dev/3rdparty/jsirois-pants4/.pants.d/compile/go/contrib.go.examples.3rdparty.go.github.com.AdRoll.goamz.dynamodb.dynamizer/pkg/linux_amd64/github.com/AdRoll/goamz/dynamodb/dynamizer.a
└── dynamodb.a
1 directory, 3 files
```
CI went green here:
https://travis-ci.org/pantsbuild/pants/builds/76318249
Bugs closed: 1998, 2013
Reviewed at https://rbcommons.com/s/twitter/r/2655/
This pushes deprecation of 2 options forward to the regularly shceduled release this coming Friday. Testing Done: CI went green here: https://travis-ci.org/pantsbuild/pants/builds/76342402 Bugs closed: 2023 Reviewed at https://rbcommons.com/s/twitter/r/2661/
The `Setup` object is used as a cache key for compiler instances, and it turns out that missing the cache for compiler instances causes a more than 2x slowdown. In this case, as soon as the filters were non-empty, the `Regex` object would fail to match (doesn't implement equals.) - Remove log options from the Setup, since they are passed into the compile method each time anyway. Testing Done: pantsbuild#2031 manually verified that the slowdown with filters is gone, and that filtering still works Reviewed at https://rbcommons.com/s/twitter/r/2666/
….0.9 # Prevent Travis-CI from running for this automated jar publish commit: # http://docs.travis-ci.com/user/how-to-skip-a-build/ [ci skip]
Just removed the lines including them, so that ci can pick them up properly by pattern matching, and ignore them otherwise. Testing Done: CI went green: https://travis-ci.org/pantsbuild/pants/builds/76369527 Bugs closed: 2027 Reviewed at https://rbcommons.com/s/twitter/r/2664/
- Adds the --binaries-path-by-id advanced option - Added case in binary_util.py for MacOS release El Capitan Testing Done: CI passed at https://travis-ci.org/pantsbuild/pants/builds/76471432 Bugs closed: 2026 Reviewed at https://rbcommons.com/s/twitter/r/2663/
This is useful when preparing BUILD files for versions of pants that aren't ready to be used yet. From discussion on pants-devel: Ignoring known unconsumed target kwargs. This change was controversial/distasteful to many on the pants-devel discussion, but was accepted because it was a fairly simple change that doesn't add much complexity. Testing Done: Added integration test. CI went green: https://travis-ci.org/pantsbuild/pants/builds/76363691 CI went green: https://travis-ci.org/pantsbuild/pants/builds/76506130 Bugs closed: 2025 Reviewed at https://rbcommons.com/s/twitter/r/2662/
Distribution.locate now first checks the paths set in the jdk_paths dictionary (according to the os name). Changed option reference in unrelated test to avoid collision with jvm subsystem. Moved some common os-name logic to a new osutil.py, rather than duplicating it in jvm.py. Testing Done: Added tests to tests/python/pants_test/java/distribution/test_distribution_integration.py. CI went green here: https://travis-ci.org/pantsbuild/pants/builds/76301770 CI went green here: https://travis-ci.org/pantsbuild/pants/builds/76310316 CI went green here: https://travis-ci.org/pantsbuild/pants/builds/76477412 CI went green here: https://travis-ci.org/pantsbuild/pants/builds/76509163 Bugs closed: 2014 Reviewed at https://rbcommons.com/s/twitter/r/2657/
Testing Done: Locally ran `./pants test contrib/go/::`. CI went green here: https://travis-ci.org/pantsbuild/pants/builds/76523378 Bugs closed: 2034 Reviewed at https://rbcommons.com/s/twitter/r/2669/
This necessitated adding support for the gopkg.in fetch protocol. Add a flag to turn on testing of remote libraries and use that to create an integration test for pantsbuild#2004. Testing Done: CI went green here: https://travis-ci.org/pantsbuild/pants/builds/76550920 Bugs closed: 2004, 2036 Reviewed at https://rbcommons.com/s/twitter/r/2671/
Kill `[compile.java]` `source` and `target` options in favor of
`[jvm-platform]` options.
What was:
```ini
[compile.java]
source: 7
target: 7
```
Should now be:
```ini
[jvm-platform]
platforms: {
'java7': {
'source': 7
'target': 7
},
}
default_platform: java7
```
Testing Done:
CI is away here:
https://travis-ci.org/pantsbuild/pants/builds/76664815
Bugs closed: 2042
Reviewed at https://rbcommons.com/s/twitter/r/2678/
Testing Done: Ran the following green locally after the fixes: ```console ./pants test \ tests/python/pants_test/backend/jvm/tasks/jvm_compile/java:java_compile_integration \ tests/python/pants_test/backend/core/tasks:cache_cleanup ``` CI went green here: https://travis-ci.org/pantsbuild/pants/builds/76676814 Bugs closed: 2043 Reviewed at https://rbcommons.com/s/twitter/r/2680/
This adds support for both dynamically discovering and injecting
Go targets into the in-memory BuildGraph as well as adding support
for materializing BUILD files for those targets to disk.
The target creation scope can optionally include direct remote
dependencies, but otherwise defaults to only handling local source
targets.
Additionally, a simple templating system is provided to allow for
BUILD materialization that suits the local BUILD style.
To support some of the BUILD gen calculations and keep path calculation
code DRY, utilities are moved into the go target hierarchy.
Auto re-gen Go BUILD files in the repo with:
`./pants buildgen.go --remote --materialize ::`
Also used this task to generate Go BUILD graphs in the Medium mono repo
by just seeding go_binary targets for their main packages' dirs with:
`echo "go_binary(name='[parent dir]')" > BUILD`
and then running `buildgen.go`, finally pinning revs in the BUILD files
generated to their `3rdparty/golang` `go_remote_library` source root.
Testing Done:
New tests for GoBuildgen:
```console
$ ./pants test.pytest --coverage=modules:pants.contrib.go.tasks.go_buildgen contrib/go/tests/python/pants_test/contrib/go/tasks/:go_buildgen
...
contrib/go/tests/python/pants_test/contrib/go/tasks/test_go_buildgen.py ................
=========== 16 passed in 1.97 seconds ============
Name Stmts Miss Branch BrMiss Cover
--------------------------------------------------------------------------------------------
contrib/go/src/python/pants/contrib/go/tasks/go_buildgen 192 4 72 8 95%
```
An example of the buildgen.go output with emphasis on contrib/go/examples/3rdparty
and a note that the FLOATING lines are warn/yellow:
```console
$ ./pants buildgen.go --remote --materialize ::
...
9:32:29 00:00 [main]
(To run a reporting server: ./pants server)
19:32:29 00:00 [bootstrap]
19:32:29 00:00 [setup]
19:32:29 00:00 [parse]
Executing tasks in goals: buildgen
19:32:30 00:01 [buildgen]
19:32:30 00:01 [go]
Auto generated the following Go BUILD files: BUILD file (import paths)
contrib/go/examples/src/go/libA/BUILD (libA)
contrib/go/examples/src/go/libC/BUILD (libC)
contrib/go/examples/src/go/libB/BUILD (libB)
contrib/go/examples/src/go/libE/BUILD (libE)
contrib/go/examples/src/go/libD/BUILD (libD)
contrib/go/examples/3rdparty/go/github.com/gorilla/context/BUILD (github.com/gorilla/context) FLOATING
contrib/go/examples/3rdparty/go/github.com/AdRoll/goamz/BUILD (github.com/AdRoll/goamz/dynamodb) 51b70b0035cf28c4ca6d719156347ac56a41d922
contrib/go/examples/src/go/hello/BUILD (hello)
contrib/go/examples/src/go/server/BUILD (server)
contrib/go/examples/3rdparty/go/github.com/bitly/go-simplejson/BUILD (github.com/bitly/go-simplejson) v0.4.3
contrib/go/examples/3rdparty/go/github.com/gorilla/mux/BUILD (github.com/gorilla/mux) FLOATING
contrib/go/examples/3rdparty/go/github.com/cbroglie/mapstructure/BUILD (github.com/cbroglie/mapstructure) 25325b46b67d1c3eb7d58bad37d34d89a31cf9ec
19:32:30 00:01 [complete]
SUCCESS
```
CI went green here:
https://travis-ci.org/pantsbuild/pants/builds/76587511
Bugs closed: 2016, 2039
Reviewed at https://rbcommons.com/s/twitter/r/2676/
Point suppliers of -source and -target directly to the jvm to use [jvm-platform]. Testing Done: CI is away here: https://travis-ci.org/pantsbuild/pants/builds/76690925 Bugs closed: 2045 Reviewed at https://rbcommons.com/s/twitter/r/2682/
The GoCompile already does the hard work of creating the go executables, they're just tucked away under `.pants.d/`. The GoBinaryCreate task just copies them over to `dist/` for easy discovery and a stable path to rely on for higher level build automation. Testing Done: CI went green here: https://travis-ci.org/pantsbuild/pants/builds/76689600 Bugs closed: 2044 Reviewed at https://rbcommons.com/s/twitter/r/2681/
Check for unsatisfied filters in interpreters cache. It fixes an issue when you run some python targets and all interpreters will be cached. Then once you'll try to run a python target with a new interpreter described in compatibility field it won't be resolved. Separate python resolver cache folders for different python interpreters. Testing Done: https://travis-ci.org/fkorotkov/pants/builds Reviewed at https://rbcommons.com/s/twitter/r/2679/
- Add pants.pantsd.watchman.Watchman - Add pants.pantsd.watchman_client.StreamableWatchmanClient, an optimized watchman client used by Watchman - Minor supporting changes to ProcessManager - Tests Testing Done: https://travis-ci.org/pantsbuild/pants/builds/76039792 + manual E2E testing in my pantsd integration branch. Bugs closed: 2010 Reviewed at https://rbcommons.com/s/twitter/r/2649/
Testing Done: CI went green here: https://travis-ci.org/pantsbuild/pants/builds/76698007 Bugs closed: 2046 Reviewed at https://rbcommons.com/s/twitter/r/2683/
Testing Done: n/a Reviewed at https://rbcommons.com/s/twitter/r/2684/
Port checkstyle code from twitter commons to pants. Currently all plugins are disabled by default, because CI fails. I will toggle this before we merge, and update ci.sh to disable the checks. Testing Done: https://travis-ci.org/pantsbuild/pants/builds/75014548 https://travis-ci.org/pantsbuild/pants/builds/76035527 https://travis-ci.org/pantsbuild/pants/builds/76168444 Bugs closed: pantsbuild#2033 Reviewed at https://rbcommons.com/s/twitter/r/2593/
This introduces a new Addressable/Target parsing system based on named Serializable objects and Addressed pointers. A new Graph implementation is built on these and one example graph is included in three serialized forms. A planner can now be built that consumes example graphs and outputs execution plan graphs. Testing Done: CI went green here: https://travis-ci.org/pantsbuild/pants/builds/83049418 Bugs closed: 2283 Reviewed at https://rbcommons.com/s/twitter/r/2914/
The lifecycle is enforced by `Graph` and includes new support for: 1. Resolved object replacement: This is used to implement 2 forms of value inheritance, aka. templating. 2. Resolved object validation: This gives a first class home for validation logic and ensures the full hydrated object closure is available to run that logic against. Additionally, `Graph` gains support for cycle detection at resolve time and it now plumbs resolved objects addresses into their `address` field. Testing Done: CI went green here: https://travis-ci.org/pantsbuild/pants/builds/83265481 Bugs closed: 2302 Reviewed at https://rbcommons.com/s/twitter/r/2920/
This moves the types built up from it for use in planner sample inputs off to their own targets module. Testing Done: CI went green here: https://travis-ci.org/pantsbuild/pants/builds/83342667 Bugs closed: 2307, 2308 Reviewed at https://rbcommons.com/s/twitter/r/2924/
Testing Done: CI running at https://travis-ci.org/pantsbuild/pants/builds/83373509 Reviewed at https://rbcommons.com/s/twitter/r/2925/
Testing Done: Changelog staged at http://pantsbuild.github.io/staging/zundel/changelog.html CI run at https://travis-ci.org/pantsbuild/pants/builds/83379614 Bugs closed: 2306 Reviewed at https://rbcommons.com/s/twitter/r/2926/
…st_url. Testing Done: https://travis-ci.org/pantsbuild/pants/builds/83174746 Bugs closed: 2295 Reviewed at https://rbcommons.com/s/twitter/r/2915/
I'm adding a pretty complete and complex set of modules representing different things: - A server project transpiled using Babel in its prepublish step - A web build tool encapsulating Webpack and a set of loaders - A web component (button) using React and CSS in a separate file, built with the web build tool, containing its own test file - A web project using React, built with the web build tool, depending on the server project and button component, and containing its own test file This works now that pantsbuild#2267 is merged to master. This also modifies npm_resolve.py to work with an existing package.json rather than simply write out a simple package.json with only name/version/dependencies fields. Currently this works by dropping dependency fields in the existing package.json and replacing them with whatever is in the BUILD target dependencies. This lets it work with an existing valid package.json that can be installed via "npm install" without Pants, but it isn't the best long-term experience as I describe in the README. Support should improve over time to let Pants simply get its dependencies from a package.json. **Note - using an existing package.json means that any "prepublish" script in the scripts field of package.json will be executed as part of the "npm install" done for target resolution (npm runs prepublish if you do an "npm install" inside the folder - https://docs.npmjs.com/misc/scripts). This is necessary if you have a source dependency that needs transpilation, but it is an arbitrary script that gets run inside the resolve.** Testing Done: - Unit test in test_npm_resolve.py tests the integration with source package.json - Integration tests in test_npm_resolve_integration.py install two of the examples: server-project and web-project. web-project has (transitive) dependencies on all the examples and 3rd party modules. server-project is nice on its own as it only depends on 3rd party modules and it includes a prepublish transpilation step. Passing CI run: https://travis-ci.org/pantsbuild/pants/builds/83805985 Bugs closed: 2168 Reviewed at https://rbcommons.com/s/twitter/r/2900/
Add options to pinger and set defaults in cache_setup. Add a unittest with mock cache servers. Plumb changes through cache_setup. Fix a broken test in test_cache_setup. Testing Done: CI passes: https://travis-ci.org/pantsbuild/pants/builds/83812152 Bugs closed: 1766, 2293 Reviewed at https://rbcommons.com/s/twitter/r/2912/
This restricts responsibility of Graph to just linking addressables. Currently Graph links via inlining fully hydrated objects, but it or an alternative could also traverse to addressables on demand. With AddressMapper extracted, an interface for BUILD file invalidation comes to the fore and allows straight-forward integration of in invalidation system that watches the build root for BUILD file additions, removals and modifications. Testing Done: CI went green here: https://travis-ci.org/pantsbuild/pants/builds/83782626 Bugs closed: 2308, 2319 Reviewed at https://rbcommons.com/s/twitter/r/2931/
I introduced a bug in https://rbcommons.com/s/twitter/r/2916/ wherein all test-cases in a class were printed if any of the test-cases in the class had failed. This occurred due to my incorrect assumption that only testcases that failed were listed in the .xml report from the junit runner. Now junit_run.py only prints a summary of those test-cases that actually failed. This fixes pantsbuild#2310. Testing Done: Manual testing + new integration test. CI went green here: https://travis-ci.org/pantsbuild/pants/builds/83782147 CI went green here: https://travis-ci.org/pantsbuild/pants/builds/83907491 Bugs closed: 2310, 2318 Reviewed at https://rbcommons.com/s/twitter/r/2930/
Previously ivy would fail on jar dependencies with a url set for the idea and export goals when javadocs and sources were set to resolve, because ivy would be unable to locate sources and javadocs for the artifact. This change disables javadocs and sources for dependencies that have an artifact that specifies a url. IdeGen and Export now resolve their own jars intead of depending on the IvyResolve task. Testing Done: Added tests to idea_integration and export_integration. CI went green here: https://travis-ci.org/gmalmquist/pants/builds/82825576 CI went green here: https://travis-ci.org/pantsbuild/pants/builds/82830646 CI went green here: https://travis-ci.org/gmalmquist/pants/builds/82951337 CI went green here: https://travis-ci.org/pantsbuild/pants/builds/83786868 CI went green here: https://travis-ci.org/gmalmquist/pants/builds/83920249 CI went green here: https://travis-ci.org/gmalmquist/pants/builds/83937520 Bugs closed: 2285 Reviewed at https://rbcommons.com/s/twitter/r/2905/
Previously, commands that did not require a GOPATH allowed any GOPATH set in the user's environment to leak through. This could lead to spurious failures when the GOPATH was invalid. Fix GoDistribution to instead forcibly nullify GOPATH when the go command does not set one. Testing Done: Got `test_go_command_no_gopath_overrides_user_gopath_issue2321` failing before the fix, then passing. CI went green here: https://travis-ci.org/pantsbuild/pants/builds/83940764 Bugs closed: 2321, 2323 Reviewed at https://rbcommons.com/s/twitter/r/2933/
tests/python/pants_test:all wasn't running all the tests
Before this change:
% ./pants test tests/python/pants_test:all | grep backend/core/tasks
INFO] Detected git repository at /Users/sbrenn/workspace/pants on branch sbrenn/test-superclass
%
Testing Done:
% ./pants test tests/python/pants_test:all | grep backend/core/tasks
INFO] Detected git repository at /Users/sbrenn/workspace/pants on branch sbrenn/test-all
tests/python/pants_test/backend/core/tasks/test_bash_completion.py .
tests/python/pants_test/backend/core/tasks/test_dependees.py ..............
tests/python/pants_test/backend/core/tasks/test_filter.py ...............
tests/python/pants_test/backend/core/tasks/test_markdown_to_html.py ..........
tests/python/pants_test/backend/core/tasks/test_mutex_task_mixin.py ....
tests/python/pants_test/backend/core/tasks/test_scm_publish.py ........
tests/python/pants_test/backend/core/tasks/test_target_filter_task_mixin.py ...
tests/python/pants_test/backend/core/tasks/test_test_task_mixin.py ..
Pull Request: pantsbuild#2322
Reviewed at https://rbcommons.com/s/twitter/r/2932/
In order to minimize duplicated code between test tasks for different languages, here is a TestTaskMixin that will keep functionality shared among tests for all languages. - Filters target list according to `_get_targets` method in language-specific class - Validates targets according to `_validate_targets` method in language-specific class - --skip option will skip tests Testing Done: Added test: ./pants test tests/python/pants_test/backend/core/task:test_task_mixin All tests pass: ./pants test tests/python/pants_test:all Pull request: pantsbuild#2288 Reviewed at https://rbcommons.com/s/twitter/r/2902/
This patch removes the `classes_by_target` and `resources_by_target` products. I also needed to split `compile_classpath` (pre-compile) from `runtime_classpath` (post-compile) in order to allow the ResourceTask subclasses to expose a unique product that wouldn't cycle between themselves or compilation. - Add `runtime_classpath` which clones the `compile_classpath` and adds compiled classes and resources - Expand ClasspathUtil to allow walking classpath contents (optionally transitively) - Remove `classes_by_target` and `resources_by_target` from all tasks, in favor of adding classpath entries - Skip `jvm_dependency_check` by default (since the previous default checked for an impossible condition in the presence of isolation) - Fix a bug in `jvm_dependency_usage` tests by setting target roots in the context - Delete ResourceMapping, since it was only used to register resources which were already being written into a target's classpath - Replace "fake" classes_by_target deps with real runtime_classpath deps in cases where targets need a runtime classpath - Adapt ResourceTask to `cache_target_dirs` and drop unnecessary relative_resource_paths - `dx_compile` now passes classpath entries directly to the dex tool (since it supports consuming jars/zips/directories directly) Testing Done: pantsbuild#2311 Bugs closed: 2311 Reviewed at https://rbcommons.com/s/twitter/r/2928/
Without proper normalization, errors were seen in OSX tests where the build_root lay in the per-user tmp dir. Testing Done: An example failure is over here: https://travis-ci.org/pantsbuild-osx/pants/jobs/83902946 Looks like: ``` AddressMapperTest.test_invalidate_build_file_removed self = <pants_test.engine.exp.test_mapper.AddressMapperTest testMethod=test_invalidate_build_file_removed> def test_invalidate_build_file_removed(self): resolved = self.address_mapper.resolve(Address.parse('a/b')) a_b_target = Target(name='b', dependencies=['//d:e'], configurations=['//a', Configuration(embedded='yes')]) self.assertEqual(a_b_target, resolved) build_file = os.path.join(self.build_root, 'a/b/b.BUILD.json') os.unlink(build_file) self.assertIs(resolved, self.address_mapper.resolve(Address.parse('a/b'))) self.address_mapper.invalidate_build_file(build_file) with self.assertRaises(ResolveError): > self.address_mapper.resolve(Address.parse('a/b')) E AssertionError: ResolveError not raised ``` CI went green here: https://travis-ci.org/pantsbuild/pants/builds/83971616 Bugs closed: 2326 Reviewed at https://rbcommons.com/s/twitter/r/2935/
The changelog is here: https://golang.org/doc/devel/release.html#go1.5.minor Testing Done: Ran green locally: `./pants clean-all test contrib/go/::` CI went green here: https://travis-ci.org/pantsbuild/pants/builds/83978830 Bugs closed: 2332 Reviewed at https://rbcommons.com/s/twitter/r/2936/
The existing approach to supplying cobertura reporting with access to coverage target source is somewhat complicated, largely with the intention of fooling cobertura into finding scala classes. Cobertura is not really designed to cover scala in the first place, so this seems fairly pointless. Removing that in favor of a much simpler approach that just passes the traget_base of relevant targets. Testing Done: Manually tested on a couple of targets, and added validation to an existing integration test that looks for the annotated source file html and verifies it contains some src. Bugs closed: 2300 Reviewed at https://rbcommons.com/s/twitter/r/2918/
Previously the android local SDK resolvers used <ibiblio/> with local file:// roots. This led to issues resolving any <dependency/> with a custom artifact file:// url itself. Simplify ResolveJarsTestMixin which no longer needs a custom ivysettings.xml as a result of this fix. Testing Done: This ran green locally (I have an android setup): ``` $ ./pants clean-all test \ tests/python/pants_test/android/:: \ tests/python/pants_test/backend/project_info/:: ``` CI went green here: https://travis-ci.org/pantsbuild/pants/builds/83968079 Bugs closed: 2324 Reviewed at https://rbcommons.com/s/twitter/r/2934/
This has been the stable release for almost a year and pantsbuild/pants itself needs this version to properly handle the ANDROID_HOME env var detection it uses in ivysetting.xml. Previously, the bootstrap was 2.3.0 and the final used by pantsbuild/pants was 2.4.0 + extras. For any workspace with a cached final, the ivysettings.xml would work, but for workspaces w/o an ivy bootstrapped, the ivysettings.xml would fail. This change gets pantsbuild/pants ivysettings.xml working no matter the bootstrap phase. Testing Done: First noted the fact that the linux CI was green: https://travis-ci.org/pantsbuild/pants/builds/84007338 But the OSX CI was very red: https://travis-ci.org/pantsbuild-osx/pants/jobs/84014487 The 2 biggest differences being linux vs OS X and travis-ci cache vs none respectively. On my local machine, before the fix, I was able to reproduce the OS X CI failure mode by deleting my ivy bootstrap cache: `rm -rf ~/.cache/pants/tools/jvm/ivy/` After the fix I was no longer able to reproduce, all ran smooth through bootstrapping. CI went green here: https://travis-ci.org/pantsbuild/pants/builds/84017782 Bugs closed: 2333 Reviewed at https://rbcommons.com/s/twitter/r/2938/
cleanup final touches cleanup
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.