Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Empty code coverage report is generated when test files are referenced as external repository #16228

Closed
VinayDodamani opened this issue Sep 7, 2022 · 6 comments

Comments

@VinayDodamani
Copy link

Description of the bug:

an empty code coverage report is generated when the test files exist in an external respository that has its own WORKSPACE file. following is the structure of my code:

local_repository(
name = "sample",
path = "sample",
)

├── BUILD
├── MODULE.bazel
├── sample
│ |── BUILD
│ │ ├──sample_module
│ │ │ ├──test_sample_module.cpp
│ │ ├── BUILD
│ └── WORKSPACE
├── WORKSPACE

and i am using the following coverage relevant flags:
coverage --instrument_test_targets
coverage --instrumentation_filter="[:]"
coverage --combined_report=lcov
coverage --test_env=VERBOSE_COVERAGE=1
coverage --test_output=all

along with this i have also set the appropriate env variables GCOV_PATH and LLVM_COV

coverage:clang --test_env=COVERAGE_GCOV_PATH=external/toolchain.path/usr/local/bin/llvm-profdata
coverage:clang --test_env=LLVM_COV=external/toolchain.path/usr/local/bin/llvm-cov

when i run the following command from the root, i get an empty coverage report:

bazel coverage --config=clang @sample//...

but, I was able to generate a coverage data when I copied the .bazelrc and MODULE.bazel inside the sample repo from the root and ran the following command in the sample repo:

bazel coverage --config=clang //...

the problem that we have is that, we have multiple packages that are external repositories and therefore copying these files everytime into the respective external repositories is not feasible.

I also went through other issues that had similar porblems and most pointed towards the --instrumentation_filter as a potential solution but that also has not worked for me. I have tried instrumenting using the --instrumentation_filter but with no success.

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

steps mentioned in the description of the bug

Which operating system are you running Bazel on?

Ubuntu 18.04.6 LTS

What is the output of bazel info release?

release 5.1.1

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

Sep 06, 2022 3:04:26 PM com.google.devtools.coverageoutputgenerator.Main getTracefiles
INFO: No lcov file found.
Sep 06, 2022 3:04:26 PM com.google.devtools.coverageoutputgenerator.Main getGcovInfoFiles
INFO: No gcov info file found.
Sep 06, 2022 3:04:26 PM com.google.devtools.coverageoutputgenerator.Main getGcovJsonInfoFiles
INFO: No gcov json file found.
Sep 06, 2022 3:04:26 PM com.google.devtools.coverageoutputgenerator.Main getProfdataFileOrNull
INFO: No .profdata file found.
Sep 06, 2022 3:04:26 PM com.google.devtools.coverageoutputgenerator.Main runWithArgs
WARNING: There was no coverage found.

INFO: LCOV coverage report is located at /root/.cache/bazel/_bazel_root/57f55395d11193aa0c12c254de9cef22/execroot/main/bazel-out/_coverage/_coverage_report.dat
and execpath is bazel-out/_coverage/_coverage_report.dat
INFO: From Coverage report generation:
Sep 06, 2022 5:04:27 PM com.google.devtools.coverageoutputgenerator.Main getTracefiles
INFO: Found 1 tracefiles.
Sep 06, 2022 5:04:27 PM com.google.devtools.coverageoutputgenerator.Main parseFilesSequentially
INFO: Parsing file bazel-out/k8-fastbuild/testlogs/external/sample/sample_module/test_sample_module/coverage.dat
Sep 06, 2022 5:04:27 PM com.google.devtools.coverageoutputgenerator.Main getGcovInfoFiles
INFO: No gcov info file found.
Sep 06, 2022 5:04:27 PM com.google.devtools.coverageoutputgenerator.Main getGcovJsonInfoFiles
INFO: No gcov json file found.
Sep 06, 2022 5:04:27 PM com.google.devtools.coverageoutputgenerator.Main getProfdataFileOrNull
INFO: No .profdata file found.
Sep 06, 2022 5:04:27 PM com.google.devtools.coverageoutputgenerator.Main runWithArgs
WARNING: There was no coverage found.
Target @sample//sample_module:test_sample_module up-to-date:
bazel-bin/external/sample/sample_module/test_sample_module
INFO: Elapsed time: 19.998s, Critical Path: 10.16s
INFO: 40 processes: 11 internal, 27 processwrapper-sandbox, 2 worker.
INFO: Build completed successfully, 40 total actions
@sample//sample_module:test_sample_module PASSED in 0.6s

Executed 1 out of 1 test: 1 test passes.

@sgowroji
Copy link
Member

sgowroji commented Sep 7, 2022

Hello @VinayDodamani, Can you share a code repo to check the above request at our end. Thanks!

@celkas
Copy link

celkas commented Sep 8, 2022

@VinayDodamani seems ot be a duplicate of #16208 ?

@VinayDodamani
Copy link
Author

VinayDodamani commented Sep 9, 2022

@celkas it does seem to be a similar issue that i am experiencing

@sgowroji
Copy link
Member

Marking it as duplicate of #16208

copybara-service bot pushed a commit that referenced this issue Oct 26, 2022
Removes hardcoded filters that result in no coverage being reported for `cc_*` targets in external repositories even when matched by the `--instrumentation_filter`.

Work towards #16228
Work towards #16208

Closes #16261.

PiperOrigin-RevId: 483911568
Change-Id: Ibca6fb39a8e946e06beeb03414ad8ccbc42f53d6
fmeum added a commit to fmeum/bazel that referenced this issue Oct 26, 2022
Removes hardcoded filters that result in no coverage being reported for `cc_*` targets in external repositories even when matched by the `--instrumentation_filter`.

Work towards bazelbuild#16228
Work towards bazelbuild#16208

Closes bazelbuild#16261.

PiperOrigin-RevId: 483911568
Change-Id: Ibca6fb39a8e946e06beeb03414ad8ccbc42f53d6
ShreeM01 pushed a commit that referenced this issue Oct 26, 2022
Removes hardcoded filters that result in no coverage being reported for `cc_*` targets in external repositories even when matched by the `--instrumentation_filter`.

Work towards #16228
Work towards #16208

Closes #16261.

PiperOrigin-RevId: 483911568
Change-Id: Ibca6fb39a8e946e06beeb03414ad8ccbc42f53d6
copybara-service bot pushed a commit that referenced this issue Nov 2, 2022
* Removes a hardcoded filter that results in the LCOV merger filtering
  out coverage of external targets even if these are matched by
  `--instrumentation_filter`.
* Lets `LazyWritePathsFileAction` write out exec rather than
  root-relative paths, as advertised by its javadocs, so that the paths
  match those that the LCOV mergers filters by.

Work towards #16228
Work towards #16208

Closes #16268.

PiperOrigin-RevId: 485580267
Change-Id: I830d3cf903462ca1799ef5f659a620dbdb92f349
fmeum added a commit to fmeum/bazel that referenced this issue Nov 2, 2022
* Removes a hardcoded filter that results in the LCOV merger filtering
  out coverage of external targets even if these are matched by
  `--instrumentation_filter`.
* Lets `LazyWritePathsFileAction` write out exec rather than
  root-relative paths, as advertised by its javadocs, so that the paths
  match those that the LCOV mergers filters by.

Work towards bazelbuild#16228
Work towards bazelbuild#16208

Closes bazelbuild#16268.

PiperOrigin-RevId: 485580267
Change-Id: I830d3cf903462ca1799ef5f659a620dbdb92f349
ShreeM01 pushed a commit that referenced this issue Nov 2, 2022
* Removes a hardcoded filter that results in the LCOV merger filtering
  out coverage of external targets even if these are matched by
  `--instrumentation_filter`.
* Lets `LazyWritePathsFileAction` write out exec rather than
  root-relative paths, as advertised by its javadocs, so that the paths
  match those that the LCOV mergers filters by.

Work towards #16228
Work towards #16208

Closes #16268.

PiperOrigin-RevId: 485580267
Change-Id: I830d3cf903462ca1799ef5f659a620dbdb92f349
@susinmotion
Copy link
Contributor

We're rolling #16637 back since it's breaking some internal tests :( Reopening to keep track of it again.

@susinmotion susinmotion reopened this Nov 14, 2022
@oquenchil
Copy link
Contributor

Continue discussion in #16208 unless they are not really duplicates. If they aren't, please remove the duplicate tag.

@oquenchil oquenchil closed this as not planned Won't fix, can't repro, duplicate, stale Jan 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants