Make PyTorch code-base clang-tidy compliant #56892

malfet · 2021-04-25T18:37:24Z

This is an automatic change generated by the following script:

#!/usr/bin/env python3
from subprocess import check_output, check_call
import os

def get_compiled_files_list():
    import json
    with open("build/compile_commands.json") as f:
        data = json.load(f)
    files = [os.path.relpath(node['file']) for node in data]
    for idx, fname in enumerate(files):
        if fname.startswith('build/') and fname.endswith('.DEFAULT.cpp'):
            files[idx] = fname[len('build/'):-len('.DEFAULT.cpp')]
    return files


def run_clang_tidy(fname):
    check_call(["python3", "tools/clang_tidy.py", "-c", "build", "-x", fname,"-s"])
    changes = check_output(["git", "ls-files", "-m"])
    if len(changes) == 0:
        return
    check_call(["git", "commit","--all", "-m", f"NOLINT stubs for {fname}"])


def main():
    git_files = check_output(["git", "ls-files"]).decode("ascii").split("\n")
    compiled_files = get_compiled_files_list()
    for idx, fname in enumerate(git_files):
        if fname not in compiled_files:
            continue
        if fname.startswith("caffe2/contrib/aten/"):
            continue
        print(f"[{idx}/{len(git_files)}] Processing {fname}")
        run_clang_tidy(fname)

if __name__ == "__main__":
    main()

facebook-github-bot · 2021-04-25T18:37:30Z

💊 CI failures summary and remediations

As of commit bb6a28a (more details on the Dr. CI page):

1/1 failures possibly* introduced in this PR
- 1/1 non-scanned failure(s)

ci.pytorch.org: 1 failed

Failed: pr/pytorch-linux-bionic-rocm4.1-py3.6

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

facebook-github-bot · 2021-04-25T23:28:08Z

@malfet has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

codecov · 2021-04-26T06:18:09Z

Codecov Report

Merging #56892 (ede758a) into master (e97c17a) will increase coverage by 0.00%.
The diff coverage is n/a.

❗ Current head ede758a differs from pull request most recent head 31b3968. Consider uploading reports for the commit 31b3968 to get more accurate results

@@           Coverage Diff           @@
##           master   #56892   +/-   ##
=======================================
  Coverage   77.20%   77.21%           
=======================================
  Files        1948     1948           
  Lines      193573   193572    -1     
=======================================
+ Hits       149455   149467   +12     
+ Misses      44118    44105   -13

Summary: Attempts to call clang-tidy on any source file in `aten/src/ATen/cpu/native` would fail with series of ``` /Users/nshulga/git/pytorch-worktree/aten/src/ATen/native/cpu/Activation.cpp:637:1: warning: variable 'REGISTER_DISPATCH' is non-const and globally accessible, consider making it const [cppcoreguidelines-avoid-non-const-global-variables] /Users/nshulga/git/pytorch-worktree/aten/src/ATen/native/cpu/Activation.cpp:638:1: error: C++ requires a type specifier for all declarations [clang-diagnostic-error] REGISTER_DISPATCH(log_sigmoid_backward_cpu_stub, &log_sigmoid_backward_cpu_kernel); ``` because those macros are only defined for cpu-arch specific compilation of above mentioned files. Fix this by introducing `map_filename` function that will map source file to its copy in `build` folder, run clang-tidy over the copy and than map it back Find it while working on #56892 Pull Request resolved: #57037 Reviewed By: walterddr Differential Revision: D28033760 Pulled By: malfet fbshipit-source-id: b67cd007000574ecc165ab4b285c0c102cbceadd

facebook-github-bot · 2021-04-28T18:50:20Z

@malfet has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot · 2021-04-28T19:42:39Z

@malfet has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot · 2021-04-28T21:11:51Z

@malfet merged this pull request in 4cb534f.

Summary: Attempts to call clang-tidy on any source file in `aten/src/ATen/cpu/native` would fail with series of ``` /Users/nshulga/git/pytorch-worktree/aten/src/ATen/native/cpu/Activation.cpp:637:1: warning: variable 'REGISTER_DISPATCH' is non-const and globally accessible, consider making it const [cppcoreguidelines-avoid-non-const-global-variables] /Users/nshulga/git/pytorch-worktree/aten/src/ATen/native/cpu/Activation.cpp:638:1: error: C++ requires a type specifier for all declarations [clang-diagnostic-error] REGISTER_DISPATCH(log_sigmoid_backward_cpu_stub, &log_sigmoid_backward_cpu_kernel); ``` because those macros are only defined for cpu-arch specific compilation of above mentioned files. Fix this by introducing `map_filename` function that will map source file to its copy in `build` folder, run clang-tidy over the copy and than map it back Find it while working on pytorch#56892 Pull Request resolved: pytorch#57037 Reviewed By: walterddr Differential Revision: D28033760 Pulled By: malfet fbshipit-source-id: b67cd007000574ecc165ab4b285c0c102cbceadd

Summary: This is an automatic change generated by the following script: ``` #!/usr/bin/env python3 from subprocess import check_output, check_call import os def get_compiled_files_list(): import json with open("build/compile_commands.json") as f: data = json.load(f) files = [os.path.relpath(node['file']) for node in data] for idx, fname in enumerate(files): if fname.startswith('build/') and fname.endswith('.DEFAULT.cpp'): files[idx] = fname[len('build/'):-len('.DEFAULT.cpp')] return files def run_clang_tidy(fname): check_call(["python3", "tools/clang_tidy.py", "-c", "build", "-x", fname,"-s"]) changes = check_output(["git", "ls-files", "-m"]) if len(changes) == 0: return check_call(["git", "commit","--all", "-m", f"NOLINT stubs for {fname}"]) def main(): git_files = check_output(["git", "ls-files"]).decode("ascii").split("\n") compiled_files = get_compiled_files_list() for idx, fname in enumerate(git_files): if fname not in compiled_files: continue if fname.startswith("caffe2/contrib/aten/"): continue print(f"[{idx}/{len(git_files)}] Processing {fname}") run_clang_tidy(fname) if __name__ == "__main__": main() ``` Pull Request resolved: pytorch#56892 Reviewed By: H-Huang Differential Revision: D27991944 Pulled By: malfet fbshipit-source-id: 5415e1eb2c1b34319a4f03024bfaa087007d7179

malfet requested a review from a team April 25, 2021 18:37

facebook-github-bot added the cla signed label Apr 25, 2021

walterddr approved these changes Apr 25, 2021

View reviewed changes

seemethere approved these changes Apr 27, 2021

View reviewed changes

malfet mentioned this pull request Apr 27, 2021

Fix clang-tidy for native CPU ops #57037

Closed

malfet force-pushed the malfet/make-all-cpp-files-clang-tidy-compliant branch from 31b3968 to c970a5f Compare April 27, 2021 19:02

malfet requested review from albanD, glaringlee and soulitzer as code owners April 27, 2021 19:02

malfet force-pushed the malfet/make-all-cpp-files-clang-tidy-compliant branch from 604cb3c to 0448cd7 Compare April 28, 2021 16:22

malfet requested review from BowenBao, neginraoof and spandantiwari as code owners April 28, 2021 16:22

malfet added 13 commits April 28, 2021 11:08

NOLINT stubs for test/cpp/jit/test_argument_spec.cpp

a2681e5

NOLINT stubs for test/cpp/jit/test_autodiff.cpp

4cd4d2b

NOLINT stubs for test/cpp/jit/test_backend.cpp

a75d12a

NOLINT stubs for test/cpp/jit/test_backend_compiler_lib.cpp

7f796e5

NOLINT stubs for test/cpp/jit/test_backend_compiler_preprocess.cpp

46865fe

NOLINT stubs for test/cpp/jit/test_backend_lib.cpp

bc78645

NOLINT stubs for test/cpp/jit/test_class_import.cpp

5bc7099

NOLINT stubs for test/cpp/jit/test_class_parser.cpp

ce91a80

NOLINT stubs for test/cpp/jit/test_class_type.cpp

9604a7a

NOLINT stubs for test/cpp/jit/test_cleanup_passes.cpp

cca29d1

NOLINT stubs for test/cpp/jit/test_code_template.cpp

a2514b7

NOLINT stubs for test/cpp/jit/test_constant_pooling.cpp

4ac7387

NOLINT stubs for test/cpp/jit/test_create_autodiff_subgraphs.cpp

85e37fb

malfet added 4 commits April 28, 2021 11:08

NOLINT stubs for torch/lib/libshm/manager.cpp

95a67be

NOLINT stubs for aten/src/ATen/native/layer_norm.cpp

4cc4821

NOLINT stubs for c10/test/util/C++17_test.cpp

7df0fd7

NOLINT stubs for torch/csrc/autograd/python_variable.cpp

1595f90

malfet force-pushed the malfet/make-all-cpp-files-clang-tidy-compliant branch from 0448cd7 to 1595f90 Compare April 28, 2021 18:11

malfet added 5 commits April 28, 2021 11:40

Fix clang-format for torch/csrc/jit/runtime/static/impl.cpp

1b5c4dd

Fix clang-format for torch/csrc/jit/python/script_init.cpp

c90e707

Fix clang-format for torch/csrc/jit/passes/shape_analysis.cpp

20512e3

Fix clang-format for torch/csrc/jit/mobile/import_data.cpp

0777fc1

Fix clang-format for test/cpp/tensorexpr/tutorial.cpp

ca10183

malfet added 9 commits April 28, 2021 11:53

Fix clang-format for test/cpp/jit/test_lite_trainer.cpp

cca301e

Fix clang-format for test/cpp/tensorexpr/test_boundsinference.cpp

60a493b

Fix clang-format for test/cpp/tensorexpr/test_cpp_codegen.cpp

33bf1fc

Fix clang-format for test/cpp/tensorexpr/test_external_calls.cpp

9a1322e

Fix clang-format for test/cpp/tensorexpr/test_kernel.cpp

89ed82c

Fix clang-format for test/cpp/tensorexpr/test_loopnest.cpp

2f8c3c9

Fix clang-tidy for test/cpp/tensorexpr/test_memdependency.cpp

1caba84

Fix clang-format for tensorexpr/test_[registerizer|simplify].cpp

51a93fe

Fix clang-format for test/cpp/tensorexpr/tutorial.cpp

bb6a28a

facebook-github-bot closed this in 4cb534f Apr 28, 2021

facebook-github-bot added the Merged label Apr 28, 2021

malfet deleted the malfet/make-all-cpp-files-clang-tidy-compliant branch April 28, 2021 21:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make PyTorch code-base clang-tidy compliant #56892

Make PyTorch code-base clang-tidy compliant #56892

malfet commented Apr 25, 2021 •

edited

Loading

facebook-github-bot commented Apr 25, 2021 •

edited

Loading

facebook-github-bot commented Apr 25, 2021

codecov bot commented Apr 26, 2021 •

edited

Loading

facebook-github-bot commented Apr 28, 2021

facebook-github-bot commented Apr 28, 2021

facebook-github-bot commented Apr 28, 2021

Make PyTorch code-base clang-tidy compliant #56892

Make PyTorch code-base clang-tidy compliant #56892

Conversation

malfet commented Apr 25, 2021 • edited Loading

facebook-github-bot commented Apr 25, 2021 • edited Loading

💊 CI failures summary and remediations

ci.pytorch.org: 1 failed

facebook-github-bot commented Apr 25, 2021

codecov bot commented Apr 26, 2021 • edited Loading

Codecov Report

facebook-github-bot commented Apr 28, 2021

facebook-github-bot commented Apr 28, 2021

facebook-github-bot commented Apr 28, 2021

malfet commented Apr 25, 2021 •

edited

Loading

facebook-github-bot commented Apr 25, 2021 •

edited

Loading

codecov bot commented Apr 26, 2021 •

edited

Loading