Skip to content

Commit

Permalink
update readme
Browse files Browse the repository at this point in the history
  • Loading branch information
CarlJi committed Oct 7, 2024
1 parent 80ea4e4 commit 7c83d3b
Show file tree
Hide file tree
Showing 2 changed files with 288 additions and 37 deletions.
176 changes: 148 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,52 +1,172 @@
# reviewbot - Comprehensive linters runner for code review scenarios
# reviewbot - Build your self-hosted automated code analysis and review server easily

[![Build Status](https://github.com/qiniu/reviewbot/actions/workflows/go.yml/badge.svg)](https://github.com/qiniu/reviewbot/actions/workflows/go.yml)
[![Go Report Card](https://goreportcard.com/badge/github.com/qiniu/reviewbot)](https://goreportcard.com/report/github.com/qiniu/reviewbot)
[![GitHub release](https://img.shields.io/github/v/tag/qiniu/reviewbot.svg?label=release)](https://github.com/qiniu/reviewbot/releases)

Its ultimate goal is to establish software engineering best practices and efficiently promote them within the organization.
Reviewbot assists you in rapidly establishing a self-hosted code analysis and review service, supporting multiple languages and coding standards. It is particularly suitable for organizations with numerous private repositories.

## Why Reviewbot?
All issues are reported during the Pull Request stage, either as `Review Comments` or `Github Annotations`, precisely pinpointing the relevant code lines.

Ensuring code quality for a limited number of repositories and languages is not difficult. We just need to use various inspection tools and configure them for the relevant repositories. But what if we are dealing with an entire organization, with different languages, a variety of new and old repositories (300+), and many historical legacy issues? How can we handle that?
- Github Check Run (Annotations)

We thought it would be best to have a centralized static analysis service that can be applied to all repositories with minimal configuration and enable efficient implementation of every new engineering practice within the organization.
<div style="display: flex; justify-content: flex-start; gap: 10px;">
<img src="./docs/static/github-check-run.png" alt="Github Check Run" width="467"/>
<img src="./docs/static/github-check-run-annotations.png" alt="Github Check Run Annotations" width="467"/>
</div>

**Reviewbot** was created for such a scenario.
- Github Pull Request Review Comments
<div style="display: flex; justify-content: flex-start;">
<img src="./docs/static/github-pr-review-comments.png" alt="Github Pull Request Review Comments" width="467"/>
</div>

It has been inspired by many tools in the industry, but with some differences:
This approach helps PR authors avoid searching for issues in lengthy console logs, significantly facilitating problem resolution.

- Similar to [golangci-lint](https://github.com/golangci/golangci-lint), **Reviewbot** is an aggregator of linters, but it includes more languages and process specifications (go/java/shell/git-flow/doc-style, etc.), and even custom specifications.
- Inspired from [reviewdog](https://github.com/reviewdog/reviewdog), **Reviewbot** primarily provides feedback in the form of review comments, pinpointing the issues down to specific lines of code. It can serve as a quality gate and continuously help the organization improve code quality, offering an elegant solution.
- Recommend deploying **Reviewbot** as a GitHub App or a Webhook Server for private execution, ensuring compatibility with private code repositories.
## Table of Contents

If you are facing similar challenges, we welcome you to try **Reviewbot**!
- [Why Reviewbot](#why-reviewbot)
- [Installation](#installation)
- [Supported Linters](#supported-linters)
- [Go](#go)
- [C/C++](#cc)
- [Lua](#lua)
- [Java](#java)
- [Git Workflow Standards](#git-workflow-standards)
- [Documentation Standards](#documentation-standards)
- [Configuration](#configuration)
- [Adjusting Execution Commands](#adjusting-execution-commands)
- [Disabling a Linter](#disabling-a-linter)
- [Executing Linters via Docker](#executing-linters-via-docker)
- [Reviewbot Operational Flow](#reviewbot-operational-flow)
- [Monitoring Detection Results](#monitoring-detection-results)
- [Contributing](#contributing)
- [License](#license)

## Linters Support
## Why Reviewbot

- go linters
- [golangci-lint](/internal/linters/go/golangci_lint/)
- [gofmt](/internal/linters/go/gofmt/)
- [gomodcheck](/internal/linters/go/gomodcheck/)
- c/c++ linters
- [cppcheck](/internal/linters/c/cppcheck/)
- lua linters
- [luacheck](/internal/linters/lua/luacheck/)
- git collaboration guidelines
- [commit msg check](/internal/linters/git-flow/commit-check/)
- doc guidelines
- [note check](/internal/linters/doc/note-check/)
Reviewbot is a self-hosted code analysis and review service supporting multiple languages and coding standards. It is particularly beneficial for organizations with numerous private repositories:

## Quickstart
- Security - Recommended self-hosting for data security and control
- Flexibility - Supports multiple languages and coding standards, with easy integration of new code inspection tools
- Observability - Supports alert notifications for timely awareness of detected issues
- Usability - Designed for zero-configuration, enabling inspection of all repositories with minimal setup

Please take a look at our [getting started guide](https://reviewbot-x.netlify.app).
Reviewbot is developed using Golang, featuring simple logic and clear code, making it easy to understand and maintain.

## Installation

Please refer to the [getting started guide](https://reviewbot-x.netlify.app/getting-started/installation).

## Supported Linters

### Go

- [golangci-lint](/internal/linters/go/golangci_lint/)
- [gofmt](/internal/linters/go/gofmt/)
- [gomodcheck](/internal/linters/go/gomodcheck/)

### C/C++

- [cppcheck](/internal/linters/c/cppcheck/)

### Lua

- [luacheck](/internal/linters/lua/luacheck/)

### Java

- [pmdcheck](/internal/linters/java/pmdcheck/)
- [stylecheck](/internal/linters/java/stylecheck/)

### Git Workflow Standards

- [commit msg check](/internal/linters/git-flow/commit-check/)

### Documentation Standards

- [note check](/internal/linters/doc/note-check/)

## Configuration

Reviewbot adheres to a **zero-configuration principle** whenever possible, with fixed code logic for general repository inspections. However, some special requirements can be met through configuration.

Note: All configurable items are defined in the `config/config.go` file. Please refer to this file for detailed configuration options.

The following are some common configuration scenarios:

### Adjusting Execution Commands

Linters are generally executed using default commands, but we can adjust these commands. For example:

This configuration means that for the `staticcheck` inspection of the `qbox/kodo` repository code, execution should occur in the `src/qiniu.com/kodo` directory.

We can even configure more complex commands, such as:

This configuration indicates that for the `golangci-lint` inspection of the `qbox/kodo` repository code, execution occurs through custom commands and arguments.

The usage of command and args here is similar to that of Kubernetes Pod command and args. You can refer to [Kubernetes Pod](https://kubernetes.io/docs/concepts/workloads/pods/) for more information.

The **$ARTIFACT** environment variable is noteworthy. This is a built-in variable in Reviewbot used to specify the output directory, facilitating the exclusion of irrelevant interference. Since Reviewbot ultimately only cares about the linters' output, and in this complex scenario, the shell script will output a lot of irrelevant information, we can use this environment variable to specify the output directory. This allows Reviewbot to parse only the files in this directory, resulting in more precise detection results.

### Disabling a Linter

We can also disable a specific linter check for a particular repository through configuration. For example:

This configuration means that the `golangci-lint` check is disabled for the `qbox/kodo` repository.

### Executing Linters via Docker

By default, Reviewbot uses locally installed linters for checks. However, in some scenarios, we might want to use Docker images to execute linters, such as:

- When the relevant linter is not installed locally
- When the target repository requires different versions of linters or dependencies
- When the target repository depends on many third-party libraries, which would be cumbersome to install locally

In these scenarios, we can configure Docker images to execute the linters. For example:

This configuration means that for the `golangci-lint` check of the `qbox/net-gslb` repository code, the `golangci/golangci-lint:v1.54.2` Docker image is used for execution.

## Reviewbot Operational Flow

Reviewbot primarily operates as a GitHub Webhook service, accepting GitHub Events, executing various checks, and providing precise feedback on the corresponding code if issues are detected.

```
Github Event -> Reviewbot -> Execute Linter -> Provide Feedback
```

### Basic Flow:

- Event received, determine if it's a Pull Request
- Retrieve code:
- Get the code affected by the PR
- Clone the main repository
- The main repository serves as a cache
- Checkout the PR and place it in a temporary directory
- Pull submodules
- If the repository uses submodule management, code is automatically pulled
- Enter Linter execution logic
- Filter linters
- By default, all supported linters apply to all repositories unless individually configured
- Individual configurations need to be explicitly specified in the configuration file
- Explicitly specified configurations override default configurations
- Execute linter
- General logic
- Execute the corresponding command and obtain the output results
- Filter the output results, only obtaining the parts relevant to this PR
- Some linters focus on code
- Some linters focus on other aspects
- Provide feedback
- Some linters provide Code Comments, precise to the code line
- Some linters provide issue comments

## Monitoring Detection Results

Reviewbot provides a monitoring dashboard for detection results, allowing you to view the detection results of all repositories and linters.

## Contributing

Your contributions to Reviewbot are essential for its long-term maintenance and improvement. Thanks for supporting Reviewbot!

### Reporting Issues

If you find a bug while working with the Reviewbot, please open an issue on GitHub and let us know what went wrong. We will try to fix it as quickly as we can.

## License
Expand Down
149 changes: 140 additions & 9 deletions README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

Reviewbot 帮助你快速搭建一个自托管的代码分析和代码审查服务,支持多种语言和多种代码规范,尤其适合有大量私有仓库的组织。

所有的问题都会在 Pull Request 阶段,以尽可能 `Review Comments``Github Annotations`形式来反馈,且精确到代码行。
所有的问题都会在 Pull Request 阶段, `Review Comments``Github Annotations`形式来反馈,且精确到代码行。

- Github Check Run (Annotations)

Expand All @@ -26,27 +26,38 @@ Reviewbot 帮助你快速搭建一个自托管的代码分析和代码审查服

- [为什么选择 Reviewbot](#why-reviewbot)
- [安装](#installation)
- [支持的代码检查工具](#supported-linters)
- [已支持的 linters](#已支持的-linters)
- [Go 语言](#go-语言)
- [C/C++](#cc)
- [Lua](#lua)
- [Java](#java)
- [Git 流程规范](#git-流程规范)
- [文档规范](#文档规范)
- [配置](#配置)
- [如何添加新的代码检查工具](#如何添加新的代码检查工具)
- [调整执行命令](#调整执行命令)
- [关闭 Linter](#关闭-Linter)
- [通过 Docker 执行 linter](#通过-Docker-执行-linter)
- [Reviewbot 运行流程](#reviewbot-运行流程)
- [观测检测结果](#观测检测结果)
- [贡献](#贡献)
- [许可证](#许可证)

## Why Reviewbot

## 安装
Reviewbot 是一个自托管的代码分析和代码审查服务,支持多种语言和多种代码规范,尤其适合有大量私有仓库的组织:

- 安全性 - 推荐自托管,数据安全可控
- 灵活性 - 支持多种语言和多种代码规范,也非常容易添加新的代码检查工具
- 可观测 - 支持 alert 通知,方便及时感知检测出的问题
- 易用性 - 面向零配置设计,不需要太多的配置,就能对所有仓库进行检测

Please take a look at our [getting started guide](https://reviewbot-x.netlify.app).
Reviewbot 基于 golang 开发,逻辑简单,代码清晰,容易理解和维护。

## 安装

## Supported Linters
请参考 [getting started guide](https://reviewbot-x.netlify.app/getting-started/installation).

### 支持的代码检查工具
### 已支持的 linters

#### Go 语言

Expand All @@ -65,7 +76,7 @@ Please take a look at our [getting started guide](https://reviewbot-x.netlify.ap
#### Java

- [pmdcheck](/internal/linters/java/pmdcheck/)
- [checkstyle](/internal/linters/java/checkstyle/)
- [stylecheck](/internal/linters/java/stylecheck/)

#### Git 流程规范

Expand All @@ -77,7 +88,127 @@ Please take a look at our [getting started guide](https://reviewbot-x.netlify.ap

## 配置

## 如何添加新的代码检查工具
`Reviewbot` 在配置方面尽可能追求 **无配置原则**,针对一般性的仓库检查都会固定到代码逻辑中。但也有一些特殊需求,可以通过配置完成.

注意: 所有的可配置项,都定义在 `config/config.go` 文件中,详细配置项可以参考这个文件。

以下是一些常见的配置场景:

### 调整执行命令

linters 一般都是用默认命令执行,但是我们也可以调整命令,比如

```yaml
qbox/kodo:
linters:
staticcheck:
workDir: "src/qiniu.com/kodo"
```
这个配置意味着,针对`qbox/kodo`仓库代码的`staticcheck`检查,要在`src/qiniu.com/kodo`目录下执行。

我们甚至可以配置更复杂的命令,比如:

```yaml
qbox/kodo:
linters:
golangci-lint:
command:
- "/bin/sh"
- "-c"
- "--"
args:
- |
source env.sh
cp .golangci.yml src/qiniu.com/kodo/.golangci.yml
cd src/qiniu.com/kodo
export GO111MODULE=auto
go mod tidy
golangci-lint run --timeout=10m0s --allow-parallel-runners=true --print-issued-lines=false --out-format=line-number >> $ARTIFACT/lint.log 2>&1
```

这个配置意味着,针对`qbox/kodo`仓库代码的`golangci-lint`检查,通过自定义命令 command 和参数 args 来执行。

这里的 command 和 args 使用姿势,与 Kubernetes Pod 的 command 和 args 类似,可以参考[Kubernetes Pod](https://kubernetes.io/docs/concepts/workloads/pods/)

**$ARTIFACT** 环境变量值得注意,这个环境变量是 `Reviewbot` 内置的,用于指定输出目录,方便排除无效干扰。因为 `Reviewbot` 最终只会关心 linters 的输出,而在这个复杂场景下,shell 脚本会输出很多无关信息,所以我们可以通过这个环境变量来指定输出目录,让 `Reviewbot` 只解析这个目录下的文件,这样得到的检测结果会更加精准。

### 关闭 Linter

我们也可以通过配置,关闭某个仓库的某个 linter 检查,比如:

```yaml
qbox/net-gslb:
linters:
golangci-lint:
enable: false
```

### 通过 Docker 执行 linter

默认情况下,`Reviewbot` 会使用本地安装的 linter 来执行检查,但有些场景下,我们可能希望使用 Docker 镜像来执行 linter,比如:

- 本地没有安装相关 linter
- 目标仓库需要不同版本的 linter 或依赖
- 目标仓库需要依赖很多第三方库,如果都安装到本地,会非常麻烦

在这些场景下,我们可以通过配置 Docker 镜像来完成 linter 的执行。比如:

```yaml
qbox/net-gslb:
linters:
golangci-lint:
dockerAsRunner:
image: "golangci/golangci-lint:v1.54.2"
```

这个配置意味着,针对`qbox/net-gslb`仓库代码的`golangci-lint`检查,使用`golangci/golangci-lint:v1.54.2`这个 Docker 镜像来执行。

## Reviewbot 运行流程

`Reviewbot` 目前主要作为 GitHub Webhook 服务运行,会接受 GitHub Events,然后执行各种检查,若检查出问题,会精确响应到对应代码上。

```
Github 事件 -> Reviewbot -> 执行 linter -> 反馈结果
```
### 基本流程如下:
- 事件进来,判断是否是 Pull Request
- 获取代码:
- 获取 PR 影响的代码
- clone 主仓
- 主仓会作为缓存
- checkout PR,并放置在临时目录
- pull 子模块
- 仓库若使用 submodule 管理则自动拉取代码
- 进入 Linter 执行逻辑
- 筛选 linter
- 默认只要支持的 linter 都对所有仓库适用,除非有单独配置
- 单独配置需要通过配置文件显式指定
- 显式指定的配置会覆盖默认的配置
- 执行 linter
- 通用逻辑
- 执行相应命令,拿到输出结果
- filter 输出的结果,只获取本次 PR 关心的部分
- 有的 linter 关心代码
- 有的 linter 关心其他
- 做反馈
- 有的 linter 给 Code Comment,精确到代码行
- 有的 linter 给 issue comment
## 如何添加新的 Linter?
- 请从 [issues](https://github.com/qiniu/reviewbot/issues) 列表中选择你要处理的 Issue.
- 当然,如果没有,你可以先提个 Issue,描述清楚你要添加的 Linter
- 编码
- 基于 linter 关注的语言或领域,[选好代码位置](https://github.com/qiniu/reviewbot/tree/master/internal/linters)
- 绝大部分的 linter 实现逻辑分三大块:
- 执行 linter,一般是调用相关的可执行程序
- 处理 linter 的输出,我们只会关注跟本次 PR 相关的输出
- 反馈 跟本次 PR 相关的输出,精确到代码行
- 部署,如果你的 linter 是外部可执行程序,那么就需要在 [Dockerfile](https://github.com/qiniu/reviewbot/blob/master/Dockerfile) 中添加如何安装这个 linter
- 文档,为方便后续的使用和运维,我们应当 [在这里](https://github.com/qiniu/reviewbot/tree/master/docs/website/docs/components) 添加合适的文档
## 贡献
Expand Down

0 comments on commit 7c83d3b

Please sign in to comment.