Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/open compass #1213

Merged
merged 29 commits into from
Jun 29, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
f27bf0e
wip
tastelikefeet Jun 21, 2024
03ec157
test code
tastelikefeet Jun 22, 2024
8fcb675
Merge commit '38e4d96bdab88f984b7f3bf8f94453f6ae63fac3' into feat/ope…
tastelikefeet Jun 22, 2024
575586f
wip
tastelikefeet Jun 24, 2024
e574fbc
lint
tastelikefeet Jun 24, 2024
52febaf
wip
tastelikefeet Jun 26, 2024
e881688
fix
tastelikefeet Jun 26, 2024
b00fe28
wip
tastelikefeet Jun 26, 2024
fea533b
fix
tastelikefeet Jun 26, 2024
ac5e835
fix
tastelikefeet Jun 26, 2024
b04b278
fix comments
tastelikefeet Jun 27, 2024
45b0a5d
fix
tastelikefeet Jun 27, 2024
d49183c
fix
tastelikefeet Jun 27, 2024
42e9397
Merge branch 'main' into feat/open-compass
tastelikefeet Jun 27, 2024
a62dd57
fix
tastelikefeet Jun 28, 2024
e96d58a
fix note
tastelikefeet Jun 28, 2024
62bceab
fix
tastelikefeet Jun 28, 2024
288b23e
Merge branch 'main' into feat/open-compass
tastelikefeet Jun 28, 2024
4e832b1
lint
tastelikefeet Jun 28, 2024
e682ab0
update docs
tastelikefeet Jun 28, 2024
09a42ab
add limit
tastelikefeet Jun 29, 2024
5e3ca9c
fix
tastelikefeet Jun 29, 2024
6d1cea8
Merge commit '78159fd8fb158dda9e78ec2bf2097e0447ff4da1' into feat/ope…
tastelikefeet Jun 29, 2024
da5e1df
reset snapshot_download to modelscope
tastelikefeet Jun 29, 2024
59a4e34
update doc
tastelikefeet Jun 29, 2024
d9db0c0
Merge commit 'cc2d8fbdc4ffeb9b15da8a5bb4b4543d9726399a' into feat/ope…
tastelikefeet Jun 29, 2024
a22d134
no message
tastelikefeet Jun 29, 2024
1fe67f9
fix
tastelikefeet Jun 29, 2024
440c7c6
add doc
tastelikefeet Jun 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix
  • Loading branch information
tastelikefeet committed Jun 27, 2024
commit d49183c7cf5fe1cd1f5c4cf49b9c6eaaff31a5b4
2 changes: 1 addition & 1 deletion docs/source/LLM/命令行参数.md
Original file line number Diff line number Diff line change
Expand Up @@ -329,7 +329,7 @@ eval参数继承了infer参数,除此之外增加了以下参数:(注意:
```
- `--eval_few_shot`: 每个评测集的子数据集的few-shot个数, 默认为`None`, 即使用数据集的默认配置. **本参数暂时废弃**
- `--eval_limit`: 每个评测集的子数据集的采样数量, 默认为`None`代表全量评测. **本参数暂时废弃**
- `--name`: 用于区分相同配置评估的结果存储路径
- `--name`: 用于区分相同配置评估的结果存储路径. 如: `{eval_output_dir}/{name}`, 默认在:`eval_outputs/defaults`, 其内部存在以时间命名的文件夹来承载每次评测结果.
- `--eval_url`: OpenAI标准的模型调用接口, 例如`http://127.0.0.1:8000/v1`. 如果使用部署的方式评估, 则需要进行设置, 通常不需要设置. 默认为`None`.
```shell
swift eval --eval_url http://127.0.0.1:8000/v1 --eval_is_chat_model true --model_type gpt4 --eval_token xxx
Expand Down
2 changes: 1 addition & 1 deletion docs/source_en/LLM/Command-line-parameters.md
Original file line number Diff line number Diff line change
Expand Up @@ -329,7 +329,7 @@ The eval parameters inherit from the infer parameters, and additionally include
```
- `--eval_few_shot`: The few-shot number of sub-datasets for each evaluation set, with a default value of `None`, meaning to use the default configuration of the dataset. **This parameter is currently deprecated.**
- `--eval_limit`: The sampling quantity for each sub-dataset of the evaluation set, with a default value of `None` indicating full-scale evaluation. **This parameter is currently deprecated.**
- `--name`: Used to differentiate the result storage path for evaluating the same configuration.
- `--name`: Used to differentiate the result storage path for evaluating the same configuration. Like: `{eval_output_dir}/{name}`, default will be `eval_outputs/defaults`, in which a timestamp named folder will hold each eval result.
- `--eval_url`: The standard model invocation interface for OpenAI, for example, `http://127.0.0.1:8000/v1`. This needs to be set when evaluating in a deployed manner, usually not needed. Default is `None`.
```shell
swift eval --eval_url http://127.0.0.1:8000/v1 --eval_is_chat_model true --model_type gpt4 --eval_token xxx
Expand Down