Skip to content

Commit

Permalink
Update Model_Test_Report.md
Browse files Browse the repository at this point in the history
  • Loading branch information
@yuanxiaosc
yuanxiaosc authored Feb 25, 2019
1 parent fe21997 commit f5d962c
Showing 1 changed file with 142 additions and 99 deletions.
241 changes: 142 additions & 99 deletions calculating_model_score/Model_Test_Report.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,38 +3,83 @@
[Task-Oriented-Dialogue-Dataset-Survey](https://github.com/AtmaHou/Task-Oriented-Dialogue-Dataset-Survey)

### Atis Intent Prediction

```
准确率: 0.9842857142857143
宏平均精确率: 0.9858767424798239
微平均精确率: 0.9842857142857143
加权平均精确率: 0.9853440415050833
宏平均召回率: 0.9849107098074714
微平均召回率: 0.9842857142857143
加权平均召回率: 0.9842857142857143
宏平均F1-score: 0.9849281321280821
微平均F1-score: 0.9842857142857143
加权平均F1-score: 0.9843205635966433
准确率: 0.9764837625979843
宏平均精确率: 0.7449854331023172
微平均精确率: 0.9764837625979843
加权平均精确率: 0.9750854106720163
宏平均召回率: 0.7499221418525216
微平均召回率: 0.9764837625979843
加权平均召回率: 0.9764837625979843
宏平均F1-score: 0.5650970439524852
微平均F1-score: 0.9764837625979843
加权平均F1-score: 0.9741996309183255
混淆矩阵输出:
[[124 0 0 0 0 0 0]
[ 0 92 0 0 0 0 0]
[ 0 2 102 0 0 0 0]
[ 0 0 0 85 0 1 0]
[ 0 0 0 0 80 0 0]
[ 0 0 0 0 0 107 0]
[ 0 0 0 0 0 8 99]]
[[ 33 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0]
[ 0 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 1]
[ 0 0 48 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0]
[ 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0]
[ 0 0 0 0 38 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0]
[ 0 0 0 0 0 18 0 0 0 0 0 0 0 0 0 0 0 0
0 0]
[ 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 0 0 0
0 1]
[ 2 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 0
0 0]
[ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0
0 0]
[ 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0
0 0]
[ 0 0 0 0 0 1 0 0 0 0 626 1 0 0 0 0 0 0
0 4]
[ 0 0 2 0 0 0 0 0 0 0 4 6 0 0 0 0 0 0
0 0]
[ 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0
0 0]
[ 0 0 0 0 0 0 0 0 0 0 0 0 0 8 0 0 0 0
0 0]
[ 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
0 0]
[ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
0 0]
[ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 0
0 0]
[ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 36
0 0]
[ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
6 0]
[ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 3]]
分类报告:
precision recall f1-score support
precision recall f1-score support
AddToPlaylist 1.00 1.00 1.00 124
BookRestaurant 0.98 1.00 0.99 92
GetWeather 1.00 0.98 0.99 104
PlayMusic 1.00 0.99 0.99 86
RateBook 1.00 1.00 1.00 80
SearchCreativeWork 0.92 1.00 0.96 107
SearchScreeningEvent 1.00 0.93 0.96 107
atis_abbreviation 0.94 1.00 0.97 33
atis_aircraft 1.00 0.89 0.94 9
atis_airfare 0.94 1.00 0.97 48
atis_airfare#atis_flight 0.00 0.00 0.00 1
atis_airline 1.00 1.00 1.00 38
atis_airport 0.95 1.00 0.97 18
atis_capacity 1.00 0.95 0.98 21
atis_city 1.00 0.67 0.80 6
atis_day_name 0.00 0.00 0.00 2
atis_distance 1.00 1.00 1.00 10
atis_flight 0.99 0.99 0.99 632
atis_flight#atis_airfare 0.86 0.50 0.63 12
atis_flight#atis_airline 0.00 0.00 0.00 1
atis_flight_no 0.89 1.00 0.94 8
atis_flight_no#atis_airline 0.00 0.00 0.00 1
atis_flight_time 1.00 1.00 1.00 1
atis_ground_fare 1.00 1.00 1.00 7
atis_ground_service 1.00 1.00 1.00 36
atis_meal 1.00 1.00 1.00 6
atis_quantity 0.33 1.00 0.50 3
avg / total 0.99 0.98 0.98 700
avg / total 0.98 0.98 0.97 893
```

### Atis Slot Filling
Expand Down Expand Up @@ -170,83 +215,38 @@ B-depart_date.today_relative 1.00 1.00 1.00 9


### Snips Intent Prediction

```
准确率: 0.9764837625979843
宏平均精确率: 0.7449854331023172
微平均精确率: 0.9764837625979843
加权平均精确率: 0.9750854106720163
宏平均召回率: 0.7499221418525216
微平均召回率: 0.9764837625979843
加权平均召回率: 0.9764837625979843
宏平均F1-score: 0.5650970439524852
微平均F1-score: 0.9764837625979843
加权平均F1-score: 0.9741996309183255
准确率: 0.9842857142857143
宏平均精确率: 0.9858767424798239
微平均精确率: 0.9842857142857143
加权平均精确率: 0.9853440415050833
宏平均召回率: 0.9849107098074714
微平均召回率: 0.9842857142857143
加权平均召回率: 0.9842857142857143
宏平均F1-score: 0.9849281321280821
微平均F1-score: 0.9842857142857143
加权平均F1-score: 0.9843205635966433
混淆矩阵输出:
[[ 33 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0]
[ 0 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 1]
[ 0 0 48 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0]
[ 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0]
[ 0 0 0 0 38 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0]
[ 0 0 0 0 0 18 0 0 0 0 0 0 0 0 0 0 0 0
0 0]
[ 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 0 0 0
0 1]
[ 2 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 0
0 0]
[ 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0
0 0]
[ 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0
0 0]
[ 0 0 0 0 0 1 0 0 0 0 626 1 0 0 0 0 0 0
0 4]
[ 0 0 2 0 0 0 0 0 0 0 4 6 0 0 0 0 0 0
0 0]
[ 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0
0 0]
[ 0 0 0 0 0 0 0 0 0 0 0 0 0 8 0 0 0 0
0 0]
[ 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
0 0]
[ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
0 0]
[ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 0
0 0]
[ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 36
0 0]
[ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
6 0]
[ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 3]]
[[124 0 0 0 0 0 0]
[ 0 92 0 0 0 0 0]
[ 0 2 102 0 0 0 0]
[ 0 0 0 85 0 1 0]
[ 0 0 0 0 80 0 0]
[ 0 0 0 0 0 107 0]
[ 0 0 0 0 0 8 99]]
分类报告:
precision recall f1-score support
precision recall f1-score support
atis_abbreviation 0.94 1.00 0.97 33
atis_aircraft 1.00 0.89 0.94 9
atis_airfare 0.94 1.00 0.97 48
atis_airfare#atis_flight 0.00 0.00 0.00 1
atis_airline 1.00 1.00 1.00 38
atis_airport 0.95 1.00 0.97 18
atis_capacity 1.00 0.95 0.98 21
atis_city 1.00 0.67 0.80 6
atis_day_name 0.00 0.00 0.00 2
atis_distance 1.00 1.00 1.00 10
atis_flight 0.99 0.99 0.99 632
atis_flight#atis_airfare 0.86 0.50 0.63 12
atis_flight#atis_airline 0.00 0.00 0.00 1
atis_flight_no 0.89 1.00 0.94 8
atis_flight_no#atis_airline 0.00 0.00 0.00 1
atis_flight_time 1.00 1.00 1.00 1
atis_ground_fare 1.00 1.00 1.00 7
atis_ground_service 1.00 1.00 1.00 36
atis_meal 1.00 1.00 1.00 6
atis_quantity 0.33 1.00 0.50 3
AddToPlaylist 1.00 1.00 1.00 124
BookRestaurant 0.98 1.00 0.99 92
GetWeather 1.00 0.98 0.99 104
PlayMusic 1.00 0.99 0.99 86
RateBook 1.00 1.00 1.00 80
SearchCreativeWork 0.92 1.00 0.96 107
SearchScreeningEvent 1.00 0.93 0.96 107
avg / total 0.98 0.98 0.97 893
avg / total 0.99 0.98 0.98 700
```

### Snips Slot Filling
Expand Down Expand Up @@ -345,3 +345,46 @@ I-object_part_of_series_type 0.00 0.00 0.00 1
avg / total 0.95 0.95 0.95 3301
```

## Snips Slot Filling and Intent Prediction

Intent Prediction

```
准确率: 0.9814285714285714
宏平均精确率: 0.9827947881945077
微平均精确率: 0.9814285714285714
加权平均精确率: 0.9825026562964853
宏平均召回率: 0.9826050118106192
微平均召回率: 0.9814285714285714
加权平均召回率: 0.9814285714285714
宏平均F1-score: 0.9821708231356349
微平均F1-score: 0.9814285714285714
加权平均F1-score: 0.9814041423909138
混淆矩阵输出:
[[124 0 0 0 0 0 0]
[ 0 92 0 0 0 0 0]
[ 0 1 103 0 0 0 0]
[ 0 0 0 86 0 0 0]
[ 0 0 0 0 80 0 0]
[ 0 0 0 2 0 105 0]
[ 0 0 0 0 0 10 97]]
分类报告:
precision recall f1-score support
AddToPlaylist 1.00 1.00 1.00 124
BookRestaurant 0.99 1.00 0.99 92
GetWeather 1.00 0.99 1.00 104
PlayMusic 0.98 1.00 0.99 86
RateBook 1.00 1.00 1.00 80
SearchCreativeWork 0.91 0.98 0.95 107
SearchScreeningEvent 1.00 0.91 0.95 107
avg / total 0.98 0.98 0.98 700
```

Slot Filling

```
```

0 comments on commit f5d962c

Please sign in to comment.