New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Profile hardware counters #85

Closed

NiuJ1ao wants to merge 40 commits into vhive-serverless:master from NiuJ1ao:perf_bench_test

Contributor

NiuJ1ao commented Jan 8, 2021 •

edited

Loading

A benchmark that bursts many requests per second to stress hardware resources. It collects user-defined hardware counters and plots charts of these counters.

NiuJ1ao requested a review from ustiugov

January 8, 2021 07:42

ustiugov requested changes

View reviewed changes

Member

ustiugov left a comment

just a minor change, the rest look good

.github/workflows/go.yml Outdated Show resolved Hide resolved

ustiugov reviewed

View reviewed changes

.github/workflows/go.yml Outdated Show resolved Hide resolved

NiuJ1ao force-pushed the perf_bench_test branch from d725b70 to 8060f8f Compare

January 8, 2021 17:14

ustiugov requested changes

View reviewed changes

Member

ustiugov left a comment

The code is fine but you need to rebase vs current master. Also, there are figures' changes in this PR for some reason.

NiuJ1ao force-pushed the perf_bench_test branch from 8060f8f to 44eb628 Compare

January 9, 2021 12:27

NiuJ1ao requested a review from ustiugov

January 9, 2021 12:29

ustiugov requested changes

View reviewed changes

Member

ustiugov left a comment

Squash two last commits in one, using rebase, as an exercise.

We should wait for the data before merging the PR.

.github/workflows/go.yml Outdated Show resolved Hide resolved

Member

ustiugov commented Jan 10, 2021

also, please add a clear PR description and sign off in it too.

ustiugov requested changes

View reviewed changes

Member

ustiugov left a comment

Need a minor modification to avoid problems in testing on a self-hosted runner.

perf_bench_test.go Outdated Show resolved Hide resolved

perf_bench_test.go Outdated Show resolved Hide resolved

Member

ustiugov commented Jan 12, 2021

also, please remove the created files or folders in the Makefile like in other bench tests.

NiuJ1ao self-assigned this

NiuJ1ao force-pushed the perf_bench_test branch from 44eb628 to 21e104f Compare

January 14, 2021 13:40

Member

ustiugov commented Jan 14, 2021

please rebase

NiuJ1ao force-pushed the perf_bench_test branch from 21e104f to 4d43d1f Compare

January 14, 2021 14:24

NiuJ1ao requested a review from ustiugov

January 14, 2021 15:43

NiuJ1ao marked this pull request as draft

January 14, 2021 15:44

Member

ustiugov commented Jan 14, 2021

Please squash the last two commits into one using git rebase.

ustiugov requested changes

View reviewed changes

Member

ustiugov left a comment

The code quality needs improvement. Some of the latest changes compromised it.

perf_bench_test.go Outdated Show resolved Hide resolved

perf_bench_test.go Outdated Show resolved Hide resolved

perf_bench_test.go Show resolved Hide resolved

perf_bench_test.go Outdated Show resolved Hide resolved

perf_bench_test.go Show resolved Hide resolved

NiuJ1ao force-pushed the perf_bench_test branch from 81874f7 to f0d33d4 Compare

January 16, 2021 15:45

NiuJ1ao marked this pull request as ready for review

January 17, 2021 13:36

ustiugov reviewed

View reviewed changes

perf_bench_test.go Outdated Show resolved Hide resolved

NiuJ1ao requested a review from ustiugov

January 17, 2021 13:44

ustiugov requested changes

View reviewed changes

Member

ustiugov left a comment

There are issues to be fixed.

.github/workflows/unit_tests.yml Outdated Show resolved Hide resolved

perf_bench_test.go Outdated Show resolved Hide resolved

perf_bench_test.go Outdated Show resolved Hide resolved

perf_bench_test.go Outdated Show resolved Hide resolved

perf_bench_test.go Show resolved Hide resolved

profile/perf.go Show resolved Hide resolved

profile/perf.go Show resolved Hide resolved

profile/perf.go Show resolved Hide resolved

profile/perf_metics.go Outdated Show resolved Hide resolved

profile/perf_metics.go Outdated Show resolved Hide resolved

ustiugov requested changes

View reviewed changes

perf_bench_test.go Outdated Show resolved Hide resolved

perf_bench_test.go Outdated Show resolved Hide resolved

perf_bench_test.go Outdated Show resolved Hide resolved

NiuJ1ao added 5 commits

January 23, 2021 02:58


          skeleton of perf_bench_test

d149b52

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          add TestBenchRequestPerSecond to bench

4340ba6

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          Minor fix bench

e44e762

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          Parallel boot VMs and minor fix bench TestBenchRequestPerSecond

027f656

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          change boot VMs back to sequential due to data race.

689e1a3

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>

NiuJ1ao added 21 commits

January 23, 2021 03:00


          Restruct UI design and refactor perf_profile.go

9d4c4b3

1. Move main function from perf_profile.go to perf_bench_test.go and plot function to plotter.go
2. Minor fix of perf implementation

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          Fix perf implementation and add tests for perf.go and plotter.go

05a77e3

User can choose profile duration and time interval, the script will omitting warmup and teardown periods and report the averages of left stats.

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          Set suitable images and corresponding RPS that stress CPU. Minor fix …

d13b315

…perf_bench_test.go and perf.go

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          refactor main loop of perf_bench_test.go and let SetWarmTime do only …

cfcc197

…once

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          delete profile bash script

6e27dd5

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          add unit test for running perf

b949049

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          change image names to vhiveease/

1f97d02

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          add testing of the profile module to github actions

56b9cc7

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>

minor fix git actions' naming and response validation check

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          remove created folders of TestBenchRequestPerSecond

3be6fd2

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          minor fix calculation error and typo of variable names

756a756

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          minor fix conflict between filepath and filename format

40ec77b

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          remove shut down VMs and only boot VMs when necessary

d623088

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          minor fix to test replacement of characters

e2f174a

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>

minor fix test expected value

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          change parameters of CSVPlotter to only accept file path and input fi…

51b8a4a

…le name

the output images will be stored under the file path, with input file.

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          split a function into logic pieces and another function uses these pi…

bed2c69

…eces to achieve different logic flow

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          Feature perf metrics: perf can automatically calculate metrics and pl…

fae9877

…ot corresponding charts

note: compatible with Intel Broadwell

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          split NewPerfStat and readPerfData into function units

8e57591

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          fix naming, descriptions and commits of running arguments and injecti…

a478426

…on function

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          merge profile test into unit test

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          change Y-axis to start from zero

e519565

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          move metrics to PMU events map to JSON file, refactor parameters of c…

b03905a

…alculation functions

add more metrics

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>

NiuJ1ao force-pushed the perf_bench_test branch from f0d33d4 to b03905a Compare

January 22, 2021 19:01

NiuJ1ao marked this pull request as draft

January 22, 2021 19:05

NiuJ1ao added 5 commits

February 5, 2021 02:36


          refactor perf profiler: split metrics into files and save interim res…

cb05a22

…ults

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          fix bug of incorrect step size on x-axis

aa19be8

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          store metric to events map in json file

c22b562

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          fix RPS calculation error, change profiler from perf to a PMU tool

fbb4bc0

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>


          integrate PMU tool to benchmark

cfde8bb

Signed-off-by: NiuJ1ao <26167136+NiuJ1ao@users.noreply.github.com>

NiuJ1ao closed this

HermioneKT pushed a commit that referenced this pull request


          # This is a combination of 154 commits.

# This is the 1st commit message:

refactor

# This is the commit message #2:

modify powerNodeSelector

# This is the commit message #3:

modify powerNodeSelector

# This is the commit message #4:

modify powerNodeSelector

# This is the commit message #5:

modify namespace

# This is the commit message #6:

modify namespace

# This is the commit message #7:

add namespace

# This is the commit message #8:

init power manager script

# This is the commit message #9:

add go code

# This is the commit message #10:

test

# This is the commit message #11:

test

# This is the commit message #12:

test

# This is the commit message #13:

test

# This is the commit message #14:

go.sum

# This is the commit message #15:

go sum

# This is the commit message #16:

go mod tidy

# This is the commit message #17:

test

# This is the commit message #18:

test

# This is the commit message #19:

test

# This is the commit message #20:

test

# This is the commit message #21:

test

# This is the commit message #22:

test

# This is the commit message #23:

test

# This is the commit message #24:

test

# This is the commit message #25:

test

# This is the commit message #26:

test

# This is the commit message #27:

test

# This is the commit message #28:

test

# This is the commit message #29:

print output

# This is the commit message #30:

test

# This is the commit message #31:

test

# This is the commit message #32:

test

# This is the commit message #33:

test

# This is the commit message #34:

test

# This is the commit message #35:

test

# This is the commit message #36:

test

# This is the commit message #37:

test

# This is the commit message #38:

test

# This is the commit message #39:

test

# This is the commit message #40:

refactor

# This is the commit message #41:

change val

# This is the commit message #42:

test

# This is the commit message #43:

test

# This is the commit message #44:

test

# This is the commit message #45:

exp1

# This is the commit message #46:

add url

# This is the commit message #47:

test

# This is the commit message #48:

test

# This is the commit message #49:

test auth

# This is the commit message #50:

test

# This is the commit message #51:

test

# This is the commit message #52:

test

# This is the commit message #53:

test

# This is the commit message #54:

test

# This is the commit message #55:

fix bug

# This is the commit message #56:

remove log

# This is the commit message #57:

test

# This is the commit message #58:

test

# This is the commit message #59:

test

# This is the commit message #60:

fix loop

# This is the commit message #61:

increase invocation

# This is the commit message #62:

test

# This is the commit message #63:

test

# This is the commit message #64:

test

# This is the commit message #65:

test

# This is the commit message #66:

test

# This is the commit message #67:

test

# This is the commit message #68:

test

# This is the commit message #69:

Test

# This is the commit message #70:

test

# This is the commit message #71:

test

# This is the commit message #72:

test

# This is the commit message #73:

test

# This is the commit message #74:

test

# This is the commit message #75:

test

# This is the commit message #76:

test

# This is the commit message #77:

test

# This is the commit message #78:

test

# This is the commit message #79:

test

# This is the commit message #80:

test

# This is the commit message #81:

test

# This is the commit message #82:

test

# This is the commit message #83:

test

# This is the commit message #84:

test

# This is the commit message #85:

test

# This is the commit message #86:

test

# This is the commit message #87:

test

# This is the commit message #88:

test

# This is the commit message #89:

test

# This is the commit message #90:

test1

# This is the commit message #91:

test

# This is the commit message #92:

test

# This is the commit message #93:

test

# This is the commit message #94:

test

# This is the commit message #95:

test

# This is the commit message #96:

test

# This is the commit message #97:

test

# This is the commit message #98:

test

# This is the commit message #99:

test

# This is the commit message #100:

test

# This is the commit message #101:

test

# This is the commit message #102:

test

# This is the commit message #103:

test

# This is the commit message #104:

test

# This is the commit message #105:

test

# This is the commit message #106:

test

# This is the commit message #107:

test

# This is the commit message #108:

test

# This is the commit message #109:

test1

# This is the commit message #110:

test

# This is the commit message #111:

test

# This is the commit message #112:

Test

# This is the commit message #113:

test

# This is the commit message #114:

test

# This is the commit message #115:

test

# This is the commit message #116:

test

# This is the commit message #117:

test

# This is the commit message #118:

test

# This is the commit message #119:

Test

# This is the commit message #120:

Test

# This is the commit message #121:

test

# This is the commit message #122:

test

# This is the commit message #123:

Test

# This is the commit message #124:

test

# This is the commit message #125:

test

# This is the commit message #126:

test

# This is the commit message #127:

test

# This is the commit message #128:

test

# This is the commit message #129:

test

# This is the commit message #130:

test

# This is the commit message #131:

test

# This is the commit message #132:

test

# This is the commit message #133:

test

# This is the commit message #134:

test

# This is the commit message #135:

test

# This is the commit message #136:

test

# This is the commit message #137:

test

# This is the commit message #138:

test

# This is the commit message #139:

test

# This is the commit message #140:

test

# This is the commit message #141:

test

# This is the commit message #142:

test

# This is the commit message #143:

Test

# This is the commit message #144:

Test

# This is the commit message #145:

Test

# This is the commit message #146:

Test

# This is the commit message #147:

test

# This is the commit message #148:

test

# This is the commit message #149:

test

# This is the commit message #150:

test

# This is the commit message #151:

test

# This is the commit message #152:

test

# This is the commit message #153:

test

# This is the commit message #154:

Test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment