Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TestAddColumn2 timeout #37001

Closed
Tracked by #41316
hawkingrei opened this issue Aug 9, 2022 · 1 comment
Closed
Tracked by #41316

TestAddColumn2 timeout #37001

hawkingrei opened this issue Aug 9, 2022 · 1 comment
Assignees
@hawkingrei
Labels
affects-6.3 affects-6.4 affects-6.5 affects-6.6 affects-7.0 affects-7.1 affects-7.5 component/ddl This issue is related to DDL of TiDB. component/test severity/major type/bug The issue is confirmed as a bug.

Comments

@hawkingrei
Copy link
Member

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

 panic: test timed out after 30m0s
goroutine 3729466 [running]:
testing.(*M).startAlarm.func1()
	/usr/local/go/src/testing/testing.go:2036 +0xbb
created by time.goFunc
	/usr/local/go/src/time/sleep.go:176 +0x48
goroutine 1 [chan receive, 30 minutes]:
testing.(*T).Run(0xc0008a8340, {0x5b71b93, 0xe}, 0x5ce6c88)
	/usr/local/go/src/testing/testing.go:1494 +0x789
testing.runTests.func1(0x0?)
	/usr/local/go/src/testing/testing.go:1846 +0x9a
testing.tRunner(0xc0008a8340, 0xc000b3f980)
	/usr/local/go/src/testing/testing.go:1446 +0x217
testing.runTests(0xc000473220?, {0x89f8ae0, 0x251, 0x251}, {0x1?, 0x7f9500000073?, 0x910e0e0?})
	/usr/local/go/src/testing/testing.go:1844 +0x7ed
testing.(*M).Run(0xc000473220)
	/usr/local/go/src/testing/testing.go:1726 +0xa85
go.uber.org/goleak.VerifyTestMain({0x6e473c0, 0xc000473220}, {0xc000033e00, 0x4, 0x4})
	/home/prow/go/pkg/mod/go.uber.org/goleak@v1.1.12/testmain.go:53 +0x5a
github.com/pingcap/tidb/ddl_test.TestMain(0x8a6e360?)
	/go/tidb/ddl/main_test.go:68 +0x60b
main.main()
	_testmain.go:1347 +0x405
goroutine 64 [select, 30 minutes]:
github.com/pingcap/badger.(*blobGCHandler).run(0xc000760200, 0xc0000121b0)
	/home/prow/go/pkg/mod/github.com/pingcap/badger@v1.5.1-0.20220314162537-ab58fbf40580/blob.go:468 +0x177
created by github.com/pingcap/badger.(*blobManager).Open
	/home/prow/go/pkg/mod/github.com/pingcap/badger@v1.5.1-0.20220314162537-ab58fbf40580/blob.go:292 +0xaa6
goroutine 102 [select, 3 minutes]:
github.com/pingcap/badger.(*writeWorker).runWriteVLog(0xc0004161c0, 0xc000012300)
	/home/prow/go/pkg/mod/github.com/pingcap/badger@v1.5.1-0.20220314162537-ab58fbf40580/writer.go:94 +0x229
created by github.com/pingcap/badger.startWriteWorker
	/home/prow/go/pkg/mod/github.com/pingcap/badger@v1.5.1-0.20220314162537-ab58fbf40580/writer.go:64 +0x40f
goroutine 61 [select]:
github.com/pingcap/badger.(*DB).updateSize(0x7f957d80ab00?, 0xc000013698)
	/home/prow/go/pkg/mod/github.com/pingcap/badger@v1.5.1-0.20220314162537-ab58fbf40580/db.go:1039 +0x1d6
created by github.com/pingcap/badger.Open
	/home/prow/go/pkg/mod/github.com/pingcap/badger@v1.5.1-0.20220314162537-ab58fbf40580/db.go:328 +0x173c
goroutine 60 [chan receive, 22 minutes]:
github.com/pingcap/tidb/ddl_test.TestAddColumn2(0x0?)
	/go/tidb/ddl/db_table_test.go:898 +0xfb0
testing.tRunner(0xc0008a8680, 0x5ce6c88)
	/usr/local/go/src/testing/testing.go:1446 +0x217
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:1493 +0x75e
goroutine 30 [chan receive]:
github.com/golang/glog.(*loggingT).flushDaemon(0x0?)
	/home/prow/go/pkg/mod/github.com/golang/glog@v1.0.0/glog.go:882 +0x8b
created by github.com/golang/glog.init.0
	/home/prow/go/pkg/mod/github.com/golang/glog@v1.0.0/glog.go:410 +0x265
goroutine 31 [select]:
go.opencensus.io/stats/view.(*worker).start(0xc000a53380)
	/home/prow/go/pkg/mod/go.opencensus.io@v0.23.0/stats/view/worker.go:276 +0x185
created by go.opencensus.io/stats/view.init.0
	/home/prow/go/pkg/mod/go.opencensus.io@v0.23.0/stats/view/worker.go:34 +0xf7
goroutine 103 [chan receive, 3 minutes, locked to thread]:
github.com/pingcap/badger.(*writeWorker).runWriteLSM(0xc0004161c0, 0xc000012300)
	/home/prow/go/pkg/mod/github.com/pingcap/badger@v1.5.1-0.20220314162537-ab58fbf40580/writer.go:141 +0x21e
created by github.com/pingcap/badger.startWriteWorker
	/home/prow/go/pkg/mod/github.com/pingcap/badger@v1.5.1-0.20220314162537-ab58fbf40580/writer.go:65 +0x4b7
goroutine 104 [chan receive, 3 minutes]:
github.com/pingcap/badger.(*writeWorker).runMergeLSM(0xc0004161c0, 0xc000012300)
	/home/prow/go/pkg/mod/github.com/pingcap/badger@v1.5.1-0.20220314162537-ab58fbf40580/writer.go:154 +0xfe
created by github.com/pingcap/badger.startWriteWorker
	/home/prow/go/pkg/mod/github.com/pingcap/badger@v1.5.1-0.20220314162537-ab58fbf40580/writer.go:66 +0x55f
goroutine 63 [select, 30 minutes]:
github.com/pingcap/badger.Open.func4()
	/home/prow/go/pkg/mod/github.com/pingcap/badger@v1.5.1-0.20220314162537-ab58fbf40580/db.go:342 +0x305
created by github.com/pingcap/badger.Open
	/home/prow/go/pkg/mod/github.com/pingcap/badger@v1.5.1-0.20220314162537-ab58fbf40580/db.go:339 +0x1cc6
goroutine 98 [chan receive, 30 minutes]:
github.com/pingcap/badger.(*DB).runFlushMemTable(0xc000c98d80, 0xc000013c80)
	/home/prow/go/pkg/mod/github.com/pingcap/badger@v1.5.1-0.20220314162537-ab58fbf40580/db.go:920 +0x14c
created by github.com/pingcap/badger.Open
	/home/prow/go/pkg/mod/github.com/pingcap/badger@v1.5.1-0.20220314162537-ab58fbf40580/db.go:361 +0x217c
goroutine 62 [select]:
github.com/pingcap/badger/epoch.(*ResourceManager).collectLoop(0x7f957d80a500?, 0xc0000136c8)
	/home/prow/go/pkg/mod/github.com/pingcap/badger@v1.5.1-0.20220314162537-ab58fbf40580/epoch/manager.go:101 +0x175
created by github.com/pingcap/badger/epoch.NewResourceManager
	/home/prow/go/pkg/mod/github.com/pingcap/badger@v1.5.1-0.20220314162537-ab58fbf40580/epoch/manager.go:79 +0x1a5
goroutine 65 [select]:
github.com/pingcap/badger.(*levelsController).runWorker(0xc0005c4000, 0xc0000121e0, 0x0)
	/home/prow/go/pkg/mod/github.com/pingcap/badger@v1.5.1-0.20220314162537-ab58fbf40580/levels.go:212 +0x2e9
created by github.com/pingcap/badger.(*levelsController).startCompact
	/home/prow/go/pkg/mod/github.com/pingcap/badger@v1.5.1-0.20220314162537-ab58fbf40580/levels.go:180 +0xb7
goroutine 105 [select, 3 minutes]:
github.com/pingcap/tidb/store/mockstore/unistore/tikv.writeDBWorker.run({0xc0005d1860?, 0xc0007607c0?})
	/go/tidb/store/mockstore/unistore/tikv/write.go:96 +0x185
created by github.com/pingcap/tidb/store/mockstore/unistore/tikv.(*dbWriter).Open
	/go/tidb/store/mockstore/unistore/tikv/write.go:199 +0x118
goroutine 106 [select, 3 minutes]:
github.com/pingcap/tidb/store/mockstore/unistore/tikv.writeLockWorker.run({0xc0005d18c0?, 0xc0007607c0?})
	/go/tidb/store/mockstore/unistore/tikv/write.go:148 +0x1e5
created by github.com/pingcap/tidb/store/mockstore/unistore/tikv.(*dbWriter).Open
	/go/tidb/store/mockstore/unistore/tikv/write.go:206 +0x1f8
goroutine 107 [select]:
github.com/pingcap/tidb/store/mockstore/unistore/tikv.(*MVCCStore).runUpdateSafePointLoop(0xc000a52c00)
	/go/tidb/store/mockstore/unistore/tikv/mvcc.go:1706 +0x2e5
created by github.com/pingcap/tidb/store/mockstore/unistore/tikv.NewMVCCStore
	/go/tidb/store/mockstore/unistore/tikv/mvcc.go:87 +0x725
goroutine 108 [select, 3 minutes]:
github.com/pingcap/tidb/store/mockstore/unistore/tikv.(*MVCCStore).StartDeadlockDetection.func1()
	/go/tidb/store/mockstore/unistore/tikv/mvcc.go:1379 +0x11a
created by github.com/pingcap/tidb/store/mockstore/unistore/tikv.(*MVCCStore).StartDeadlockDetection
	/go/tidb/store/mockstore/unistore/tikv/mvcc.go:1377 +0x145
goroutine 109 [select]:
github.com/tikv/client-go/v2/oracle/oracles.(*pdOracle).updateTS(0xc000c2caf0, {0x6e66208, 0xc000144018}, 0x42d06a8?)
	/home/prow/go/pkg/mod/github.com/tikv/client-go/v2@v2.0.1-0.20220809034808-2ed2113d1090/oracle/oracles/pd.go:229 +0x1e5
created by github.com/tikv/client-go/v2/oracle/oracles.NewPdOracle
	/home/prow/go/pkg/mod/github.com/tikv/client-go/v2@v2.0.1-0.20220809034808-2ed2113d1090/oracle/oracles/pd.go:77 +0x21d
goroutine 110 [select]:
github.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop(0xc000cb0d00, 0x437dbda?)
	/home/prow/go/pkg/mod/github.com/tikv/client-go/v2@v2.0.1-0.20220809034808-2ed2113d1090/internal/locate/region_cache.go:449 +0x1cb
created by github.com/tikv/client-go/v2/internal/locate.NewRegionCache
	/home/prow/go/pkg/mod/github.com/tikv/client-go/v2@v2.0.1-0.20220809034808-2ed2113d1090/internal/locate/region_cache.go:420 +0x518
goroutine 111 [select]:
github.com/tikv/client-go/v2/tikv.(*KVStore).runSafePointChecker(0xc000184000)
	/home/prow/go/pkg/mod/github.com/tikv/client-go/v2@v2.0.1-0.20220809034808-2ed2113d1090/tikv/kv.go:234 +0x1a9
created by github.com/tikv/client-go/v2/tikv.NewKVStore
	/home/prow/go/pkg/mod/github.com/tikv/client-go/v2@v2.0.1-0.20220809034808-2ed2113d1090/tikv/kv.go:189 +0x6a5
goroutine 112 [select]:
github.com/tikv/client-go/v2/tikv.(*KVStore).safeTSUpdater(0xc000184000)
	/home/prow/go/pkg/mod/github.com/tikv/client-go/v2@v2.0.1-0.20220809034808-2ed2113d1090/tikv/kv.go:530 +0x319
created by github.com/tikv/client-go/v2/tikv.NewKVStore
	/home/prow/go/pkg/mod/github.com/tikv/client-go/v2@v2.0.1-0.20220809034808-2ed2113d1090/tikv/kv.go:190 +0x70f
goroutine 113 [select, 30 minutes]:
github.com/dgraph-io/ristretto.(*lfuPolicy).processItems(0xc000760f00)
	/home/prow/go/pkg/mod/github.com/dgraph-io/ristretto@v0.1.1-0.20220403145359-8e850b710d6d/policy.go:67 +0x10c
created by github.com/dgraph-io/ristretto.newPolicy
	/home/prow/go/pkg/mod/github.com/dgraph-io/ristretto@v0.1.1-0.20220403145359-8e850b710d6d/policy.go:51 +0x2db
goroutine 114 [select]:
github.com/dgraph-io/ristretto.(*Cache).processItems(0xc000c2cbd0)
	/home/prow/go/pkg/mod/github.com/dgraph-io/ristretto@v0.1.1-0.20220403145359-8e850b710d6d/cache.go:476 +0x258
created by github.com/dgraph-io/ristretto.NewCache
	/home/prow/go/pkg/mod/github.com/dgraph-io/ristretto@v0.1.1-0.20220403145359-8e850b710d6d/cache.go:213 +0x11a6
goroutine 3729679 [semacquire]:
sync.runtime_Semacquire(0xc007a91248?)
	/usr/local/go/src/runtime/sema.go:62 +0x25
sync.(*WaitGroup).Wait(0xc007a91240)
	/usr/local/go/src/sync/waitgroup.go:139 +0xa6
github.com/pingcap/tidb/executor.(*HashAggExec).prepare4ParallelExec.func2()
	/go/tidb/executor/aggregate.go:876 +0x59
created by github.com/pingcap/tidb/executor.(*HashAggExec).prepare4ParallelExec
	/go/tidb/executor/aggregate.go:875 +0x99e
goroutine 3729677 [select]:
github.com/pingcap/tidb/executor.(*HashAggFinalWorker).getPartialInput(...)
	/go/tidb/executor/aggregate.go:637
github.com/pingcap/tidb/executor.(*HashAggFinalWorker).consumeIntermData(0xc0073d6240, {0x6ecb0d0, 0xc004fe4b40})
	/go/tidb/executor/aggregate.go:658 +0x1c5
github.com/pingcap/tidb/executor.(*HashAggFinalWorker).run(0xc0073d6240, {0x6ecb0d0, 0xc004fe4b40}, 0xc007a91240)
	/go/tidb/executor/aggregate.go:765 +0xf6
created by github.com/pingcap/tidb/executor.(*HashAggExec).prepare4ParallelExec
	/go/tidb/executor/aggregate.go:873 +0x6e5
goroutine 3729672 [runnable]:
github.com/pingcap/tidb/executor.(*HashAggPartialWorker).getChildInput(...)
	/go/tidb/executor/aggregate.go:436
github.com/pingcap/tidb/executor.(*HashAggPartialWorker).run(0xc00df11f00, {0x6ecb0d0, 0xc004fe4b40}, 0xc007a91230, 0x5)
	/go/tidb/executor/aggregate.go:476 +0x2ac
created by github.com/pingcap/tidb/executor.(*HashAggExec).prepare4ParallelExec
	/go/tidb/executor/aggregate.go:861 +0x305
goroutine 34348 [chan receive]:
github.com/pingcap/tidb/executor.(*HashJoinExec).Next(0xc006ffaf00, {0x6e66208?, 0xc000144008}, 0xc00cbc80f0)
	/go/tidb/executor/join.go:712 +0xf7a
github.com/pingcap/tidb/executor.Next({0x6e66208, 0xc000144008}, {0x6e69cc0, 0xc006ffaf00}, 0xc00cbc80f0)
	/go/tidb/executor/executor.go:323 +0x5c4
github.com/pingcap/tidb/executor.(*SortExec).fetchRowChunks(0xc0047dea20, {0x6e66208, 0xc000144008})
	/go/tidb/executor/sort.go:198 +0xdd4
github.com/pingcap/tidb/executor.(*SortExec).Next(0xc0047dea20, {0x6e66208, 0xc000144008}, 0xc00cb17b30)
	/go/tidb/executor/sort.go:116 +0x794
github.com/pingcap/tidb/executor.Next({0x6e66208, 0xc000144008}, {0x6e6a7c0, 0xc0047dea20}, 0xc00cb17b30)
	/go/tidb/executor/executor.go:323 +0x5c4
github.com/pingcap/tidb/executor.(*ProjectionExec).unParallelExecute(0xc0047dec60, {0x6e66208, 0xc000144008}, 0xc00cbc80a0)
	/go/tidb/executor/projection.go:198 +0x2e5
github.com/pingcap/tidb/executor.(*ProjectionExec).Next(0xc0047dec60, {0x6e66208, 0xc000144008}, 0x20?)
	/go/tidb/executor/projection.go:185 +0x93
github.com/pingcap/tidb/executor.Next({0x6e66208, 0xc000144008}, {0x6e6a280, 0xc0047dec60}, 0xc00cbc80a0)
	/go/tidb/executor/executor.go:323 +0x5c4
github.com/pingcap/tidb/executor.(*ExecStmt).next(0xc009b3ed20, {0x6e66208, 0xc000144008}, {0x6e6a280, 0xc0047dec60}, 0xc00c8b87e0?)
	/go/tidb/executor/adapter.go:898 +0x8a
github.com/pingcap/tidb/executor.(*recordSet).Next(0xc00cbc8050, {0x6e66208, 0xc000144008}, 0xc00cbc80a0)
	/go/tidb/executor/adapter.go:152 +0x165
github.com/pingcap/tidb/util/sqlexec.DrainRecordSet({0x6e66208, 0xc000144008}, {0x6e66c18, 0xc007bcfad0}, 0xbf?)
	/go/tidb/util/sqlexec/restricted_sql_executor.go:225 +0xff
github.com/pingcap/tidb/ddl.(*session).execute(0xc006fda8e0, {0x6e66208, 0xc000144008}, {0xc007b3c780, 0xbf}, {0x5b75dca, 0xf})
	/go/tidb/ddl/ddl.go:1652 +0x3be
github.com/pingcap/tidb/ddl.(*ddl).getJob(0xc007736880, 0x8a55954?, 0x0, 0xc00e881a10)
	/go/tidb/ddl/job_table.go:98 +0x1fd
github.com/pingcap/tidb/ddl.(*ddl).getGeneralJob(0xc007736880, 0xc006fda8e0)
	/go/tidb/ddl/job_table.go:128 +0x78
github.com/pingcap/tidb/ddl.(*ddl).loadDDLJobAndRun(0xc007736880, 0xc00e881c98?, 0xc005cf1d88, 0xc00e881e00)
	/go/tidb/ddl/job_table.go:204 +0x409
github.com/pingcap/tidb/ddl.(*ddl).startDispatchLoop(0xc007736880)
	/go/tidb/ddl/job_table.go:188 +0x94e
github.com/pingcap/tidb/util.(*WaitGroupWrapper).Run.func1()
	/go/tidb/util/wait_group_wrapper.go:33 +0x74
created by github.com/pingcap/tidb/util.(*WaitGroupWrapper).Run
	/go/tidb/util/wait_group_wrapper.go:31 +0xe5
goroutine 3729665 [semacquire]:
sync.runtime_Semacquire(0xc006ffb0f8?)
	/usr/local/go/src/runtime/sema.go:62 +0x25
sync.(*WaitGroup).Wait(0xc006ffb0f0)
	/usr/local/go/src/sync/waitgroup.go:139 +0xa6
github.com/pingcap/tidb/executor.(*HashJoinExec).waitJoinWorkersAndCloseResultChan(0xc006ffaf00)
	/go/tidb/executor/join.go:419 +0x45
github.com/pingcap/tidb/util.WithRecovery(0xc0054bf100, 0x0)
	/go/tidb/util/misc.go:95 +0x6e
created by github.com/pingcap/tidb/executor.(*HashJoinExec).fetchAndProbeHashTable
	/go/tidb/executor/join.go:362 +0x785
goroutine 34441 [select, 23 minutes]:
github.com/pingcap/tidb/statistics/handle.(*Handle).drainColTask(0xc00585ac60, 0xc004ecf9e0)
	/go/tidb/statistics/handle/handle_hist.go:371 +0x185
github.com/pingcap/tidb/statistics/handle.(*Handle).HandleOneTask(0xc005000000?, 0x0, 0x15a0105?, {0x7f953172a898, 0xc004fe4240}, 0x0?)
	/go/tidb/statistics/handle/handle_hist.go:191 +0xf4
github.com/pingcap/tidb/statistics/handle.(*Handle).SubLoadWorker(0xc00585ac60, {0x6ecb0d0?, 0xc004fe4240}, 0x0?, 0xc006eed888)
	/go/tidb/statistics/handle/handle_hist.go:167 +0x188
created by github.com/pingcap/tidb/domain.(*Domain).StartLoadStatsSubWorkers
	/go/tidb/domain/domain.go:1390 +0x2c5
goroutine 34432 [select]:
github.com/pingcap/tidb/domain.(*Domain).LoadSysVarCacheLoop.func1()
	/go/tidb/domain/domain.go:1102 +0x1f1
created by github.com/pingcap/tidb/domain.(*Domain).LoadSysVarCacheLoop
	/go/tidb/domain/domain.go:1093 +0x325
goroutine 34349 [runnable]:
github.com/pingcap/tidb/ddl.(*worker).start(0xc007736f00, 0xc0067b6180)
	/go/tidb/ddl/ddl_worker.go:192 +0x534
created by github.com/pingcap/tidb/ddl.(*ddl).prepareWorkers4legacyDDL
	/go/tidb/ddl/ddl.go:633 +0x55e
goroutine 3729662 [select]:
github.com/pingcap/tidb/executor.(*HashJoinExec).runJoinWorker(0xc006ffaf00, 0x2, {0xc007a90e48, 0x1, 0x1})
	/go/tidb/executor/join.go:465 +0x7b1
github.com/pingcap/tidb/executor.(*HashJoinExec).fetchAndProbeHashTable.func2()
	/go/tidb/executor/join.go:359 +0xf8
github.com/pingcap/tidb/util.WithRecovery(0xc00c72a680, 0xc0054bf0d0)
	/go/tidb/util/misc.go:95 +0x6e
created by github.com/pingcap/tidb/executor.(*HashJoinExec).fetchAndProbeHashTable
	/go/tidb/executor/join.go:357 +0x3f7
goroutine 3729660 [select]:
github.com/pingcap/tidb/executor.(*HashJoinExec).runJoinWorker(0xc006ffaf00, 0x0, {0xc007a90e48, 0x1, 0x1})
	/go/tidb/executor/join.go:465 +0x7b1
github.com/pingcap/tidb/executor.(*HashJoinExec).fetchAndProbeHashTable.func2()
	/go/tidb/executor/join.go:359 +0xf8
github.com/pingcap/tidb/util.WithRecovery(0xc00c72a600, 0xc0054bf0b0)
	/go/tidb/util/misc.go:95 +0x6e
created by github.com/pingcap/tidb/executor.(*HashJoinExec).fetchAndProbeHashTable
	/go/tidb/executor/join.go:357 +0x3f7
goroutine 34442 [select, 23 minutes]:
github.com/pingcap/tidb/statistics/handle.(*Handle).drainColTask(0xc00585ac60, 0xc004ecf9e0)
	/go/tidb/statistics/handle/handle_hist.go:371 +0x185
github.com/pingcap/tidb/statistics/handle.(*Handle).HandleOneTask(0xc005000000?, 0x0, 0xc0091cac30?, {0x7f953172a898, 0xc004fe4480}, 0xc004db2110?)
	/go/tidb/statistics/handle/handle_hist.go:191 +0xf4
github.com/pingcap/tidb/statistics/handle.(*Handle).SubLoadWorker(0xc00585ac60, {0x6ecb0d0?, 0xc004fe4480}, 0x5018c19?, 0xc006eed888)
	/go/tidb/statistics/handle/handle_hist.go:167 +0x188
created by github.com/pingcap/tidb/domain.(*Domain).StartLoadStatsSubWorkers
	/go/tidb/domain/domain.go:1390 +0x2c5
goroutine 34353 [runnable]:
github.com/pingcap/tidb/domain.(*Domain).loadSchemaInLoop(0xc006eed7c0, {0x6e661d0, 0xc0056d1180}, 0x7a120)
	/go/tidb/domain/domain.go:615 +0x2a5
created by github.com/pingcap/tidb/domain.(*Domain).Init
	/go/tidb/domain/domain.go:897 +0x13ee
goroutine 3729675 [select]:
github.com/pingcap/tidb/executor.(*HashAggFinalWorker).getPartialInput(...)
	/go/tidb/executor/aggregate.go:637
github.com/pingcap/tidb/executor.(*HashAggFinalWorker).consumeIntermData(0xc0073d60c0, {0x6ecb0d0, 0xc004fe4b40})
	/go/tidb/executor/aggregate.go:658 +0x1c5
github.com/pingcap/tidb/executor.(*HashAggFinalWorker).run(0xc0073d60c0, {0x6ecb0d0, 0xc004fe4b40}, 0xc007a91240)
	/go/tidb/executor/aggregate.go:765 +0xf6
created by github.com/pingcap/tidb/executor.(*HashAggExec).prepare4ParallelExec
	/go/tidb/executor/aggregate.go:873 +0x6e5
goroutine 3729671 [runnable]:
github.com/pingcap/tidb/executor.(*HashAggPartialWorker).getChildInput(...)
	/go/tidb/executor/aggregate.go:436
github.com/pingcap/tidb/executor.(*HashAggPartialWorker).run(0xc00df11e40, {0x6ecb0d0, 0xc004fe4b40}, 0xc007a91230, 0x5)
	/go/tidb/executor/aggregate.go:476 +0x2ac
created by github.com/pingcap/tidb/executor.(*HashAggExec).prepare4ParallelExec
	/go/tidb/executor/aggregate.go:861 +0x305
goroutine 3729659 [chan receive]:
github.com/pingcap/tidb/executor.(*HashAggExec).parallelExec(0xc0052c2000, {0x6e66208, 0xc000144008}, 0xc00cbc8140)
	/go/tidb/executor/aggregate.go:908 +0x2a5
github.com/pingcap/tidb/executor.(*HashAggExec).Next(0xc0052c2000, {0x6e66208, 0xc000144008}, 0xc00cbc8140)
	/go/tidb/executor/aggregate.go:777 +0x147
github.com/pingcap/tidb/executor.Next({0x6e66208, 0xc000144008}, {0x6e69c80, 0xc0052c2000}, 0xc00cbc8140)
	/go/tidb/executor/executor.go:323 +0x5c4
github.com/pingcap/tidb/executor.(*HashJoinExec).fetchProbeSideChunks(0xc006ffaf00, {0x6e66208, 0xc000144008})
	/go/tidb/executor/join.go:219 +0x2ec
github.com/pingcap/tidb/executor.(*HashJoinExec).fetchAndProbeHashTable.func1()
	/go/tidb/executor/join.go:346 +0xbd
github.com/pingcap/tidb/util.WithRecovery(0xc006141600, 0xc0054bf0a0)
	/go/tidb/util/misc.go:95 +0x6e
created by github.com/pingcap/tidb/executor.(*HashJoinExec).fetchAndProbeHashTable
	/go/tidb/executor/join.go:344 +0x296
goroutine 34356 [select, 23 minutes]:
github.com/pingcap/tidb/domain.(*Domain).globalConfigSyncerKeeper(0xc006eed7c0)
	/go/tidb/domain/domain.go:560 +0x1b8
created by github.com/pingcap/tidb/domain.(*Domain).Init
	/go/tidb/domain/domain.go:902 +0x1592
goroutine 34444 [select, 3 minutes]:
github.com/pingcap/tidb/domain.(*Domain).DumpFileGcCheckerLoop.func1()
	/go/tidb/domain/domain.go:1306 +0x1f9
created by github.com/pingcap/tidb/domain.(*Domain).DumpFileGcCheckerLoop
	/go/tidb/domain/domain.go:1297 +0xaf
goroutine 3729673 [semacquire]:
sync.runtime_Semacquire(0xc007a91238?)
	/usr/local/go/src/runtime/sema.go:62 +0x25
sync.(*WaitGroup).Wait(0xc007a91230)
	/usr/local/go/src/sync/waitgroup.go:139 +0xa6
github.com/pingcap/tidb/executor.(*HashAggExec).waitPartialWorkerAndCloseOutputChs(0xc0052c2000, 0xc004995a00?)
	/go/tidb/executor/aggregate.go:824 +0x4f
github.com/pingcap/tidb/executor.(*HashAggExec).prepare4ParallelExec.func1()
	/go/tidb/executor/aggregate.go:864 +0x67
created by github.com/pingcap/tidb/executor.(*HashAggExec).prepare4ParallelExec
	/go/tidb/executor/aggregate.go:863 +0x634
goroutine 40217 [select]:
github.com/pingcap/tidb/ddl.(*ddl).DoDDLJob(0xc007736880, {0x6ecb0d0, 0xc005869200}, 0xc004819680)
	/go/tidb/ddl/ddl.go:982 +0xe4f
github.com/pingcap/tidb/ddl.(*ddl).AddColumn(0x1578085?, {0x6ecb0d0, 0xc005869200}, {{{0x5b60b68, 0x4}, {0x5b60b68, 0x4}}, {{0x5c1a5d1, 0x2}, {0x5c1a5d1, ...}}}, ...)
	/go/tidb/ddl/ddl_api.go:3577 +0x7f5
github.com/pingcap/tidb/ddl.(*ddl).AlterTable(0xc007736880, {0x6e66278, 0xc00c75c150}, {0x6ecb0d0, 0xc005869200}, 0xc007f8fbc0)
	/go/tidb/ddl/ddl_api.go:3097 +0xbd0
github.com/pingcap/tidb/executor.(*DDLExec).executeAlterTable(0xc00630ea00, {0x6e66278, 0xc00c75c150}, 0xc007f8fbc0)
	/go/tidb/executor/ddl.go:361 +0x295
github.com/pingcap/tidb/executor.(*DDLExec).Next(0xc00630ea00, {0x6e66278, 0xc00c75c120}, 0xc006a04af0?)
	/go/tidb/executor/ddl.go:146 +0x445
github.com/pingcap/tidb/executor.Next({0x6e66278, 0xc00c75c120}, {0x6e69ac0, 0xc00630ea00}, 0xc006a04b40)
	/go/tidb/executor/executor.go:323 +0x5c4
github.com/pingcap/tidb/executor.(*ExecStmt).next(0xc006faf860, {0x6e66278, 0xc00c75c120}, {0x6e69ac0, 0xc00630ea00}, 0xc000656a28?)
	/go/tidb/executor/adapter.go:898 +0x8a
github.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelayExecutor(0xc006faf860, {0x6e66278, 0xc00c75c120}, {0x6e69ac0?, 0xc00630ea00})
	/go/tidb/executor/adapter.go:698 +0x6f2
github.com/pingcap/tidb/executor.(*ExecStmt).handleNoDelay(0xc006faf860, {0x6e66278, 0xc00c75c120}, {0x6e69ac0, 0xc00630ea00?}, 0x0)
	/go/tidb/executor/adapter.go:548 +0x29d
github.com/pingcap/tidb/executor.(*ExecStmt).Exec(0xc006faf860, {0x6e66278, 0xc00c75c120})
	/go/tidb/executor/adapter.go:497 +0x108a
github.com/pingcap/tidb/session.runStmt({0x6e66208, 0xc000144008}, 0xc005869200, {0x6e751f0, 0xc006faf860?})
	/go/tidb/session/session.go:2142 +0x75b
github.com/pingcap/tidb/session.(*session).ExecuteStmt(0xc005869200, {0x6e66208, 0xc000144008}, {0x6e7b1e8, 0xc007f8fbc0})
	/go/tidb/session/session.go:2008 +0xedb
github.com/pingcap/tidb/session.(*session).Execute(0xc005869200, {0x6e66208, 0xc000144008}, {0x5c1a5c5, 0x32})
	/go/tidb/session/session.go:1540 +0x47d
github.com/pingcap/tidb/ddl_test.backgroundExec({0x6e8b1b0, 0xc000760f40}, {0x5c1a5c5, 0x32}, 0x6e66208?)
	/go/tidb/ddl/db_integration_test.go:1228 +0x196
created by github.com/pingcap/tidb/ddl_test.TestAddColumn2
	/go/tidb/ddl/db_table_test.go:897 +0xf92
goroutine 34347 [select, 23 minutes]:
github.com/pingcap/tidb/ddl.(*delRange).startEmulator(0xc00a082de0)
	/go/tidb/ddl/delete_range.go:160 +0x1af
created by github.com/pingcap/tidb/ddl.(*delRange).start
	/go/tidb/ddl/delete_range.go:142 +0xef
goroutine 34350 [runnable]:
github.com/pingcap/tidb/ddl.(*worker).start(0xc007736f80, 0xc0067b6180)
	/go/tidb/ddl/ddl_worker.go:192 +0x534
created by github.com/pingcap/tidb/ddl.(*ddl).prepareWorkers4legacyDDL
	/go/tidb/ddl/ddl.go:633 +0x55e
goroutine 34352 [select]:
github.com/pingcap/tidb/ddl.(*ddl).PollTiFlashRoutine(0xc007736880)
	/go/tidb/ddl/ddl_tiflash_api.go:576 +0x349
github.com/pingcap/tidb/util.(*WaitGroupWrapper).Run.func1()
	/go/tidb/util/wait_group_wrapper.go:33 +0x74
created by github.com/pingcap/tidb/util.(*WaitGroupWrapper).Run
	/go/tidb/util/wait_group_wrapper.go:31 +0xe5
goroutine 3729676 [select]:
github.com/pingcap/tidb/executor.(*HashAggFinalWorker).getPartialInput(...)
	/go/tidb/executor/aggregate.go:637
github.com/pingcap/tidb/executor.(*HashAggFinalWorker).consumeIntermData(0xc0073d6180, {0x6ecb0d0, 0xc004fe4b40})
	/go/tidb/executor/aggregate.go:658 +0x1c5
github.com/pingcap/tidb/executor.(*HashAggFinalWorker).run(0xc0073d6180, {0x6ecb0d0, 0xc004fe4b40}, 0xc007a91240)
	/go/tidb/executor/aggregate.go:765 +0xf6
created by github.com/pingcap/tidb/executor.(*HashAggExec).prepare4ParallelExec
	/go/tidb/executor/aggregate.go:873 +0x6e5
goroutine 34366 [select]:
github.com/pingcap/tidb/domain.(*Domain).globalBindHandleWorkerLoop.func1()
	/go/tidb/domain/domain.go:1184 +0x27d
created by github.com/pingcap/tidb/domain.(*Domain).globalBindHandleWorkerLoop
	/go/tidb/domain/domain.go:1171 +0xfa
goroutine 3729680 [runnable]:
sync.runtime_Semacquire(0xc007a91228?)
	/usr/local/go/src/runtime/sema.go:62 +0x25
sync.(*WaitGroup).Wait(0xc007a91220)
	/usr/local/go/src/sync/waitgroup.go:139 +0xa6
github.com/pingcap/tidb/executor.(*HashAggExec).waitAllWorkersAndCloseFinalOutputCh(0xc0052c2000, {0xc00e608fb8, 0x3, 0xc008299210?})
	/go/tidb/executor/aggregate.go:836 +0x5e
created by github.com/pingcap/tidb/executor.(*HashAggExec).prepare4ParallelExec
	/go/tidb/executor/aggregate.go:884 +0xaf2
goroutine 3729678 [select]:
github.com/pingcap/tidb/executor.(*HashAggFinalWorker).getPartialInput(...)
	/go/tidb/executor/aggregate.go:637
github.com/pingcap/tidb/executor.(*HashAggFinalWorker).consumeIntermData(0xc0073d6300, {0x6ecb0d0, 0xc004fe4b40})
	/go/tidb/executor/aggregate.go:658 +0x1c5
github.com/pingcap/tidb/executor.(*HashAggFinalWorker).run(0xc0073d6300, {0x6ecb0d0, 0xc004fe4b40}, 0xc007a91240)
	/go/tidb/executor/aggregate.go:765 +0xf6
created by github.com/pingcap/tidb/executor.(*HashAggExec).prepare4ParallelExec
	/go/tidb/executor/aggregate.go:873 +0x6e5
goroutine 34355 [select]:
github.com/pingcap/tidb/domain.(*Domain).infoSyncerKeeper(0xc006eed7c0)
	/go/tidb/domain/domain.go:537 +0x290
created by github.com/pingcap/tidb/domain.(*Domain).Init
	/go/tidb/domain/domain.go:901 +0x1512
goroutine 3729663 [select]:
github.com/pingcap/tidb/executor.(*HashJoinExec).runJoinWorker(0xc006ffaf00, 0x3, {0xc007a90e48, 0x1, 0x1})
	/go/tidb/executor/join.go:465 +0x7b1
github.com/pingcap/tidb/executor.(*HashJoinExec).fetchAndProbeHashTable.func2()
	/go/tidb/executor/join.go:359 +0xf8
github.com/pingcap/tidb/util.WithRecovery(0xc00c72a6c0, 0xc0054bf0e0)
	/go/tidb/util/misc.go:95 +0x6e
created by github.com/pingcap/tidb/executor.(*HashJoinExec).fetchAndProbeHashTable
	/go/tidb/executor/join.go:357 +0x3f7
goroutine 34357 [select]:
github.com/pingcap/tidb/domain.(*Domain).topologySyncerKeeper(0xc006eed7c0)
	/go/tidb/domain/domain.go:583 +0x28c
created by github.com/pingcap/tidb/domain.(*Domain).Init
	/go/tidb/domain/domain.go:905 +0x163a
goroutine 3729664 [select]:
github.com/pingcap/tidb/executor.(*HashJoinExec).runJoinWorker(0xc006ffaf00, 0x4, {0xc007a90e48, 0x1, 0x1})
	/go/tidb/executor/join.go:465 +0x7b1
github.com/pingcap/tidb/executor.(*HashJoinExec).fetchAndProbeHashTable.func2()
	/go/tidb/executor/join.go:359 +0xf8
github.com/pingcap/tidb/util.WithRecovery(0xc00c72a700, 0xc0054bf0f0)
	/go/tidb/util/misc.go:95 +0x6e
created by github.com/pingcap/tidb/executor.(*HashJoinExec).fetchAndProbeHashTable
	/go/tidb/executor/join.go:357 +0x3f7
goroutine 3729674 [select]:
github.com/pingcap/tidb/executor.(*HashAggFinalWorker).getPartialInput(...)
	/go/tidb/executor/aggregate.go:637
github.com/pingcap/tidb/executor.(*HashAggFinalWorker).consumeIntermData(0xc0073d6000, {0x6ecb0d0, 0xc004fe4b40})
	/go/tidb/executor/aggregate.go:658 +0x1c5
github.com/pingcap/tidb/executor.(*HashAggFinalWorker).run(0xc0073d6000, {0x6ecb0d0, 0xc004fe4b40}, 0xc007a91240)
	/go/tidb/executor/aggregate.go:765 +0xf6
created by github.com/pingcap/tidb/executor.(*HashAggExec).prepare4ParallelExec
	/go/tidb/executor/aggregate.go:873 +0x6e5
goroutine 34367 [select]:
github.com/pingcap/tidb/domain.(*Domain).handleEvolvePlanTasksLoop.func1()
	/go/tidb/domain/domain.go:1221 +0x1a5
created by github.com/pingcap/tidb/domain.(*Domain).handleEvolvePlanTasksLoop
	/go/tidb/domain/domain.go:1214 +0x14a
goroutine 34440 [select, 23 minutes]:
github.com/pingcap/tidb/statistics/handle.(*Handle).drainColTask(0xc00585ac60, 0xc004ecf9e0)
	/go/tidb/statistics/handle/handle_hist.go:371 +0x185
github.com/pingcap/tidb/statistics/handle.(*Handle).HandleOneTask(0x0?, 0x0, 0x0?, {0x7f953172a898, 0xc004fe4000}, 0x0?)
	/go/tidb/statistics/handle/handle_hist.go:191 +0xf4
github.com/pingcap/tidb/statistics/handle.(*Handle).SubLoadWorker(0xc00585ac60, {0x6ecb0d0?, 0xc004fe4000}, 0x36800f9?, 0xc006eed888)
	/go/tidb/statistics/handle/handle_hist.go:167 +0x188
created by github.com/pingcap/tidb/domain.(*Domain).StartLoadStatsSubWorkers
	/go/tidb/domain/domain.go:1390 +0x2c5
goroutine 34439 [select, 23 minutes]:
github.com/pingcap/tidb/statistics/handle.(*Handle).drainColTask(0xc00585ac60, 0xc004ecf9e0)
	/go/tidb/statistics/handle/handle_hist.go:371 +0x185
github.com/pingcap/tidb/statistics/handle.(*Handle).HandleOneTask(0xc00c84b500?, 0x0, 0x1576919?, {0x7f953172a898, 0xc004735d40}, 0x1578085?)
	/go/tidb/statistics/handle/handle_hist.go:191 +0xf4
github.com/pingcap/tidb/statistics/handle.(*Handle).SubLoadWorker(0xc00585ac60, {0x6ecb0d0?, 0xc004735d40}, 0xc004585ee0?, 0xc006eed888)
	/go/tidb/statistics/handle/handle_hist.go:167 +0x188
created by github.com/pingcap/tidb/domain.(*Domain).StartLoadStatsSubWorkers
	/go/tidb/domain/domain.go:1390 +0x2c5
goroutine 34351 [select]:
github.com/pingcap/tidb/ddl.(*ddl).startCleanDeadTableLock(0xc007736880)
	/go/tidb/ddl/ddl.go:1101 +0x237
created by github.com/pingcap/tidb/ddl.(*ddl).Start
	/go/tidb/ddl/ddl.go:664 +0x73a
goroutine 3729661 [select]:
github.com/pingcap/tidb/executor.(*HashJoinExec).runJoinWorker(0xc006ffaf00, 0x1, {0xc007a90e48, 0x1, 0x1})
	/go/tidb/executor/join.go:465 +0x7b1
github.com/pingcap/tidb/executor.(*HashJoinExec).fetchAndProbeHashTable.func2()
	/go/tidb/executor/join.go:359 +0xf8
github.com/pingcap/tidb/util.WithRecovery(0xc00c72a640, 0xc0054bf0c0)
	/go/tidb/util/misc.go:95 +0x6e
created by github.com/pingcap/tidb/executor.(*HashJoinExec).fetchAndProbeHashTable
	/go/tidb/executor/join.go:357 +0x3f7
goroutine 34346 [select, 22 minutes]:
github.com/pingcap/tidb/ddl.(*ddl).limitDDLJobs(0xc007736880)
	/go/tidb/ddl/ddl_worker.go:276 +0x21c
github.com/pingcap/tidb/util.(*WaitGroupWrapper).Run.func1()
	/go/tidb/util/wait_group_wrapper.go:33 +0x74
created by github.com/pingcap/tidb/util.(*WaitGroupWrapper).Run
	/go/tidb/util/wait_group_wrapper.go:31 +0xe5
goroutine 34443 [select, 23 minutes]:
github.com/pingcap/tidb/statistics/handle.(*Handle).drainColTask(0xc00585ac60, 0xc004ecf9e0)
	/go/tidb/statistics/handle/handle_hist.go:371 +0x185
github.com/pingcap/tidb/statistics/handle.(*Handle).HandleOneTask(0xc00c84b5c0?, 0x0, 0x1576919?, {0x7f953172a898, 0xc004fe46c0}, 0x1578085?)
	/go/tidb/statistics/handle/handle_hist.go:191 +0xf4
github.com/pingcap/tidb/statistics/handle.(*Handle).SubLoadWorker(0xc00585ac60, {0x6ecb0d0?, 0xc004fe46c0}, 0xc0007e8ee0?, 0xc006eed888)
	/go/tidb/statistics/handle/handle_hist.go:167 +0x188
created by github.com/pingcap/tidb/domain.(*Domain).StartLoadStatsSubWorkers
	/go/tidb/domain/domain.go:1390 +0x2c5
goroutine 34354 [select, 3 minutes]:
github.com/pingcap/tidb/domain.(*Domain).topNSlowQueryLoop(0xc006eed7c0)
	/go/tidb/domain/domain.go:505 +0x2d6
created by github.com/pingcap/tidb/domain.(*Domain).Init
	/go/tidb/domain/domain.go:900 +0x1492
goroutine 34428 [select, 3 minutes]:
github.com/pingcap/tidb/domain.(*Domain).LoadPrivilegeLoop.func1()
	/go/tidb/domain/domain.go:1052 +0x1f1
created by github.com/pingcap/tidb/domain.(*Domain).LoadPrivilegeLoop
	/go/tidb/domain/domain.go:1043 +0x4aa
goroutine 34445 [select, 3 minutes]:
github.com/pingcap/tidb/domain.(*Domain).LoadSigningCertLoop.func1()
	/go/tidb/domain/domain.go:1661 +0x173
created by github.com/pingcap/tidb/domain.(*Domain).LoadSigningCertLoop
	/go/tidb/domain/domain.go:1654 +0xaf

https://prow.pingcap.net/view/gs/pingcapprow/logs/tidb_race/1556966033699377152

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiDB version? (Required)
@hawkingrei hawkingrei added the type/bug The issue is confirmed as a bug. label Aug 9, 2022
@ti-chi-bot ti-chi-bot added may-affects-4.0 This bug maybe affects 4.0.x versions. may-affects-5.0 This bug maybe affects 5.0.x versions. may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.0 may-affects-6.1 may-affects-6.2 labels Aug 10, 2022
@ChenPeng2013 ChenPeng2013 removed may-affects-4.0 This bug maybe affects 4.0.x versions. may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-5.0 This bug maybe affects 5.0.x versions. may-affects-6.0 may-affects-6.1 may-affects-6.2 labels Aug 10, 2022
@Defined2014 Defined2014 mentioned this issue Feb 14, 2023
Open
@jebter jebter added the component/ddl This issue is related to DDL of TiDB. label Sep 11, 2023
@D3Hunter
Copy link
Contributor

open for too long without response, plz reopen it when needed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hawkingrei hawkingrei

Labels
affects-6.3 affects-6.4 affects-6.5 affects-6.6 affects-7.0 affects-7.1 affects-7.5 component/ddl This issue is related to DDL of TiDB. component/test severity/major type/bug The issue is confirmed as a bug.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@D3Hunter @hawkingrei @jebter @ChenPeng2013 @VelocityLight @ti-chi-bot