Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix thread pool hang #56346

Merged
merged 4 commits into from
Jul 27, 2021
Merged

Fix thread pool hang #56346

merged 4 commits into from
Jul 27, 2021

Conversation

kouvel
Copy link
Member

@kouvel kouvel commented Jul 27, 2021

Fixes #55642

- In dotnet#53471 the thread count goal was moved out of `ThreadCounts`, it turns out that are a few subtle races that it was avoiding. There are other ways to fix it, but I've added the goal back into `ThreadCounts` for now.
- Reverted PR dotnet#55985, which worked around the issue in the CI

Fixes dotnet#55642
@kouvel kouvel added this to the 6.0.0 milestone Jul 27, 2021
@kouvel kouvel self-assigned this Jul 27, 2021
@ghost
Copy link

ghost commented Jul 27, 2021

Tagging subscribers to this area: @mangod9
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes #55642

Author: kouvel
Assignees: kouvel
Labels:

area-System.Threading

Milestone: 6.0.0

@stephentoub
Copy link
Member

Thanks for tracking this down!

There was some suspicion this may also have been contributing to some of the CI hangs we've seen recently. It'll be interesting to see if their rate decreases after this goes in.

@danmoseley
Copy link
Member

Is there anything that could be done to create a test that would catch this kind of failure? Or do we already have that in the Quic tests 😄

@wfurt
Copy link
Member

wfurt commented Jul 27, 2021

Seems like tricky business @danmoseley. It was found by standard conformance Stream tests (with synchronous IO).
On 2Core machine and sync-over-async inside MsQuic wrapper.

Copy link
Member

@wfurt wfurt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The QUIC part looks good to me. I'll give it try tomorrow.

@danmoseley
Copy link
Member

True, but I have seen @kouvel devise unit tests that consistently repro the most sporadic issues, so I thought I'd ask 😄

@davidfowl
Copy link
Member

I wonder if coyote could come in handy here https://microsoft.github.io/coyote/

@kouvel
Copy link
Member Author

kouvel commented Jul 27, 2021

I had some difficulty in reproing it initially when I tried but upon trying a few more things I got a fairly frequent repro now on my Windows machine, added a test.

kouvel added a commit to kouvel/dotnet-diagnostics that referenced this pull request Jul 27, 2021
- Depends on dotnet/runtime#56346
- Reverted commit 3d57bee from PR dotnet#2324 since the relevant change to `ThreadCounts` was reverted in dotnet/runtime#56346
@kouvel kouvel merged commit eaaf43f into dotnet:main Jul 27, 2021
@kouvel kouvel deleted the BcFix branch July 27, 2021 16:59
kouvel added a commit to dotnet/diagnostics that referenced this pull request Jul 27, 2021
- Depends on dotnet/runtime#56346
- Reverted commit 3d57bee from PR #2324 since the relevant change to `ThreadCounts` was reverted in dotnet/runtime#56346
@ghost ghost locked as resolved and limited conversation to collaborators Aug 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[QUIC] Intermittent hangs in CI
6 participants