Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Http2_PendingSend_SendsReset test failing in CI #2131

Closed
wfurt opened this issue Jan 24, 2020 · 4 comments · Fixed by #35569
Closed

Http2_PendingSend_SendsReset test failing in CI #2131

wfurt opened this issue Jan 24, 2020 · 4 comments · Fixed by #35569
Assignees
@alnikola
Labels
area-System.Net.Http test-bug Problem in test source code (most likely) test-run-core Test failures in .NET Core test runs
Milestone
5.0.0

Comments

@wfurt
Copy link
Member

wfurt commented Jan 24, 2020

https://helix.dot.net/api/2019-06-17/jobs/cd21187c-48d5-439a-bcd6-7b022dfe2148/workitems/System.Net.Http.Functional.Tests/console

   System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http2.Http2_PendingSend_SendsReset(waitForData: False) [FAIL]
      Assert.Equal() Failure
      Expected: 0
      Actual:   1
      Stack Trace:
        /_/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http2.cs(1678,0): at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http2.<>c__DisplayClass59_0.<<Http2_PendingSend_SendsReset>b__1>d.MoveNext()
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(83,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks)
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(111,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks)
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(71,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout)
        /_/src/libraries/Common/tests/System/Net/Http/Http2LoopbackServer.cs(186,0): at System.Net.Test.Common.Http2LoopbackServer.CreateClientAndServerAsync(Func`2 clientFunc, Func`2 serverFunc, Int32 timeout)
        /_/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http2.cs(1643,0): at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http2.Http2_PendingSend_SendsReset(Boolean waitForData)

I saw 22 failures in the last 30 days in Kusto. This test is marked as OuterLoop because it is using Task.Delay. But there is no real reason to be unreliable since it us using LoopbackServer.
@wfurt wfurt added area-System.Net.Http test-run-core Test failures in .NET Core test runs os-arch-linux labels Jan 24, 2020
@Dotnet-GitSync-Bot Dotnet-GitSync-Bot added the untriaged New issue has not been triaged by the area owner label Jan 24, 2020
@scalablecory scalablecory mentioned this issue Jan 27, 2020
Merged
@karelz karelz removed the untriaged New issue has not been triaged by the area owner label Feb 14, 2020
@karelz karelz added this to the 5.0 milestone Feb 14, 2020
@AriNuer
Copy link

AriNuer commented Mar 2, 2020 •
edited
Loading

same test failed 'System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http2.Http2_PendingSend_SendsReset' has failed here runtime-libraries outerloop#20200301.1

Details:
https://dev.azure.com/dnceng/public/_build/results?buildId=542108&view=ms.vss-test-web.build-test-results-tab&runId=17175548&resultId=102753&paneView=debug

Error message
Assert.Equal() Failure
Expected: 0
Actual: 1

Stack trace
   at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http2.<>c__DisplayClass59_0.<<Http2_PendingSend_SendsReset>b__1>d.MoveNext() in /_/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http2.cs:line 1677
--- End of stack trace from previous location ---
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 83
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 111
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 71
   at System.Net.Test.Common.Http2LoopbackServer.CreateClientAndServerAsync(Func`2 clientFunc, Func`2 serverFunc, Int32 timeout) in /_/src/libraries/Common/tests/System/Net/Http/Http2LoopbackServer.cs:line 191
   at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http2.Http2_PendingSend_SendsReset(Boolean waitForData) in /_/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http2.cs:line 1642
--- End of stack trace from previous location ---

@stephentoub stephentoub mentioned this issue Mar 6, 2020
Merged
@jkotas jkotas added os-linux Linux OS (any supported distro) and removed os-arch-linux labels Mar 8, 2020
@ManickaP ManickaP mentioned this issue Mar 12, 2020
Merged
@AriNuer
Copy link

AriNuer commented Mar 19, 2020

Test System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http2.Http2_PendingSend_SendsReset(waitForData: False) failed again here runtime-libraries outerloop:20200318.1

Error message:
Assert.Equal() Failure
Expected: 0
Actual: 1

Stack trace:
at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http2.<>c__DisplayClass59_0.<<Http2_PendingSend_SendsReset>b__1>d.MoveNext() in /_/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http2.cs:line 1677
--- End of stack trace from previous location ---
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 83
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 111
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 71
   at System.Net.Test.Common.Http2LoopbackServer.CreateClientAndServerAsync(Func`2 clientFunc, Func`2 serverFunc, Int32 timeout) in /_/src/libraries/Common/tests/System/Net/Http/Http2LoopbackServer.cs:line 191
   at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http2.Http2_PendingSend_SendsReset(Boolean waitForData) in /_/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http2.cs:line 1642
--- End of stack trace from previous location ---

Details:
https://dev.azure.com/dnceng/public/_build/results?buildId=563484&view=ms.vss-test-web.build-test-results-tab&runId=17734980&resultId=103182&paneView=debug

@AriNuer
Copy link

AriNuer commented Mar 31, 2020

Test System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http2.Http2_PendingSend_SendsReset failed again here runtime-libraries outerloop:20200330.1
Error message:

Assert.Equal() Failure
Expected: 0
Actual: 1

Stack trace:

 at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http2.<>c__DisplayClass59_0.<<Http2_PendingSend_SendsReset>b__1>d.MoveNext() in /_/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http2.cs:line 1677
--- End of stack trace from previous location ---
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 83
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 111
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 71
   at System.Net.Test.Common.Http2LoopbackServer.CreateClientAndServerAsync(Func`2 clientFunc, Func`2 serverFunc, Http2Options http2Options, Int32 timeout) in /_/src/libraries/Common/tests/System/Net/Http/Http2LoopbackServer.cs:line 196
   at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http2.Http2_PendingSend_SendsReset(Boolean waitForData) in /_/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http2.cs:line 1642
--- End of stack trace from previous location ---

Details:
https://dev.azure.com/dnceng/public/_build/results?buildId=579439&view=ms.vss-test-web.build-test-results-tab&runId=18211948&resultId=102663&paneView=debug

@karelz karelz removed the os-linux Linux OS (any supported distro) label Apr 3, 2020
@aik-jahoda aik-jahoda self-assigned this Apr 9, 2020
@v-haren
Copy link

v-haren commented Apr 21, 2020

also failed in job: runtime-libraries outerloop 20200420.2

failed test: System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http2.Http2_PendingSend_SendsReset(waitForData: False)

Error message

Assert.Equal() Failure
Expected: 0
Actual: 1


Stack trace
   at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http2.<>c__DisplayClass59_0.<<Http2_PendingSend_SendsReset>b__1>d.MoveNext() in /_/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http2.cs:line 1677
--- End of stack trace from previous location ---
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 83
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 111
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 71
   at System.Net.Test.Common.Http2LoopbackServer.CreateClientAndServerAsync(Func`2 clientFunc, Func`2 serverFunc, Http2Options http2Options, Int32 timeout) in /_/src/libraries/Common/tests/System/Net/Http/Http2LoopbackServer.cs:line 196
   at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http2.Http2_PendingSend_SendsReset(Boolean waitForData) in /_/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http2.cs:line 1642
--- End of stack trace from previous location ---

@alnikola alnikola assigned alnikola and unassigned aik-jahoda Apr 27, 2020
@alnikola alnikola mentioned this issue Apr 28, 2020
Merged
alnikola added a commit that referenced this issue Apr 28, 2020
@alnikola
Server and client are synchronized via TaskCompletionSource (#35569)
63ae78b 
Http2_PendingSend_SendsReset test fails because client sends the second request with EndOfStream flag sooner than the server reads the last RstStream frame of the first request. This is caused by too short delay before the second request is made on the client. This PR replaces Task.Delay with a synchronization via TaskCompletionSource.

Fixes #2131
@karelz karelz added the test-bug Problem in test source code (most likely) label May 1, 2020
@ghost ghost locked as resolved and limited conversation to collaborators Dec 11, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@alnikola alnikola

Labels
area-System.Net.Http test-bug Problem in test source code (most likely) test-run-core Test failures in .NET Core test runs
Projects
None yet
Milestone
5.0.0
Development

Successfully merging a pull request may close this issue.

Server and client are synchronized via TaskCompletionSource alnikola/runtime
8 participants
@aik-jahoda @karelz @jkotas @wfurt @v-haren @AriNuer @Dotnet-GitSync-Bot @alnikola