Unexpected message id from the client (sync ids different) #12640
Comments
|
Currently I find this issue ~10-20 per day in my logfile. Unfortunately I have no idea where this is coming from. The problem is that everytime a error message is shown to the user... |
|
I activated the debug logging, and got the following (unfortunately no special information):
|
|
Additional info: Once today I got the following message:
and some milliseconds after "Unexpected message id from the client. Expected sync id: 2, got 1. more details logged on DEBUG level." the following:
|
|
Without only this information it would feel like one server message to the client is lost as the server should for each response send a syncId that it expects the client to have in the next request. In short: Without any way to reproduce this is really hard to investigate. @Legioth would you have any ideas on how @thomaskiesl could try to investigate the issue on their end? |
|
Does anyone have an idea how to find this out? I got this message every day multiple times? Additional logging? |
|
There's supposed to be some additional information about the message payload after There are a couple of possible causes for why old sequence numbers would show up unexpectedly. Finding out the cause is usually very difficult without a good way of reproducing.
|
|
@Legioth I recevie for example the following info within the debug file: |
|
I'm experiencing this issue too, same vaadin version |
|
Also getting this on Vaadin 14.7.0. Upon getting this it leads to an internal error page on the UI. I was able to replicate it by setting up a Chrome network throttling profile with a Download of 100 kbits/s, Upload of 50 kbits/s and a latency of 500. Then I quickly clicked on a paged table's paginator so that it would change pages and data being fetched and from time to time clicked on a row on the table. After a minute or so the UI went blank while the server log showed Before that I also see some messages in the logs. Issue seems to be related to #9778 |
|
I'm using Vaadin 24.2.3 with Java 21, SpringBoot 3.2 and Spring Framework 6.1 (on macOS Ventura 13.5) and still see this issue: Any news on this? Cheers, |
|
Same problem here on JDK17, vaadin 24.1.11, SB3.1 - fairly trivial application. 20% of e2e-Tests fail randomly with something similar to |
|
Same issue for me on vaadin version= 24.2.6 and springboot=3.0.6 |
|
Same issue for me. |
|
We have the same issue. It randomly occurs after logging in when opening the first UI after Login. Interestingly the Message Start output ist empty. When debugging I see that the rpcArray is empty which causes the debug output to be empty.
It just happens on one application and I can't find the difference that causes it. |
|
We have the same issue. Java: 17 |
|
Same Issue here, makes the Web basically unusable... |
|
Any updates on this? Many of my users are not happy with this situation!? |
mshabarov
added
Impact: High
Severity: Major
and removed
Impact: Low
Severity: Minor
labels
|
That error is actually reported multiple times and has been discussed on many tickets. The error itself is not a bug as is, but logged as part of the failsafe mechanism in Vaadin. When packets are sent back and forth between server and browser, they include serial number so that both parties can verify that no packet is being lost. There are natural reasons for packet loss
When communication goes un-sync Vaadin detects that and does resynchronize. That is done on purpose in order to restore the operation. When you see this error, one should observe does it happen in correlation with some usage pattern of the application, frequency, when it is happening etc. All these can give some hints whether it is due some natural cause of the error. As you see the list of the natural causes are all flaky, hence deterministic re-production may be difficult. The cases where it has been a bug in Vaadin have been difficult to debug, but with rigorous attempt of following the statistics, close collaboration with the reporter, ruling out natural causes we have been able to find those and fixed them. Naturally we would like to hope that we have identified all of them, and only the natural causes remain. |
|
I see the error on my development machine from time to time. So I can exclude 1,2,4,5. We actually switched from Sophos to SentinelOne in the mean time so I guess I can also rule out 3. |
|
Thanks everyone who commented and thanks @ssindelar for the feedback and willing be up for the meeting. This is that I was going to propose: have a video call and investigate the issue together. What I'm afraid of is that this may be not only limited to logging, but to a resync issue in Vaadin that may cause broken UI, e.g. empty page and page load freezing. Based on the observations, we may need to implement more robust logic for resync and UIDL overall. Any volunteers are appreciated who can meet with us and show us the problem. Feel free to ping me, e.g. in our Discord ( |
|
Happening to me too. Running vaadin 24.3.8 on JDK 21, all dependencies for springboot as defined by vaadin-bom.
Only way to login again is by closing the browser and start clean again. I haven't seen this case while using Chrome. Hope this helps for debugging. |
This sounds like there's something with the security configuration that causes messages to be eaten after logging in again or maybe that some of the tracking state is not reset. |
|
Same seems to be tentative issue java.lang.UnsupportedOperationException: Unexpected message id from the client. Expected sync id: 0, got 1. more details logged on DEBUG level. After investigations base on @TatuLund inputs: Azure WAF was set into prevent mode without filtering unnecessary groups of rules. ~1 of 7 heartbeat calls was rejected, and some of updates calls were rejected too. all of those are false positives of some SQL injection rules.
from my POV I am happy that app crash due to those blockers on waf, otherwise it would have been even harder to follow other bugs that might come... |
|
Currently it's not happening anymore. vaadin 24.3.12 |
|
I think I tracked down the problem. Yesterday in the late afternoon the problem resurfaced with a nearly 100% reproduceability long enough so I could do some debugging before disappearing again a couple of minutes later. I think the cause of the problem is a request with the "wrong" session id. The expection with the "Unexpected message id from the client" is a result of this but not the cause of the problem. Sadly it couldn't test my findings because it error disappeared so suddenly it reoccured. What I found:
The first thing I noticed was that the second request after clicking the Login Button was recieving a session expired response. To understand whats happening we need to do a short deep dive into Spring Security and Tomcats session handling. When the response of the first request arrives the cookie is updated with the new session id. The second request is send most of the time before the first one returns. So it get's send with the "old" session id. This usually works because the login takes some time and the second request has got the session before the id is changed. With a breakpoint in ManagerBase.findSession I was able to reproduce the exception by stopping both threads and letting the login update the session id before continueing the second request: While it's currently not happening without the breakpoints and manual continuation, I'm very confident I would see a request with the outdated session id that can't find it's session. When a VaadinRequest can't find the session a reconnect is triggered. Now it depends on the timing of the reconnect if the user ends up at the Login or the page after. A reconnect on the page after the login usually goes unnoticed because it looks like a refresh or a little longer load time. |
|
@ssindelar thank you very much for the thorough investigation! This is really helpful! |
|
I found a good way to reproduce it. Setting the tomcat max threads to 1 forces a serial processing of the requests and now the second request always gets the sessionExpired:true response. Spring boot: The bug only occurs when using the vaadin login component. Session id usage of a "failed" login I'm no Javascript expert but waiting for the first request to finish before sending out the second should fix it. A possible (bad) workaround would be to make the login slower, so the second request is started after the Session ID is updated. That also explains why we only get the issue on one application because this application has the fastest Login. The others are doing a REST Request to another service for the login. |
|
Shouldn't this bug be moved to github.com/vaadin/vaadin-login/issues ? |
|
Hi, we are also facing this issue randomly. Forcing Chrome to Low-end mobile profile increases chances to hit the bug. We are on vaadin 24.3.12 |
|
Hi, I am facing the same issue. In my case, it always happens during login. It dont appear during local development but when uploaded to cloud, it returns same errors. many thanks, |
|
We also have this issue in production. It's great that finally, there is some way to reproduce it, thanks @ssindelar! However, I don't think login is the only place where this can happen. We have screenshots of customers where this apparently happens in the middle of a working session. Therefore, I wonder, what is it that login does "wrong" that leads to this issue? |
|
An update on the login issue. The problem seems to be that when you submit the form, Spring Security changes the sessionId, but at the same time the client also sends a UIDL login event with the "old" session cookie. |
|
In my applications, this error appears more often when end user device is mobile Safari on iPad with mobile connections. |
|
@timomeinen , then you should check this issue instead: #20010 |
|
Thank you, @TatuLund. I also suspected that the push might be causing the issue, so I disabled Interestingly, I have a similar application used exclusively by backoffice users on desktop clients, and it doesn’t exhibit these issues. Note: Our MDM prevents iOS18 until now. So all clients use iOS17. |
|
Our users are also seeing this sporadically, result is a rather ugly 'invalid json response from server' on the screen. No login form involved, desktop/laptop devices on internal corporate network. Push disabled. Latest 24.4.x |
|
@jorgheymans 'invalid json response from server' is not usually related to synchronization as that error is more often related to unexpected closing of the session while client is still running. Then when there is next request, Vaadin does bootstraping and severs the index page, which is naturally a invalid json. This invalid json error can be avoided if there is Vaadin-Refresh token in the content served. Then client will notice this and also restart itself. |
@mcollovati Your explanation makes sense to me. How does this apply when you invalidate the session manually using Is your draft going to improve things for these situations of things as well? |
|
The draft change on Flow would help in the case of concurrent requests, when one of those invalidates the session and also triggers a redirect. Not sure about your case. If there are no concurrent requests, the cause of the warnings may be different. |
and others
Description of the bug
I receive sometimes the same error message on my production system. But I do not have any idea when this happen:
2021-12-27 07:27:43,174 INFO | https-jsse-nio-18443-exec-10 | com.vaadin.flow.server.communication.ServerRpcHandler | Ignoring old duplicate message from the client. Expected: 1748, got: 1747It seems it is the same bug as #9778
Minimal reproducible example
Unfortunately I'm not possible to reproduce the issue. I only find the messages in the logfile.
Versions
The text was updated successfully, but these errors were encountered: