Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assertion failed: received sequence number doesn't match request sequence numbe #448

Closed
PKHeb opened this issue May 24, 2019 · 1 comment

Comments

@PKHeb
Copy link

PKHeb commented May 24, 2019

I see a similar issue with the same error closed, but i am able to repro in the latest version, but only when i do spark submit without the default settings.
Example:--executor-cores 2 --num-executors 15, with the default settings it just works fine.
the issue worsens as i increase the number of executorcores and executors.
Exception:
ava.lang.IllegalStateException: In partition 28 of approved_nhv_np, with consumer group sparkconsumerlocal, request seqNo 418108744 is less than the received seqNo 418108745. The earliest seqNo is 417139352 and the last seqNo is 418520434
at org.apache.spark.eventhubs.client.CachedEventHubsReceiver.checkCursor(CachedEventHubsReceiver.scala:152)
at org.apache.spark.eventhubs.client.CachedEventHubsReceiver.org$apache$spark$eventhubs$client$CachedEventHubsReceiver$$receive(CachedEventHubsReceiver.scala:172)
at org.apache.spark.eventhubs.client.CachedEventHubsReceiver$.receive(CachedEventHubsReceiver.scala:237)
at org.apache.spark.eventhubs.rdd.EventHubsRDD.compute(EventHubsRDD.scala:120)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
SBT Entry:
libraryDependencies += "com.microsoft.azure" %% "azure-eventhubs-spark" % "2.3.12"

@nyaghma
Copy link
Contributor

nyaghma commented Jun 24, 2021

We have fixed an issue in the connector resulting in "IllegalStateException" in PR #490. However, the issue you are seeing is different since the requested seq# is within the range of available events in the service.
Also, we have identified and fixed an issue on the service side where the seq# of the returned event was ahead of the requested seq# even when the requested seq# was within the range. It seems like you have hit that issue.
Please upgrade the library to the latest version and let us know if you see the error again.

@nyaghma nyaghma closed this as completed Jun 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants