You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I see a similar issue with the same error closed, but i am able to repro in the latest version, but only when i do spark submit without the default settings.
Example:--executor-cores 2 --num-executors 15, with the default settings it just works fine.
the issue worsens as i increase the number of executorcores and executors.
Exception:
ava.lang.IllegalStateException: In partition 28 of approved_nhv_np, with consumer group sparkconsumerlocal, request seqNo 418108744 is less than the received seqNo 418108745. The earliest seqNo is 417139352 and the last seqNo is 418520434
at org.apache.spark.eventhubs.client.CachedEventHubsReceiver.checkCursor(CachedEventHubsReceiver.scala:152)
at org.apache.spark.eventhubs.client.CachedEventHubsReceiver.org$apache$spark$eventhubs$client$CachedEventHubsReceiver$$receive(CachedEventHubsReceiver.scala:172)
at org.apache.spark.eventhubs.client.CachedEventHubsReceiver$.receive(CachedEventHubsReceiver.scala:237)
at org.apache.spark.eventhubs.rdd.EventHubsRDD.compute(EventHubsRDD.scala:120)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
SBT Entry:
libraryDependencies += "com.microsoft.azure" %% "azure-eventhubs-spark" % "2.3.12"
The text was updated successfully, but these errors were encountered:
We have fixed an issue in the connector resulting in "IllegalStateException" in PR #490. However, the issue you are seeing is different since the requested seq# is within the range of available events in the service.
Also, we have identified and fixed an issue on the service side where the seq# of the returned event was ahead of the requested seq# even when the requested seq# was within the range. It seems like you have hit that issue.
Please upgrade the library to the latest version and let us know if you see the error again.
I see a similar issue with the same error closed, but i am able to repro in the latest version, but only when i do spark submit without the default settings.
Example:--executor-cores 2 --num-executors 15, with the default settings it just works fine.
the issue worsens as i increase the number of executorcores and executors.
Exception:
ava.lang.IllegalStateException: In partition 28 of approved_nhv_np, with consumer group sparkconsumerlocal, request seqNo 418108744 is less than the received seqNo 418108745. The earliest seqNo is 417139352 and the last seqNo is 418520434
at org.apache.spark.eventhubs.client.CachedEventHubsReceiver.checkCursor(CachedEventHubsReceiver.scala:152)
at org.apache.spark.eventhubs.client.CachedEventHubsReceiver.org$apache$spark$eventhubs$client$CachedEventHubsReceiver$$receive(CachedEventHubsReceiver.scala:172)
at org.apache.spark.eventhubs.client.CachedEventHubsReceiver$.receive(CachedEventHubsReceiver.scala:237)
at org.apache.spark.eventhubs.rdd.EventHubsRDD.compute(EventHubsRDD.scala:120)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
SBT Entry:
libraryDependencies += "com.microsoft.azure" %% "azure-eventhubs-spark" % "2.3.12"
The text was updated successfully, but these errors were encountered: