Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[AUDIT] ReaderIterator signature changed #10713

Closed
Tracked by #9259
razajafri opened this issue Apr 16, 2024 · 2 comments
Closed
Tracked by #9259

[AUDIT] ReaderIterator signature changed #10713

razajafri opened this issue Apr 16, 2024 · 2 comments
Assignees
@razajafri
Labels
audit_4.0.0 Audit related tasks for 4.0.0 bug Something isn't working wontfix This will not be worked on

Comments

@razajafri
Copy link
Collaborator

As part of apache/spark@17430fe Spark made a change to stop sending the PID manually. A part of that change was to change the signature of the ReaderIterator to take an Option[Long] instead of the previous Option[Int]. This is a compilation error in the Plugin for Spark 4.0.0
@razajafri razajafri added bug Something isn't working ? - Needs Triage Need team to review and classify audit_4.0.0 Audit related tasks for 4.0.0 labels Apr 16, 2024
@razajafri razajafri mentioned this issue Apr 16, 2024
Closed
@razajafri razajafri changed the title ReaderIterator signature changed [AUDIT] ReaderIterator signature changed Apr 16, 2024
@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Apr 16, 2024
@amahussein
Copy link
Collaborator

@razajafri FWIW, the spark commits have been reverted

  • Revert "[SPARK-45302][PYTHON] Remove PID communication between Python workers when no demon is used" apache/spark@e8f529bb89a
  • Revert "Revert "[SPARK-45302][PYTHON] Remove PID communication between Python workers when no demon is used"" apache/spark@b3c11ef8e09
  • Revert "[SPARK-45302][PYTHON] Remove PID communication between Pythonworkers when no demon is used" apache/spark@c6aaa18e6cf

I will let you decide if this issue can be closed.

@razajafri
Copy link
Collaborator Author

The changes have been reverted in Spark

@razajafri razajafri self-assigned this May 29, 2024
@sameerz sameerz added the wontfix This will not be worked on label Jun 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@razajafri razajafri

Labels
audit_4.0.0 Audit related tasks for 4.0.0 bug Something isn't working wontfix This will not be worked on
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@mattahrens @sameerz @razajafri @amahussein