Skip to content

Commit

Permalink
SPARK-2350: Don't NPE while launching drivers
Browse files Browse the repository at this point in the history
Prior to this change, we could throw a NPE if we launch a driver while
another one is waiting, because removing from an iterator while iterating
over it is not safe.
  • Loading branch information
aarondav committed Jul 3, 2014
1 parent bc7041a commit 1cf1cf4
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -481,7 +481,7 @@ private[spark] class Master(
// First schedule drivers, they take strict precedence over applications
val shuffledWorkers = Random.shuffle(workers) // Randomization helps balance drivers
for (worker <- shuffledWorkers if worker.state == WorkerState.ALIVE) {
for (driver <- waitingDrivers) {
for (driver <- List(waitingDrivers: _*)) { // iterate over a copy of waitingDrivers
if (worker.memoryFree >= driver.desc.mem && worker.coresFree >= driver.desc.cores) {
launchDriver(worker, driver)
waitingDrivers -= driver
Expand Down

0 comments on commit 1cf1cf4

Please sign in to comment.