reading spark log json file leads to PanicException

[romanovacca]

Polars version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of Polars.

Issue description

I'm trying to read autogenerated spark log files which automatically get created when a spark job runs. The log file consists of different 'events' and for some reason a panic exception occurs when I try to read the "org.apache.spark.sql.execution.ui.SparkListenerSQLExecutionStart" event provided in the example.

Included is the data on which the error occurs
test-spark.zip

Note that other parts of the file are read fine by the parser and i left them out to make it easier to debug.

Reproducible example

import polars as pl
pl.read_ndjson("test-spark")

Expected behavior

I would expect this to work and return all the values, and if a part of the json is empty, it should return it as empty/null

Let say this is intended, then at least I would expect to not get a panic exception.
To me it sounds like at some point in the processing we are indexing an empty vector. So adding some code to check this and prevent the issue would be a better solution.

Installed versions

'0.16.1'

[cmdlineluser]

It looks to be because of the empty list in "children": [] on line 26.

This appears to be a MRE:

import io
import polars as pl

pl.read_ndjson(io.StringIO("""{"foo": {"bar": []}}"""))

PanicException: index out of bounds: the len is 0 but the index is 0

thread '<unnamed>' panicked at 
'index out of bounds: the len is 0 but the index is 0', 
../polars-io/src/ndjson_core/buffer.rs:171:41

https://github.com/pola-rs/polars/blob/master/polars/polars-io/src/ndjson_core/buffer.rs#L170

[ritchie46]

Thanks for the breakdown @cmdlineluser 👍