Core: Defer reading Avro metadata until ManifestFile is read

[rdblue]

This improves job planning performance by moving ManifestFiles.read setup into the ParallelIterator that is used to plan tasks. ParallelIterator accepts an Iterable of CloseableIterable. The outer iterable is iterated over to submit tasks that run in the worker pool. In ManifestGroup, the Iterable that was returned would call ManifestFiles.read to prepare the inner iterable, but the ManifestReader needs to read Avro file metadata and will open a stream. That initial file open was running in the consuming thread as tasks were submitted, instead of in the worker pool.

This updates ManifestGroup to use a custom Iterable that defers calling ManifestFiles.read until the inner iterable is used.

[lirui-apache]

We encountered authorization failures reading manifest files after applied this PR, and thought it might be related. Since the worker pool in use is by default a global static pool, the threads in this pool may not be able to see the correct UGI submitting the operation. And hit permission denied errors when trying to open the stream. I wonder whether we should call doAs in the pool, or whether we should let different users use separate pools?