[BUG] test_parquet_read_encryption fails

[revans2]

Describe the bug
When I run the python integration tests on Spark 3.2.0 or spark_3.3.0

env -u SPARK_CONF_DIR SPARK_HOME=~/spark_3.2.0/ ./run_pyspark_from_build.sh -k 'parquet and encryption'

Most of them fail with an error like

Caused by: java.lang.ClassNotFoundException: Class org.apache.parquet.crypto.keytools.mocks.InMemoryKMS not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2571)
	at org.apache.parquet.hadoop.util.ConfigurationUtil.getClassFromConfig(ConfigurationUtil.java:33)

All I did was to run buildall.sh and then cd into the integration tests directory and run the tests.

I looks like

@pytest.mark.skipif(os.environ.get('INCLUDE_PARQUET_HADOOP_TEST_JAR', 'false') == 'false', reason='INCLUDE_PARQUET_HADOOP_TEST_JAR is disabled')

is the culprit. It is being set to true in the run_pyspark_from_build.sh without ever even checking if the hadoop dependency was placed where it is expected to be. But honestly I have no idea because I don't know what jar is expected to be on the classpath. It could be the fact that I build all of the versions and somehow the wrong dependency was downloaded. Either way this is a bad user experience.

[revans2]

FAILED ../../src/main/python/parquet_test.py::test_parquet_read_encryption[reader_confs3-] - py4j.protocol.Py4JJavaError: An error occurred while calling o310.parquet.
FAILED ../../src/main/python/parquet_test.py::test_parquet_read_encryption[reader_confs0-] - py4j.protocol.Py4JJavaError: An error occurred while calling o310.parquet.
FAILED ../../src/main/python/parquet_test.py::test_parquet_read_encryption[reader_confs1-parquet] - py4j.protocol.Py4JJavaError: An error occurred while calling o310.parquet.
FAILED ../../src/main/python/parquet_test.py::test_parquet_read_encryption[reader_confs2-parquet] - py4j.protocol.Py4JJavaError: An error occurred while calling o310.parquet.
FAILED ../../src/main/python/parquet_test.py::test_parquet_read_encryption[reader_confs5-] - py4j.protocol.Py4JJavaError: An error occurred while calling o310.parquet.
FAILED ../../src/main/python/parquet_test.py::test_parquet_read_encryption[reader_confs2-] - py4j.protocol.Py4JJavaError: An error occurred while calling o310.parquet.
FAILED ../../src/main/python/parquet_test.py::test_parquet_read_encryption[reader_confs4-] - py4j.protocol.Py4JJavaError: An error occurred while calling o310.parquet.
FAILED ../../src/main/python/parquet_test.py::test_parquet_read_encryption[reader_confs3-parquet] - py4j.protocol.Py4JJavaError: An error occurred while calling o310.parquet.
FAILED ../../src/main/python/parquet_test.py::test_parquet_read_encryption[reader_confs0-parquet] - py4j.protocol.Py4JJavaError: An error occurred while calling o310.parquet.
FAILED ../../src/main/python/parquet_test.py::test_parquet_read_encryption[reader_confs1-] - py4j.protocol.Py4JJavaError: An error occurred while calling o310.parquet.
FAILED ../../src/main/python/parquet_test.py::test_parquet_read_encryption[reader_confs4-parquet] - py4j.protocol.Py4JJavaError: An error occurred while calling o310.parquet.
FAILED ../../src/main/python/parquet_test.py::test_parquet_read_encryption[reader_confs5-parquet] - py4j.protocol.Py4JJavaError: An error occurred while calling o400.parquet.