Skip to content

Commit

Permalink
Remove references of shims folder in docs [skip ci] (#5009)
Browse files Browse the repository at this point in the history
* removing mention of shims folder in docs

Signed-off-by: Raza Jafri <rjafri@nvidia.com>

* empty commit to stop requiring CI

Signed-off-by: Raza Jafri <rjafri@nvidia.com>

* Update CONTRIBUTING.md

Co-authored-by: Jason Lowe <jlowe@nvidia.com>

Co-authored-by: Raza Jafri <rjafri@nvidia.com>
Co-authored-by: Jason Lowe <jlowe@nvidia.com>
  • Loading branch information
3 people authored Mar 23, 2022
1 parent be1873a commit e0e8c17
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 9 deletions.
8 changes: 3 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,11 +160,9 @@ Spark version, open [Maven tool window](https://www.jetbrains.com/help/idea/2021
select one of the `release3xx` profiles (e.g, `release320`) for Apache Spark 3.2.0, and click "Reload"
if not triggered automatically.

There is a known issue with the shims/spark3xx submodules. After being enabled once, a module such as shims/spark312
may remain active in IDEA even though you explicitly disable the Maven profile `release312` in the Maven tool window.
With an extra IDEA shim module loaded the IDEA internal build "Build->Build Project" is likely to fail
(whereas it has no adverse effect on Maven build). As a workaround, locate the pom.xml under the extraneous IDEA module,
right-click on it and select "Maven->Ignore Projects".
There is a known issue where, even after selecting a different Maven profile in the Maven submenu, the source folders from
a previously selected profile may remain active. To get around this you have to manually reload the Maven project from
the Maven side menu.

If you see Scala symbols unresolved (highlighted red) in IDEA please try the following steps to resolve it:
- Make sure there are no relevant poms in "File->Settings->Build Tools->Maven->Ignored Files"
Expand Down
7 changes: 3 additions & 4 deletions docs/dev/shims.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,10 +89,9 @@ by having the documented facade classes with a shim specifier in their package n
The second issue that every parent class/trait in the inheritance graph is loaded using the classloader outside
Plugin's control. Therefore, all this bytecode must reside in the conventional jar location, and it must
be bitwise-identical across *all* shims. The only way to keep the source code for shared functionality unduplicated,
(i.e., in `sql-plugin/src/main/scala` as opposed to be duplicated either in `shims/spark3xx` submodules or over
`sql-plugin/src/main/3*/scala` source code roots) is to delay inheriting `ShuffleManager` until as late as possible,
as close as possible to the facade class where we have to split the source code anyways. Use traits as much
as possible for flexibility.
(i.e., in `sql-plugin/src/main/scala` as opposed to be duplicated in `sql-plugin/src/main/3*/scala` source code roots)
is to delay inheriting `ShuffleManager` until as late as possible, as close as possible to the facade class where we
have to split the source code anyway. Use traits as much as possible for flexibility.

### Late Initialization of Public Classes' Ancestors

Expand Down

0 comments on commit e0e8c17

Please sign in to comment.