Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove references of shims folder in docs [skip ci] #5009

Merged
merged 3 commits into from
Mar 23, 2022
Merged

Remove references of shims folder in docs [skip ci] #5009

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5d8d7e4
removing mention of shims folder in docs
razajafri Mar 22, 2022
4505e9e
empty commit to stop requiring CI
razajafri Mar 22, 2022
85ce638
Update CONTRIBUTING.md
razajafri Mar 22, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 3 additions & 5 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,11 +160,9 @@ Spark version, open [Maven tool window](https://www.jetbrains.com/help/idea/2021
select one of the `release3xx` profiles (e.g, `release320`) for Apache Spark 3.2.0, and click "Reload"
if not triggered automatically.

There is a known issue with the shims/spark3xx submodules. After being enabled once, a module such as shims/spark312
may remain active in IDEA even though you explicitly disable the Maven profile `release312` in the Maven tool window.
With an extra IDEA shim module loaded the IDEA internal build "Build->Build Project" is likely to fail
(whereas it has no adverse effect on Maven build). As a workaround, locate the pom.xml under the extraneous IDEA module,
right-click on it and select "Maven->Ignore Projects".
There is a known issue where, even after selecting a different maven profile in maven submenu, the source folders from
previously selected profile may remain active. To get around this you have to manually reload the maven project from
the maven side menu
razajafri marked this conversation as resolved.
Show resolved Hide resolved

If you see Scala symbols unresolved (highlighted red) in IDEA please try the following steps to resolve it:
- Make sure there are no relevant poms in "File->Settings->Build Tools->Maven->Ignored Files"
Expand Down
7 changes: 3 additions & 4 deletions docs/dev/shims.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,10 +89,9 @@ by having the documented facade classes with a shim specifier in their package n
The second issue that every parent class/trait in the inheritance graph is loaded using the classloader outside
Plugin's control. Therefore, all this bytecode must reside in the conventional jar location, and it must
be bitwise-identical across *all* shims. The only way to keep the source code for shared functionality unduplicated,
(i.e., in `sql-plugin/src/main/scala` as opposed to be duplicated either in `shims/spark3xx` submodules or over
`sql-plugin/src/main/3*/scala` source code roots) is to delay inheriting `ShuffleManager` until as late as possible,
as close as possible to the facade class where we have to split the source code anyways. Use traits as much
as possible for flexibility.
(i.e., in `sql-plugin/src/main/scala` as opposed to be duplicated in `sql-plugin/src/main/3*/scala` source code roots)
is to delay inheriting `ShuffleManager` until as late as possible, as close as possible to the facade class where we
have to split the source code anyway. Use traits as much as possible for flexibility.

### Late Initialization of Public Classes' Ancestors

Expand Down