Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable Monolithic build #9948

Closed
wants to merge 32 commits into from

Conversation

assignUser
Copy link
Collaborator

@assignUser assignUser commented May 28, 2024

This PR adds an option that combines all targets into a single velox target and build a single unified libvelox.a. This is entirely optional and defaults to off. This will make it easier for downstream users to use Velox by just linking to velox::velox in their own cmake instead of having to look through our cmake which targets they need. (Once we have a config etc.).

I have yet to test the install functionality (it's strictly a bonus, the monolithic build is the goal for now).

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 28, 2024
Copy link

netlify bot commented May 28, 2024

Deploy Preview for meta-velox canceled.

Name Link
🔨 Latest commit 6298482
🔍 Latest deploy log https://app.netlify.com/sites/meta-velox/deploys/669f1ae5762be900080400e7

@assignUser assignUser force-pushed the monolithic-build branch 2 times, most recently from 307cbcf to 4b38c9d Compare May 28, 2024 21:15
@assignUser assignUser changed the title [WIP] Enable Monolithic build Enable Monolithic build May 28, 2024
@assignUser
Copy link
Collaborator Author

I rebased, this should fix the test failures but the build works so I think this is review ready.

@kgpai I didn't add the license header to the dbgn cmakelists as it's absence looked intentional? Should we add it or exclude the file from the test?

@assignUser assignUser marked this pull request as ready for review May 28, 2024 21:17
Copy link
Contributor

@cryos cryos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, a few high level comments to consider.

CMake/VeloxUtils.cmake Outdated Show resolved Hide resolved
CMake/VeloxUtils.cmake Show resolved Hide resolved
velox/common/base/CMakeLists.txt Outdated Show resolved Hide resolved
Copy link
Collaborator

@majetideepak majetideepak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small comment.
I used libvelox.a in the Presto C++ build and it works!

CMake/VeloxUtils.cmake Outdated Show resolved Hide resolved
@assignUser assignUser force-pushed the monolithic-build branch 2 times, most recently from 5c50a2c to d28f42f Compare June 11, 2024 10:34
line_width: 80
tab_size: 2
use_tabchars: false
max_pargs_hwrap: 4
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default for this is six which I think is just a bit too much as it get's hard to read.

@assignUser
Copy link
Collaborator Author

While testing the new format I noticed that there are A LOT of files that are not correctly formatted but didn't show up because we only look at touched files. You can repro this by running ./scripts/check.py format tree --fix velox and check the changed files in git.

We should probably apply formatting across everything, merge that as a standalone commit and than create a git blame revs ignore file to avoid that commit showing up everywhere when people git blame. @kgpai @pedroerp @mbasmanova ?

@majetideepak
Copy link
Collaborator

@assignUser Linux Build / Linux release with adapters (pull_request) is now passing.

@majetideepak
Copy link
Collaborator

@assignUser Run Checks / Code Format (pull_request) is still failing.

@assignUser
Copy link
Collaborator Author

That's because of the changes to cake format now needing pyyaml. Once the container is rebuild it will pass. But I can also add temporary line to the job.

@majetideepak
Copy link
Collaborator

Once the container is rebuild it will pass. But I can also add temporary line to the job.

Let's add a temporary line to ensure it passes.

@assignUser
Copy link
Collaborator Author

I have opened #10262 as a follow up.

Copy link
Collaborator

@majetideepak majetideepak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @assignUser !

@majetideepak
Copy link
Collaborator

@assignUser can you rebase with main to get the linux CI jobs running?

@assignUser
Copy link
Collaborator Author

@majetideepak done, thanks

Copy link
Contributor

@kgpai kgpai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits, looks good . We might want to add some documentation or check to ensure that velox_* constructs are used in cmake over the cmake defaults in future.

@@ -68,6 +69,7 @@ option(
"Build a minimal set of components, including DWIO (file format readers/writers).
This will override other build options."
OFF)
option(VELOX_MONO_LIBRARY "Build single unified library." OFF)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: instead of MONO should we call it MONOLITHIC or UNIFIED (I prefer monolithic) . Unfortunately there are many projects called mono and saying MONOLITHIC just makes it very explicit.

# Create a library for each invocation.
velox_base_add_library(${TARGET} ${library_type} ${ARGN})
endif()
velox_install_library_headers()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesnt velox_base_add_library install library headers ?

@kgpai kgpai added the ready-to-merge PR that have been reviewed and are ready for merging. PRs with this tag notify the Velox Meta oncall label Jul 19, 2024
@majetideepak
Copy link
Collaborator

@assignUser there is a new merge conflict.

@facebook-github-bot
Copy link
Contributor

@kevinwilfong has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@kevinwilfong
Copy link
Contributor

Hi @assignUser I'm seeing a conflict in pyvelox/CMakeLists.txt, do you mind rebasing?

@assignUser
Copy link
Collaborator Author

@kevinwilfong done :)

@facebook-github-bot
Copy link
Contributor

@kevinwilfong has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@kevinwilfong merged this pull request in 9f64e42.

Copy link

Conbench analyzed the 1 benchmark run on commit 9f64e42f.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Merged ready-to-merge PR that have been reviewed and are ready for merging. PRs with this tag notify the Velox Meta oncall
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants