Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Qualification tool support UI code generation #5470

Merged
merged 9 commits into from
May 18, 2022
Merged

Qualification tool support UI code generation #5470

merged 9 commits into from
May 18, 2022

Conversation

amahussein
Copy link
Collaborator

@amahussein amahussein commented May 12, 2022 •
edited
Loading

Signed-off-by: Ahmed Hussein (amahussein) a@ahussein.me

fixes #5176

The code changes generate html/js static content which the user can load into their browser.

How it works:

  • Static HTML/JS files are added to the repo in tools/src/main/resources/ui
  • After analyzing all the application in the eventLogs, the Qualification.qualifyApps would launch the UI report generator.
  • First, the static files are going to be copied to the report output folder
  • Second, the case classes will be serialized to JSON and written as JS variables.
  • When the user loads the HTML/index.html file, the JS will be read and rendered in the viewer.

How to use the feature

  • The feature is disabled by default. To generate UI-report add --ui-enabled to the command-line. For example 
    java -cp $CLASS_PATH com.nvidia.spark.rapids.tool.qualification.QualificationMain \
         --ui-enabled \
         --output-directory \
        qualification-output spark-events
  • Run the qualificationTool in the CLI, then load output-folder/ui-report/html/index.html in your folder

Tools used:

  • Bootstrap4
  • Datatables plugin
  • JQuery and Mustache.

@amahussein
QualUI. add notice file to resources [skip ci]
a1eb3a4 
Signed-off-by: Ahmed Hussein (amahussein) <a@ahussein.me>
@amahussein amahussein added this to the May 2 - May 20 milestone May 12, 2022
@amahussein amahussein self-assigned this May 12, 2022
@amahussein
Copy link
Collaborator Author

updated list of dependencies:

The folder structure after downloading the assets is shown below:

assets
    bootstrap
          bootstrap.bundle.min.js
          bootstrap.min.css
    datatables
          buttons.bootstrap4.min.css
          buttons.bootstrap4.min.js
          buttons.html5.min.js
          dataTables.bootstrap4.min.css
          dataTables.bootstrap4.min.js
          dataTables.buttons.min.js
          dataTables.responsive.min.js
          dataTables.searchPanes.min.js
          dataTables.select.min.js
          jquery.dataTables.min.js
          responsive.bootstrap4.min.css
          responsive.bootstrap4.min.js
          searchPanes.bootstrap4.min.css
          searchPanes.bootstrap4.min.js
          select.bootstrap4.min.css
    jquery
          jquery.min.js
          LICENSE.txt
    mustache-js
          LICENSE
          mustache.min.js
    spur
        LICENSE
        spur.min.css

@amahussein
QualUI. pull UI dependencies during build [skip ci]
51ddde8 
Signed-off-by: Ahmed Hussein (amahussein) <a@ahussein.me>
@amahussein
ui-fix. add disclaimer and config app-details-view
a414199 
Signed-off-by: Ahmed Hussein (amahussein) <a@ahussein.me>

1- add disclaimer at top of page
2- add conf to enable/disable app view details. Default is false
3- Matt requests
	Change "Total Speed-up" to "Estimated Speed-up" and round to 1 digit after the decimal
	GPU Opportunity description typo: "speed-up" should be "sped up" or even change to "accelerated"
	add disclaimer at top of dashboard page: "Disclaimer: projections are based on TPC-DS benchmark queries run using Spark 3.X.  Additionally, estimates are given assuming supported data formats and expressions are used in the application."

4- qualUI. generate js file
5- qualUI. add uiEnabledArgument
6- qualUI. add qualArgs ui enabled by default
7- qualUI. remove appInoRecords from UI usage
8- qualUI. change ui config and gitignore
9- qualUI. cleanup code
@amahussein amahussein marked this pull request as ready for review May 14, 2022 05:09
@amahussein
Copy link
Collaborator Author

build

@amahussein amahussein changed the title [WIP] QualUI. support UI code generation QualUI. support UI code generation May 14, 2022
tgravescs
tgravescs reviewed May 16, 2022
View reviewed changes
tools/pom.xml Outdated Show resolved Hide resolved
tools/prepare-ui-libraries.xml Outdated Show resolved Hide resolved
tools/prepare-ui-libraries.xml Show resolved Hide resolved
tools/src/main/resources/ui/NOTICE Outdated Show resolved Hide resolved
tools/src/main/resources/ui/html/index.html Outdated Show resolved Hide resolved
tools/src/main/resources/ui/html/raw.html Outdated Show resolved Hide resolved
tools/src/main/resources/ui/html/raw.html Outdated Show resolved Hide resolved
tools/src/main/resources/ui/js/ui-config.js Outdated Show resolved Hide resolved
tools/src/main/scala/org/apache/spark/sql/rapids/tool/ui/QualificationReportGenerator.scala Outdated Show resolved Hide resolved
tools/src/main/scala/org/apache/spark/sql/rapids/tool/ui/QualificationReportGenerator.scala Outdated Show resolved Hide resolved
@amahussein
fix notice/license and address PR comments
a2d794a 
Signed-off-by: Ahmed Hussein (amahussein) <a@ahussein.me>
@amahussein
remove comment block from js code
8df31b0 
Signed-off-by: Ahmed Hussein (amahussein) <a@ahussein.me>
@amahussein
Copy link
Collaborator Author

build

@amahussein
checksum verification of ui dowanloaded files
a966f9a 
Signed-off-by: Ahmed Hussein (amahussein) <a@ahussein.me>
@amahussein
Copy link
Collaborator Author

build

@abellina abellina self-requested a review May 17, 2022 13:29
@amahussein
modify license file
58ab410 
Signed-off-by: Ahmed Hussein (amahussein) <a@ahussein.me>
@amahussein
modify license file-2
aea8eed 
Signed-off-by: Ahmed Hussein (amahussein) <a@ahussein.me>
@tgravescs tgravescs changed the title QualUI. support UI code generation Qualification tool support UI code generation May 17, 2022
@amahussein
add license to jar and address cosmetic UI comments
5daec45 
Signed-off-by: Ahmed Hussein (amahussein) <a@ahussein.me>
@amahussein
Copy link
Collaborator Author

build

1 similar comment
@amahussein
Copy link
Collaborator Author

build

@amahussein amahussein mentioned this pull request May 17, 2022
Closed
@tgravescs tgravescs merged commit 6a1fa70 into NVIDIA:branch-22.06 May 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mattahrens mattahrens mattahrens left review comments

@nartal1 nartal1 nartal1 left review comments

@tgravescs tgravescs tgravescs approved these changes

@jlowe jlowe Awaiting requested review from jlowe jlowe is a code owner

@revans2 revans2 Awaiting requested review from revans2 revans2 is a code owner

@GaryShen2008 GaryShen2008 Awaiting requested review from GaryShen2008 GaryShen2008 is a code owner

@NvTimLiu NvTimLiu Awaiting requested review from NvTimLiu NvTimLiu is a code owner

@abellina abellina Awaiting requested review from abellina

Assignees

@amahussein amahussein

Labels
tools
Projects
None yet
Milestone
May 2 - May 20
Development

Successfully merging this pull request may close these issues.

[FEA] Qualification tool UI
4 participants
@amahussein @tgravescs @mattahrens @nartal1