-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Model card creation #181
Merged
annmary-roy
merged 35 commits into
HewlettPackard:master
from
varkha-d-sharma:model_card
Jul 17, 2024
Merged
Model card creation #181
Changes from 33 commits
Commits
Show all changes
35 commits
Select commit
Hold shift + click to select a range
c98e011
created popup component
886117d
adding popup component
e5fb666
added rendering code
2913ef3
updating popup component
c40f3c2
Added close and download button
0a11f80
Deleted cmf file
a513b6c
updating model_card
197a556
Merge branch 'model_card' of https://github.com/varkha-d-sharma/cmf i…
148c01e
Wrote an rest api for model card and axios interface
varkha-d-sharma f6fe3a2
model card api update
b89b544
json filename update
AyeshaSanadi 0f669ba
readded popup
c3169fa
adding more changes required for model card creation
varkha-d-sharma 838902c
added some json related changes to do some testing on popup side
varkha-d-sharma c3d0fbc
made some changes
varkha-d-sharma 1f0236c
changes related to popup content
AyeshaSanadi ca80640
updated code for json filename
AyeshaSanadi 4dc75ac
made some changes to display data properly in popup
varkha-d-sharma 81e956c
Align items inside popup component to left
AyeshaSanadi d3d6b98
Made some changes in popup data fromat
varkha-d-sharma 55a0dd7
Added table structure for artifact data
AyeshaSanadi 434b8f2
added style to table
AyeshaSanadi b14ce89
adding some code for popup
varkha-d-sharma 6976f7b
made changes in table code
AyeshaSanadi 9b3a7ad
keeping old_index.jsx file
AyeshaSanadi 018372c
converted table to model card
AyeshaSanadi 216b133
Added the code to refine popup model card
varkha-d-sharma 1a02682
Made changes regarding modelcard structure
AyeshaSanadi 638d4a4
mistakenly commited wrong user name
varkha-d-sharma a83f06e
addressing review comments
varkha-d-sharma 89e3389
Addressed review comments and commented on get_model_data function
AyeshaSanadi 9a8953b
Updating supported versions from <=3.10 to <3.11. So that, all the ve…
AyeshaSanadi 8b4d338
Made proper alignment of close and download button
AyeshaSanadi 53b2852
Merge branch 'HewlettPackard:master' into model_card
varkha-d-sharma d84845b
Addressing review comments
varkha-d-sharma File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -6,6 +6,75 @@ | |
from server.app.query_visualization_execution import query_visualization_execution | ||
from fastapi.responses import FileResponse | ||
|
||
|
||
async def get_model_data(mlmdfilepath, modelId): | ||
''' | ||
This function retrieves the necessary model data required for generating a model card. | ||
|
||
Arguments: | ||
mlmdfilepath (str): The file path to the metadata. | ||
modelId (int): The ID of the model for which data is required. | ||
|
||
Returns: | ||
This function returns a tuple of DataFrames containing the following: | ||
|
||
model_data_df (DataFrame): Metadata related to the model itself. | ||
model_exe_df (DataFrame): Metadata of the executions in which the specified modelId was an input or output. | ||
model_input_df (DataFrame): Metadata of input artifacts that led to the creation of the model. | ||
model_output_df (DataFrame): Metadata of artifacts that used the model as an input. | ||
The returned DataFrames provide comprehensive metadata for the specified model, aiding in the creation of detailed and accurate model cards. | ||
''' | ||
query = cmfquery.CmfQuery(mlmdfilepath) | ||
pd.set_option('display.max_columns', None) | ||
df = pd.DataFrame() | ||
# get artifact info - we may remove this to remove redundancy as we can get from the input only | ||
model_data_df = pd.DataFrame() | ||
model_exe_df = pd.DataFrame() | ||
model_input_df = pd.DataFrame() | ||
model_output_df = pd.DataFrame() | ||
|
||
# get name from id | ||
modelName = "" | ||
df = query.get_all_artifacts_by_ids_list([modelId]) | ||
modelType = df['type'].tolist()[0] | ||
if modelType == "Model": | ||
modelName = df['name'].tolist()[0] | ||
if modelName == "": | ||
return model_data_df, model_exe_df, model_input_df, model_output_df | ||
else: | ||
return model_data_df, model_exe_df, model_input_df, model_output_df | ||
|
||
# model's own data | ||
model_data_df = query.get_artifact(modelName) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. why two calls ..we should have got all this in the first call |
||
|
||
# model's executions data with props and custom props | ||
exe_df = query.get_all_executions_for_artifact(modelName) | ||
exe_ids = [] | ||
if not exe_df.empty: | ||
exe_df.drop(columns=['execution_type_name', 'execution_name'], inplace=True) | ||
exe_ids = exe_df['execution_id'].tolist() | ||
|
||
|
||
if not exe_ids: | ||
return model_data_df, model_exe_df, model_input_df, model_output_df | ||
model_exe_df = query.get_all_executions_by_ids_list(exe_ids) | ||
model_exe_df.drop(columns=['Python_Env', 'Git_Start_Commit', 'Git_End_Commit'], inplace=True) | ||
|
||
in_art_ids = [] | ||
# input artifacts | ||
in_art_ids.extend(query._get_input_artifacts(exe_ids)) | ||
if modelId in in_art_ids: | ||
in_art_ids.remove(modelId) | ||
model_input_df = query.get_all_artifacts_by_ids_list(in_art_ids) | ||
|
||
out_art_ids = [] | ||
# output artifacts | ||
out_art_ids.extend(query._get_output_artifacts(exe_ids)) | ||
out_art_ids.remove(modelId) | ||
model_output_df = query.get_all_artifacts_by_ids_list(out_art_ids) | ||
|
||
return model_data_df, model_exe_df, model_input_df, model_output_df | ||
|
||
async def get_executions_by_ids(mlmdfilepath, pipeline_name, exe_ids): | ||
''' | ||
Args: | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why two return statements here ?