-
Notifications
You must be signed in to change notification settings - Fork 720
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Question about the visualization of the feature importance. #491
Comments
It would be good to be able to configure this, but it's not something we currently support. You can however get the local explanation values and plot them however you like. To do this, use predict_and_contrib: |
|
What was the chaos in plotly? I think the main reason was because you get scroll bars in Jupyter Notebook if your cell is too big and, well, it's unreasonable to show thousands of terms if your model has that many, so they need to be clipped at some point. But I didn't write the UI, so @Harsha-Nori and @nopdive would know more. |
Thanks for your reply! Now I am trying to train an EBM with 20-25 features (including pairwise features). The problem displayed may be caused by my lack of familiarity with ploty package. I think it can be solved. |
Hi @JWKKWJ123 - You can also kind of hack the UI to show you what you want by simply removing the first 15 terms from the model, which will then show you the next 15 terms if you visualize it afterwards. Just beware that the predicted value shown in the UI will no longer be valid since the model has been edited. |
Hi @paulbkoch , |
Hi @JWKKWJ123 - have you made any progress on this? - I've got a model with over 100 terms and would like to visualise the global importances (preferably via plotly). I know I can access the FI values via
But was wondering if there's a nicer way (especially to have it be the same style as the local explanations). |
It's possible to get more than 15 terms displayed by changing the top_n value here:
But there's a catch. This change doesn't increase the vertical height of the iframe in which it resides, so if you make top_n too large, the terms will be scrunched together. It's possible to zoom into regions though by dragging the mouse over the area you're interested in. |
|
Dear Paul, |
"min_max" is something simpler. If you're looking at the graphs visually then "min_max" is the vertical difference between the highest point on the graph and the lowest point on the graph. The "avg_weight" importance of a feature/term can be calculated by looking up the contribution value on the feature/term's graph for each sample in the training set, then taking the absolute values, then averaging those. If you were to look at our code that does this calculation, you'd see that instead of iterating over all the samples, we use an equivalent method that leverages the bin weights that we preserve in the model. This has the advantage that we don't need access to the original training set to calculate it. Since "min_max" and "avg_weight" are different metrics, their orderings will not be identical. It is possible to imagine many other ways you might want to measure feature/term importances. Another one that we'll probably add at some point is the change in metrics like log loss when you remove each individual feature from the model. This will require adding an "X" parameter to the function since we'll then need access to a dataset. This would also add utility in terms of calculating importances on test sets, etc.. Averaging the feature/term importances across folds should work. If the folds have different sample weights, you might want to take the weighted average. |
Thank you so much for the detailed reply. I looked at our code that does this calculation. If I understand correctly, "min_max" is the maximum contribution of each feature in all samples (on training set) minus the minimum contribution (no absolute value is taken). If a feature have high contribution to a certain class for all the samples, then the ‘max – min’ will be a small number, is my understanding correct? |
Yes, your understanding is correct. "min_max" is a better indicator of extreme contributions from a small number of samples vs "avg_weight". Both of these metrics have their place, but I'd generally recommend using "avg_weight" unless have a specific reason to use "min_max". There are a lot of ways to handle the pairwise disagreement issues. With interactions there are sort of two ways to think about them in the context of having multiple models. One way to think about the problem is to say that if a pair is present in one model, but not the other, then it's essentially present in both models but has a contribution value of zero in the model where it is missing. Another way to think about the problem is to understand that the "interaction=10" parameter in the EBM constructor is merely a threshold and the pairwise interaction should really be in both models but just didn't make the cut to be included in the model where it is missing. We have the same issue in the implementation of the merge_ebms function where we have to decide how to merge two EBM models with pair disagreements. Right now merge_ebms assumes a contribution of zero, but I plan to add a blending option that will allow the caller to choose which assumption they want. Internally in InterpretML we have another similar issue when constructing the outer_bags since the pairs can disagree there too. We handled it by first building the mains model, then we measure the pair strengths within each outer bag and come to a consensus set of pairs, then we continue boosting on the consensus set of pairs. It is possible today to replicate this process yourself, although it's a more advanced process that we hope to simplify in the future. Our documentation has an example of how you would do this: https://interpret.ml/docs/python/examples/custom-interactions.html I'm a little unclear on what you're asking regarding the pairwise term calculations. If you're asking how we choose which pairs to include in the model, the answer is that we continue to use the FAST algorithm from this paper: If you're asking how we calculate the pair importances, the code that performs that calculation is here. It works for both mains and pairs: interpret/python/interpret-core/interpret/glassbox/_ebm/_ebm.py Lines 1819 to 1833 in 45ee7a8
And if you wanted to boil it down, this is the most critical line of that process. For interactions, mean_abs_score and self.bin_weights_[i] are both tensors where the number of dimensions equals the number of features within the interaction: interpret/python/interpret-core/interpret/glassbox/_ebm/_ebm.py Lines 1826 to 1828 in 45ee7a8
|
Thank you very much for the detailed reply! |
Hi Paul, |
Hi @JWKKWJ123 -- Can you post some code showing how you're doing these averages and making the folds. Doing these kinds of averages and comparing importances values and/or ordering them is really going to depend on the minutia of how these calculations are made. |
Hi Paul,
|
Looks reasonable to me. |
Hi all,
When I used the visualize() to output the global/local explanations, I found that the visualize() can only consider up to 15 features (including pairwise features). I didn't find a parameter that I can edit in the visualize() function. I would like to ask is there a way to draw a plot include more than 15 features using interpretml?
Thanks!
The text was updated successfully, but these errors were encountered: