Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add the error string code to analytics [CC-786] #1895

Merged
merged 1 commit into from
May 11, 2021

Conversation

teodora-sandu
Copy link
Contributor

@teodora-sandu teodora-sandu commented May 7, 2021

What does this PR do?

This PR defines error string codes for IaC errors and includes them in the analytics, to improve error readability.

Where should the reviewer start?

  • src/cli/commands/test/iac-local-execution/error-utils.ts: new utility function for creating the error string code from the error type
  • src/cli/commands/test/iac-local-execution/types.ts: comment letting people know if they add a new error, the name of the error will be used for error string codes
  • src/cli/commands/test/index.ts and src/cli/index.ts: code for adding the new field to the analytics
  • the rest is adding the strCode field to IaC custom errors using the utility function from src/cli/commands/test/iac-local-execution/error-utils.ts

How should this be manually tested?

  1. Run npm run build
  2. Run a failing scan: node ./dist/cli iac test test/fixtures/iac/kubernetes/pod-invalid.yaml --experimental -d
  3. Notice the error-str-code being included under the metadata field.

Any background context you want to provide?

What are the relevant tickets?

Screenshots

Screenshot 2021-05-07 at 11 48 02

@github-actions
Copy link
Contributor

github-actions bot commented May 7, 2021

Messages
📖

This PR will not trigger a new version. It doesn't include any commit message with feat or fix.

Generated by 🚫 dangerJS against 5dc0140

@teodora-sandu teodora-sandu marked this pull request as ready for review May 7, 2021 12:56
@teodora-sandu teodora-sandu requested a review from a team as a code owner May 7, 2021 12:56
@teodora-sandu teodora-sandu requested a review from a team May 7, 2021 12:56
@teodora-sandu teodora-sandu requested a review from a team as a code owner May 7, 2021 12:56
package.json Outdated Show resolved Hide resolved
Copy link
Contributor

@ipapast ipapast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love it! Left a non-blocking comment for the constant case.

@teodora-sandu teodora-sandu force-pushed the feat/add-error-string-code-analytics branch 2 times, most recently from 2ab09bf to aad186d Compare May 7, 2021 14:36
@teodora-sandu teodora-sandu self-assigned this May 7, 2021
@teodora-sandu teodora-sandu force-pushed the feat/add-error-string-code-analytics branch from aad186d to dfae9e7 Compare May 7, 2021 15:00
@teodora-sandu teodora-sandu changed the title feat: add the error string code to analytics feat: add the error string code to analytics [CC-786] May 7, 2021
@teodora-sandu teodora-sandu force-pushed the feat/add-error-string-code-analytics branch from dfae9e7 to 834e17d Compare May 7, 2021 15:57
@teodora-sandu teodora-sandu changed the title feat: add the error string code to analytics [CC-786] chore: add the error string code to analytics [CC-786] May 7, 2021
Copy link
Contributor

@jan-stehlik jan-stehlik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @teodora-sandu , nice work 🎉 I have 2 smaller non-blocking comments below. I also have a bigger comment/question - I think we are missing tests for this entire implementation? what if someone horribly modifies the behaviour, would any automated test safeguard against that?

const upperCase = require('lodash.uppercase');

export function getErrorStringCode(code) {
return upperCase(IaCErrorCodes[code]).replace(/ /g, '_');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a huge fan of introducing 3rd party library into our already polluted repo. From what I understand we need a way to have value in format MISSING_REQUIRED_FIELDS_IN_KUBERNETES_YAML_ERROR. I think we have 2 decent options:

  1. create enum with keys in upper case delimetered by underscore in types.ts denoted with capital letters as is convention for constants and keep the original enum IaCErrorCodes
export enum IAC_ERROR_CODES {
  MISSING_REQUIRED_FIELDS_IN_KUBERNETES_YAML_ERROR = 1031,
  // ...
}
  1. create new enum just like above and delete original one. Make sure to replace all the occurences. This seems like a better option but is more laborious

The reason why i'm raising this is because dependencies in snyk cli got extremely out of hand (not to mention performance) and we got into habit to install library for any small job that can be easily written by us.

This is, however, not a blocking comment, just imo opportunity to improve a solution

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reasoning behind using this getErrorStringCode function is that we don't want to duplicate the work to add a new error (i.e. have to add an error code to IaCErrorCodes and a new error string code to some other new enum). And we don't want to remove the IaCErrorCodes enum because we want to have both error codes and readable error string codes. This function allows us to create new errors much more easily and not have to think about what error string code to give it, typos, etc.

The added package is pretty small https://bundlephobia.com/result?p=lodash.uppercase@4.3.0 in my opinion, as it only exposes the lodash _.upperCase method. So while we could have done this without this new package, I would say the benefits outweigh the added bundle size. But let me know if you disagree

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've re-implemented this function using built-in JavaScript methods, so we're not importing lodash.uppercase anymore 😃

Comment on lines 162 to +163
analytics.add('error-code', error.code);
analytics.add('error-str-code', error.strCode);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: I wonder if we can standardise the way to submit analytics keys. We use camelCase in some places, snake_case in others and lowercase-delimitered-with-dash here. I would prefer if we used standardised approach everywhere (ideally camelCase or snake_case rather than inventing new case). What do you think?

Suggested change
analytics.add('error-code', error.code);
analytics.add('error-str-code', error.strCode);
analytics.add('errorCode', error.code);
analytics.add('errorStringCode', error.strCode);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this would require cooperation from multiple teams, as these fields might be used either by their Grafana dashboards or other dashboards/analytics to capture trends. I agree that it'd be better if we standardised this, but probably needs a wider discussion and is not a change that belongs to this PR

@teodora-sandu teodora-sandu force-pushed the feat/add-error-string-code-analytics branch from 834e17d to 70e6617 Compare May 11, 2021 08:04
@teodora-sandu
Copy link
Contributor Author

teodora-sandu commented May 11, 2021

Thank you for your comprehensive, in-depth review @jan-stehlik! I've added two unit tests for the new getErrorStringCode function.

@teodora-sandu teodora-sandu force-pushed the feat/add-error-string-code-analytics branch from 70e6617 to 5dc0140 Compare May 11, 2021 08:20
Copy link
Contributor

@jan-stehlik jan-stehlik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice work @teodora-sandu 💯

import { getErrorStringCode } from '../../../../src/cli/commands/test/iac-local-execution/error-utils';
import { IaCErrorCodes } from '../../../../src/cli/commands/test/iac-local-execution/types';

describe('getErrorStringCode', () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😍

@teodora-sandu teodora-sandu merged commit 93624dc into master May 11, 2021
@teodora-sandu teodora-sandu deleted the feat/add-error-string-code-analytics branch May 11, 2021 09:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants