Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update logging configuration for better debug and error tracking #3260

Merged
merged 8 commits into from
Jul 17, 2024

Conversation

RafaelJohn9
Copy link
Contributor

@RafaelJohn9 RafaelJohn9 commented Jun 21, 2024

Fix #3192

  • Added rotating file handlers for Django and Pontoon loggers

  • Console logging now uses verbose format in DEBUG mode

  • Improved log formatting for clarity and consistency

  • added *.log in .gitignore to prevent log files from being pushed

ruff.toml Outdated
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please revert this change in permission (not sure if you meant to commit something else, given the commit subject).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh yea sorry about that, I was trying to fix the ruff format style in /pontoon/settings/base.py
but sadly I have confilicting views

github actions pylint ruff says my ruff format has conflicts on the file

but locally I have this
image

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you running ruff through make? If not, what's the version of ruff?

I checked locally with 0.4.7, and I see the same error as the CI

diff --git a/pontoon/settings/base.py b/pontoon/settings/base.py
index 03c629d8..fd419406 100644
--- a/pontoon/settings/base.py
+++ b/pontoon/settings/base.py
@@ -764,14 +764,14 @@ LOGGING = {
         },
         "django_file": {
             "class": "logging.handlers.RotatingFileHandler",
-            "filename": os.path.join(path('django_debug.log')),
+            "filename": os.path.join(path("django_debug.log")),
             "maxBytes": 1024 * 1024 * 2,  # 2 MB
             "backupCount": 3,
             "formatter": "verbose" if DEBUG else "simple",
         },
         "pontoon_file": {
             "class": "logging.handlers.RotatingFileHandler",
-            "filename": os.path.join(path('pontoon_debug.log')),
+            "filename": os.path.join(path("pontoon_debug.log")),
             "maxBytes": 1024 * 1024 * 2,  # 2 MB
             "backupCount": 3,
             "formatter": "verbose" if DEBUG else "simple",

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No am not running it through make, my ruff version is 0.4.6

@RafaelJohn9
Copy link
Contributor Author

@flodolo found what the fuss was about and I have fixed it

Copy link
Collaborator

@mathjazz mathjazz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the patch! Nice work!

I'd like the logging settings to remain exactly the same as they are ATM.

All the changes (logging to a file, logging format) should be optional and controlled via environment variables.

pontoon/settings/base.py Show resolved Hide resolved
@RafaelJohn9
Copy link
Contributor Author

Thanks for the patch! Nice work!

I'd like the logging settings to remain exactly the same as they are ATM.

All the changes (logging to a file, logging format) should be optional and controlled via environment variables.

sure thing I'll update that

@RafaelJohn9
Copy link
Contributor Author

hey @mathjazz , I have updated the PR, sorry for the late update.

@mathjazz
Copy link
Collaborator

mathjazz commented Jul 3, 2024

Thanks for the update!

Logging is apparently still more verbose than on main, e.g. I see the following message in the logs:

[server] [DEBUG:django.db.backends] 2024-07-03 10:24:58,130 (0.000) SELECT "base_project"."id", "base_project"."total_strings", "base_project"."approved_strings", "base_project"."pretranslated_strings", "base_project"."strings_with_errors", "base_project"."strings_with_warnings", "base_project"."unreviewed_strings", "base_project"."name", "base_project"."slug", "base_project"."data_source", "base_project"."can_be_requested", "base_project"."configuration_file", "base_project"."disabled", "base_project"."date_created", "base_project"."date_disabled", "base_project"."sync_disabled", "base_project"."system_project", "base_project"."visibility", "base_project"."langpack_url", "base_project"."info", "base_project"."deadline", "base_project"."priority", "base_project"."contact_id", "base_project"."admin_notes", "base_project"."latest_translation_id", "base_project"."tags_enabled", "base_project"."pretranslation_enabled" FROM "base_project" WHERE "base_project"."id" IN (1) ORDER BY "base_project"."id" ASC; args=(1,); alias=default

Could we split this patch into two pieces? Namely:

  1. In this patch, fix Ability to log in a file #3192, i.e. only add ability to optionally log to a file.
  2. File a separate bug to enable more verbose logging, and hash out the motiviation for it in the issue first.

@RafaelJohn9
Copy link
Contributor Author

Thanks for the update!

Logging is apparently still more verbose than on main, e.g. I see the following message in the logs:

[server] [DEBUG:django.db.backends] 2024-07-03 10:24:58,130 (0.000) SELECT "base_project"."id", "base_project"."total_strings", "base_project"."approved_strings", "base_project"."pretranslated_strings", "base_project"."strings_with_errors", "base_project"."strings_with_warnings", "base_project"."unreviewed_strings", "base_project"."name", "base_project"."slug", "base_project"."data_source", "base_project"."can_be_requested", "base_project"."configuration_file", "base_project"."disabled", "base_project"."date_created", "base_project"."date_disabled", "base_project"."sync_disabled", "base_project"."system_project", "base_project"."visibility", "base_project"."langpack_url", "base_project"."info", "base_project"."deadline", "base_project"."priority", "base_project"."contact_id", "base_project"."admin_notes", "base_project"."latest_translation_id", "base_project"."tags_enabled", "base_project"."pretranslation_enabled" FROM "base_project" WHERE "base_project"."id" IN (1) ORDER BY "base_project"."id" ASC; args=(1,); alias=default

Could we split this patch into two pieces? Namely:

1. In this patch, fix [Ability to log in a file #3192](https://github.com/mozilla/pontoon/issues/3192), i.e. only add ability to optionally log to a file.

2. File a separate bug to enable more verbose logging, and hash out the motivation for it in the issue first.

Sure thing. Sorry about the more verbose logging, I'll remove the more verbose logging and focus on the task at hand

@RafaelJohn9
Copy link
Contributor Author

@mathjazz I have removed the simple, logging option, I have looked at the verbose logging options and I noticed it's currently the same as yours.

@RafaelJohn9
Copy link
Contributor Author

hello @mathjazz am really interested in this project, this PR is taking almost a month now 😅, if you don't mind could you give one of your social platforms you are active in so that we could talk more about how you want me to approach this issue, and hence a smooth workflow for future issues and PRs

pontoon/settings/base.py Outdated Show resolved Hide resolved
pontoon/settings/base.py Outdated Show resolved Hide resolved
pontoon/settings/base.py Outdated Show resolved Hide resolved
pontoon/settings/base.py Outdated Show resolved Hide resolved
@mathjazz
Copy link
Collaborator

mathjazz commented Jul 12, 2024

Hey @RafaelJohn9, thanks for your contribution and patience!

See the README for contact information (I'm mathjazz on chat.mozilla.org and other channels):
https://github.com/mozilla/pontoon?tab=readme-ov-file#contributing-to-pontoon

As mentioned earlier, #3192 is only about adding the ability to log to a file. There's still a few changes in this PR that are not needed for that.

As soon as this is addressed, we can ship this.

@RafaelJohn9
Copy link
Contributor Author

RafaelJohn9 commented Jul 12, 2024

Hey @RafaelJohn9, thanks for your contribution and patience!

See the README for contact information (I'm mathjazz on chat.mozilla.org and other channels):
https://github.com/mozilla/pontoon?tab=readme-ov-file#contributing-to-pontoon

As mentioned earlier, #3192 is only about adding the ability to log to a file. There's still a few changes in this PR that are not needed for that.

As soon as this is addressed, we can ship this.

@mathjazz
I'll contact you tomorrow if you are okay with it, to talk more about this

RafaelJohn9 added a commit to RafaelJohn9/pontoon that referenced this pull request Jul 13, 2024
@RafaelJohn9
Copy link
Contributor Author

I have corrected the changes that you reviewed above.

@mathjazz
Copy link
Collaborator

Thanks for the update.

Functionality looks good, please fix the linting error and we're done!

@RafaelJohn9
Copy link
Contributor Author

done, 👍
Learned a lot in this PR,
thanks @mathjazz for your patience, 🤝 means a lot.

@mathjazz
Copy link
Collaborator

mathjazz commented Jul 16, 2024

@RafaelJohn9 Could you please document the newly added environment variable:
https://github.com/mozilla/pontoon/blob/main/docs/admin/deployment.rst#environment-variables

Sorry I missed that earlier!

@RafaelJohn9
Copy link
Contributor Author

@mathjazz done, lemme know if there is anything else required

Copy link
Collaborator

@mathjazz mathjazz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updating the docs. I've pushed a minimal text style change directly.

Nice work with the patch, let's merge!

@mathjazz mathjazz merged commit 185a7e0 into mozilla:main Jul 17, 2024
2 checks passed
@RafaelJohn9 RafaelJohn9 deleted the error_logging branch July 17, 2024 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ability to log in a file
3 participants