[FEA] Logging in libcudf

[devavret]

Certain libcudf functions need to send warnings to users.
Examples:

1. [BUG] Fixed-point types with precision < 10 (for Spark) cannot be successfully read back from parquet #7152 : We deviate from typical parquet writer behaviour and as a result the written file is unreadable by certain other libraries. We'd prefer to warn the user about it but not throw an exception. [BUG] Fixed-point types with precision < 10 (for Spark) cannot be successfully read back from parquet #7152 (comment)
2. We want to warn the user when despite specifying, GDS is not available to be used Use cuFile for Parquet IO when available #7444 (comment)

These warnings can be part of a log and in general libcudf can have logging similar to RMM.

[jlowe]

Similar to rapidsai/rmm#564 I'd like to see the ability for the application to provide a logging sink to better control the handling of any log messages. Emitting uncontrollable messages to stdout or creating files in the current directory will be problematic for some use-cases. Providing a log sink adapter or some similar interface allows the libcudf log messages to be funneled to any existing logging mechanism the app may be using.

[github-actions]

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

[github-actions]

This issue has been labeled inactive-90d due to no recent activity in the past 90 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

[vuule]

Some new potential uses in cuIO:
warn on fallback to host JSON tree traversal
warn on compression failure in Parque and ORC writers
info about nvCOMP use (vs internal implementations)
info about KvikIO use (vs internal implementations)
info about cuFile use

[vuule]

Finally revisiting this :)

Initial design thoughts:

• Use spdlog library (header-only).
• Have a global logger that's initialized on first use.
• Default logging level is WARNING (or maybe OFF initially).
• Provide a way to customize logging level and sink. One or more of the following:
  • At build time - CMake flags to set level (maybe also sink, e.g. file vs stdout);
  • During intialization - environment variables, again can control level and basic sink options.
  • At runtime (any point) - libcudf API that allows more control, like providing custom sink(s).

@jlowe is a libcudf API what you had in mind in your request? It would need to be called before any other libcudf functions to avoid potentially logging with default settings.

[vyasr]

Just a note, spdlog can be used header-only or precompiled. Given that we use it header-only in rmm since rmm itself is header-only, it may not help compile times in libcudf to use its compiled mode here, but if we do go the spdlog route it's worth checking on the compile times before @davidwendt comes after you 😄

[GregoryKimball]

Now that we have merged #12637, I'd like to close this issue and continue the discussion for additional logging items in new issue(s).

[vuule]

@GregoryKimball regarding the additional logging items: I opened a PR for cuIO warnings that are not related to integrations #13043. cuIO ntegrations should be covered as well in 23.06.