Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Logs to OpenTelemetry vocabulary #91

Merged

Conversation

tigrannajaryan
Copy link
Member

This commit proposes the initial vocabulary for logs and event to be used across
OpenTelemetry project.

This is an initial proposal and we can extend / clarify the vocabulary as we go.

This commit proposes the initial vocabulary for logs and event to be used across
OpenTelemetry project.

This is an initial proposal and we can extend / clarify the vocabulary as we go.
Copy link
Member

@arminru arminru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's great that you take care of having consistent and well-defined terms right from the start! 👍

text/0091-logs-vocabulary.md Show resolved Hide resolved
text/0091-logs-vocabulary.md Outdated Show resolved Hide resolved

### Log Attributes

Key/value pairs contained in a Log Record.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd also expect "log attributes" to be metadata about a whole log file/stream (e.g. producer, host or service information, time span, size) rather than attributes on a single log record/entry.

Copy link

@jaredcnance jaredcnance Mar 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there are two classes of data which can be included in structured logs:

  • attributes: key/value pairs that are specific to a single log record and often have high cardinality values (e.g. timestamp)
  • metadata: key/value pairs that apply to one or more log records, generally constant for a given span (e.g. traceId, customerId, etc.) or may be constant for the lifetime of the process (e.g. host, containerId, etc.).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with the distinction.

The constants part of "metadata" is stored in a Resource in OpenTelemetry. There is already a definition of a Resource.

The non-const parts are Attributes of a log record. The fact that they can be identical for a batch of log records is a data modeling topic and does not belong to the vocabulary (at least not yet, until we have the data model). We can have further clarification of types of attributes, but for now I would refrain from adding anything to the vocabulary. It is best to discuss this in a Data Model proposal (which I believe we inevitably need to have).

(BTW, "metadata" is not a well defined term, IMO it is best to avoid using it in this discussion since people have different understanding of what metadata means).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with both of your comments.
My comment was about the term "log attributes" itself, which sounds like attributes that apply to the whole log file/stream (collection of logs) at once. These could therefore, for example, be expressed and stored in a Resource or preamble to a log file/stream rather than one single log record. I'd rather call the key/value pairs contained in a single log record "log record attributes" to avoid this ambiguity.

text/0091-logs-vocabulary.md Outdated Show resolved Hide resolved
text/0091-logs-vocabulary.md Outdated Show resolved Hide resolved
text/0091-logs-vocabulary.md Outdated Show resolved Hide resolved
Copy link
Member

@mtwo mtwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this, I approve modulo some of the changes that @arminru has requested

@tigrannajaryan tigrannajaryan force-pushed the feature/tigran/log-vocabulary branch 2 times, most recently from cda2ccf to 991ed1d Compare March 25, 2020 19:38
@tigrannajaryan
Copy link
Member Author

I also added Log with a recommendation not to use it without additional qualifiers to avoid confusion.

@bogdandrutu bogdandrutu merged commit 7208791 into open-telemetry:master Mar 27, 2020
@tigrannajaryan tigrannajaryan deleted the feature/tigran/log-vocabulary branch March 27, 2020 18:49
tigrannajaryan added a commit to tigrannajaryan/rfcs that referenced this pull request Apr 13, 2020
* Add Logs to OpenTelemetry vocabulary

This commit proposes the initial vocabulary for logs and event to be used across
OpenTelemetry project.

This is an initial proposal and we can extend / clarify the vocabulary as we go.

* Address PR comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants