Split httpconv and netconv by signal

[MrAlias]

The attributes required or recommended for http/net depend on the signal. Currently there are only one set of attributes returned for a situation for all signals.

Action Items

• Develop a design for the split here
• Implement in code

[MrAlias]

Related open-telemetry/opentelemetry-specification#2818

[MrAlias]

• Document the existing httpconv and netconv packages are for tracing only: Document httpconv and netconv packages as trace providing only trace semantic convention #3823
  • v1.13.0/httpconv
  • v1.13.0/netconv
  • v1.14.0/httpconv
  • v1.14.0/netconv
  • v1.15.0/httpconv
  • v1.15.0/netconv
  • v1.16.0/httpconv
  • v1.16.0/netconv
  • v1.17.0/httpconv
  • v1.17.0/netconv
  • v1.18.0/httpconv
  • v1.18.0/netconv
• Generation httptraceconv, httpmetricconv, and nettraceconv for all new semantic convention packages
  • Add internal generation of metric semantic conventions for HTTP
  • Update generation code to create httptraceconv, httpmetricconv, and nettraceconv for new semconv packages
• Restore http.target from ServerRequest, reverted in Remove http.target attr from ServerRequest #3687
  • Once it is clear that the ServerRequest from httptraceconv should not be used for metrics and there is an appropriate alternative, the vulnerability that PR addressed should not be relevant
  • The revert should probably not be back-ported to the semconv/v1.{13...18}.0 though give there isn't a viable alternative for metrics that is also going to be back-ported

[Petrie]

• Generation httptraceconv, httpmetricconv, and nettraceconv for all new semantic convention packages

  • Add internal generation of metric semantic conventions for HTTP
  • Update generation code to create httptraceconv, httpmetricconv, and nettraceconv for new semconv packages

Hi @MrAlias, have you begun working on this task yet? If not, I would be delighted to work on it.
While it may be difficult for me, I think I grasp the mission.

[MrAlias]

I have started.

[MrAlias]

@Petrie I am getting pulled off of this to work on the metric API. If you still have time to work on the task you mentioned I can assign this to you instead?

[Petrie]

Sure, thanks. I'll take care of it. ❤️

[Petrie]

Here is a prototype related to task two in #3744 (comment) for release v1.19.0
I have some questions about the prototype that I'd like to confirm

• Do I understand the main objective of this problem?
• Are there any major issues with the prototype ?
• In method ServerRequestMetric,ServerActiveRequestsMetric, is it properly server, route, code pass by parameter?
• Is the passing of the server, route, and code parameters to the ServerRequestMetric, ServerActiveRequestsMetric method correct?

Extend semconv/internal/v2/http.go by adding three new methods

1. Method ServerRequestMetric

   func (c *HTTPConv) ServerRequestMetric(server string, route string, code int, req *http.Request) []attribute.KeyValue {}

   According to the metric http.yaml metric.http.server.duration, metric.http.server.request.size and metric.http.server.response.size metric share the same ref attributes( http.method, http.status_code, http.flavor) and extend attributes (http.scheme, http.route, net.host.name, net.host.port). As a result, all these metrics can use the ServerRequestMetric method for attributes.

2. Method ServerActiveRequestsMetric
   This method for metric http.server.active_requests

   func (c *HTTPConv) ServerActiveRequestsMetric(server string, code int, req *http.Request) []attribute.KeyValue {}

3. Method ClientRequestMetric

   func (c *HTTPConv) ClientRequestMetric(req *http.Request) []attribute.KeyValue {}

   According to the metric http.yaml metric.http.client.duration, metric.http.client.request.size and metric.http.client.response.size share the same ref attributes (http.method, http.status_code, http.flavor,net.sock.peer.addr) and extend attributes (net.peer.name, net.peer.port). As a result, all these metrics can use the ClientRequestMetric method for attributes.

Add three function to internal/tools/.../httpmetricconv/http.go.tmpl

These three functions are simply invoking the relevant methods for the HTTPConv struct in the go.opentelemetry.io/otel/semconv/internal/v2 package, similar to how other functions work.

func ServerMetric(server string, route string, code int, req *http.Request) []attribute.KeyValue {}
func ServerActiveRequestsMetric(server string, code int, req *http.Request) []attribute.KeyValue {}
func ClientMetric(req *http.Request) []attribute.KeyValue {}

Generate httptraceconv, httpmetricconv, and nettraceconv

1. Create a new file http.go.tmpl in the internal/tools/semconvkit/templates/httpmetricconv directory to generate the httpmetricconv .
2. To generate the httptraceconv, rename the httpconv directory to httptraceconv in the internal/tools/semconvkit/templates directory.
3. To generate the nettraceconv tool, rename the netconv directory to nettraceconv in the internal/tools/semconvkit/templates directory.
4. Extend the functionality of the semconvkit tool by implementing support for generating httpmetricconv.

Eagerly await guidance. @MrAlias

[MrAlias]

@Petrie heres a few ideas:

• The metric semantic conventions not only specify the attributes for metrics but also the metrics themselves (name, unit, description, type). I was thinking about offering functions to create these metrics appropriately and functions to provide the correct attributes.
• There are multiple metrics in the semantic conventions for both clients and servers. You split out the active requests metric, but all the other metric attributes will need to be tailored to the different requests.
  • Given there is going to be a lot of repeated code here if each function discovers their own attributes each time, it is worth exploring how to generate all attributes with a function that also can be filtered. That or maybe some composable runner for attributes.
• It might make more sense to start the internal work in a new v3 given there will backwards incompatible changes re-introduced to the tracing portion (adding back in the full URL to server request attributes).
• This should not update any of the existing semconv packages (at least not now). It will be used for the next semconv package release.
  • I would use the generation tool for an existing semconv package just to check this will work for v1.19.0.

[Petrie]

• There are multiple metrics in the semantic conventions for both clients and servers. You split out the active requests metric, but all the other metric attributes will need to be tailored to the different requests.
  • Given there is going to be a lot of repeated code here if each function discovers their own attributes each time, it is worth exploring how to generate all attributes with a function that also can be filtered. That or maybe some composable runner for attributes.

There are only three methods for attribute composition:

func (c *HTTPConv) serverRequestMetric(req *http.Request, server string, route string, code int) []attribute.KeyValue {}
func (c *HTTPConv) clientRequestMetric(req *http.Request) []attribute.KeyValue {}
func (c *HTTPConv) ServerActiveRequestsMetric(req *http.Request, server string, code int) []attribute.KeyValue {}

• Metric metric.http.server.duration, metric.http.server.request.size and metric.http.server.response.size will share method serverRequestMetric.
• Metric metric.http.client.duration, metric.http.client.request.size and metric.http.client.response.size will share method clientRequestMetric

Is it necessary for us to investigate the use of a composable runner for attributes ?
@MrAlias