I cannot make GraphQLInstrumentation produce spans

[emiwidp]

What version of OpenTelemetry are you using?

"@opentelemetry/api": "^1.4.1",
"@opentelemetry/auto-instrumentations-node": "^0.36.4",
"@opentelemetry/core": "^1.11.0",
"@opentelemetry/exporter-metrics-otlp-grpc": "^0.37.0",
"@opentelemetry/exporter-trace-otlp-grpc": "^0.37.0",
"@opentelemetry/exporter-trace-otlp-http": "^0.37.0",
"@opentelemetry/instrumentation": "^0.37.0",
"@opentelemetry/instrumentation-express": "^0.32.1",
"@opentelemetry/instrumentation-graphql": "^0.33.2",
"@opentelemetry/instrumentation-grpc": "^0.37.0",
"@opentelemetry/instrumentation-http": "^0.37.0",

What version of Node are you using?

16.16.0

What did you do?

I have an apollo server application written in typescript, using apollo server v.4.3.0. I have tried to register my instrumentation in the following way:

import { HttpInstrumentation } from "@opentelemetry/instrumentation-http";
import { BatchSpanProcessor, ConsoleSpanExporter, SimpleSpanProcessor } from "@opentelemetry/sdk-trace-base";
import { ExpressInstrumentation } from "@opentelemetry/instrumentation-express";
import { registerInstrumentations } from "@opentelemetry/instrumentation";
import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node";
import { GrpcInstrumentation } from "@opentelemetry/instrumentation-grpc"
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-grpc";
import { Resource } from "@opentelemetry/resources";
import { GraphQLInstrumentation } from "@opentelemetry/instrumentation-graphql";

const provider = new NodeTracerProvider({
  resource: Resource.default().merge(
    new Resource({
      "service.name": process.env.SERVICE_NAME,
    })
  ),
});

provider.addSpanProcessor(new SimpleSpanProcessor(new ConsoleSpanExporter()));

provider.register();

registerInstrumentations({
  instrumentations: [
    new HttpInstrumentation(),
    new ExpressInstrumentation(),
    new GraphQLInstrumentation(),
    new GrpcInstrumentation()
  ],
});

After I run my service I make a request to a mutation.

What did you expect to see?

I expect to see any traces (/spans?) in the console logs that mentions anything graphql-related. Some information about resolvers and such.

What did you see instead?

As for graphql spans, complete silence. I only see traces in the console log describing grpc- and http-requests that are made by the data sources. For example (among others):
{ traceId: 'f0f566152aa900573e30fdc8bb9ccf5e', parentId: '0bba32f6f1a44054', traceState: undefined, name: 'some.grpc.namespace.SomeMethod', id: '90dbfc2a9b0f9160', kind: 2, timestamp: 1680614886317000, duration: 309645, attributes: { 'rpc.system': 'grpc', 'rpc.method': 'SomeMethod', 'rpc.service': 'some.grpc.namespace.SomeMethod', 'net.peer.name': 'localhost', 'net.peer.port': 9408, 'rpc.grpc.status_code': 0 }, status: { code: 0 }, events: [], links: [] }

Additional context

The documentation says that supportet versions are: >=14 <16. Im not sure supported versions of what.
I have also tried to downgrade the graphql version from 16.to 15.7.2.

[Flarna]

Is it ensured that above OTel init code is executed before graphql is required?

[emiwidp]

Initializing the OTel things is the first thing done in the index.ts file.

Here is the first few lines in my index.ts file (removed some imports that seems completely irrelevant):

import "./src/opentelemetry/open-telemetry-tracing"; //Leads to the above OTel init code.
import { ApolloServer, ApolloServerPlugin, BaseContext } from "@apollo/server";
import { expressMiddleware } from "@apollo/server/express4";
import { ApolloServerPluginDrainHttpServer } from "@apollo/server/plugin/drainHttpServer";
import { ApolloServerErrorCode } from '@apollo/server/errors';
import gql from "graphql-tag";
import express from "express";
import http from "http";

[emiwidp]

I am using esbuild. Could it be a similar issue as this one? open-telemetry/opentelemetry-js#2786

I might say that I am not deeply familiar with node/js/typescript/esbuild, so there might be something that is obvious to some, that completely goes over my head

[Flarna]

I'm also not familiar with esbuild and how the result looks like.

bundlers tend to break Otel instrumentations. Reason is that instrumentations "listen" to require("graphql") but if graphql is bundled into a single JS file no such require happens and as a result on instrumentation is triggered.

As far as I remember at least webpack has an option to exclude specific modules to be included into the bundle.

[emiwidp]

I excluded graphql and that worked, thanks a lot!

[clintonb]

I'm facing the same issue, also with Webpack and TypeScript. Can anyone provide an example of how to configure?

It would also be great if the documentation was updated. The README claims this instrumentation is automatic, but it clearly is not as automatic as other instrumentations.

[emiwidp]

I agree that it would be nice to have in the readme file.

I was using esbuild, but what I had to do is to exclude graphql when building the index.js file:
esbuild --external:graphql.

Also, since the graphql-libraries are now not built into the index-file, I had to add a postbuild script that copies the needed graphql-library-files in the folder into my build folder (the folder where the resulting index.js is put)

[Flarna]

yes, bundlers like webpack break instrumentations. I'm not an expert of these so can't provide a guideline how to avoid that these tools break OTel.

Could you create a PR to add the limitations you faced and solutions you found into the Otel docs?

[timonmasberg]

I'm facing the same issue, also with Webpack and TypeScript. Can anyone provide an example of how to configure?

It would also be great if the documentation was updated. The README claims this instrumentation is automatic, but it clearly is not as automatic as other instrumentations.

Did you find a solution? I am currently facing the same, excluding graphql in the Webpack config did not help.

[Flarna]

Maybe try to not use any bundler and not use ESM modules (import) (refs: open-telemetry/opentelemetry-js#3698).

Also important is to register instrumentations before the instrumented modules are required.

[clintonb]

Maybe try to not use any bundler and not use ESM modules (import).

Rewriting the app is a non-starter for us.

[Flarna]

I didn't mean you should rewrite all your apps. My statement was intended to give some hints how to analyze this further and to get a solution step by step. I don't know your app nor I'm an bundler expert.

Support for bundlers (either document how to use them or improve the instrumentation libs itself) requires help from people using and understanding them.