missing lines in typescript source maps

[LarsDenBakker]

I'm using esbuild during testing to compile away typescript. I'm using istanbul to generate code coverage results, which uses https://www.npmjs.com/package/source-map to convert the instrumented code back to the original code. With TSC I get accurate code coverage results, but with esbuild I don't.

After investigation, I think it's because esbuild doesn't print some lines in the source maps while TSC does.

I'm using this source file:

export function foo() {
  return 'bar';
}

export function bar(condition: boolean) {
  if (condition) {
    return 1;
  }

  return 2;
}

This is the esbuild output:

export function foo() {
  return 'bar';
}
export function bar(condition) {
  if (condition) {
    return 1;
  }
  return 2;
}
//# sourceMappingURL=data:application/json;base64,ewogICJ2ZXJzaW9uIjogMywKICAic291cmNlcyI6IFsiL1VzZXJzL3JpMDRwaS9Xb3Jrc3BhY2UvY29kZS9taW5lL3dlYi9kZW1vL3Byb2plY3RzL2xpdC1lbGVtZW50LXRzL215LWVsZW1lbnQudHMiXSwKICAic291cmNlc0NvbnRlbnQiOiBbImV4cG9ydCBmdW5jdGlvbiBmb28oKSB7XG4gIHJldHVybiAnYmFyJztcbn1cblxuZXhwb3J0IGZ1bmN0aW9uIGJhcihjb25kaXRpb246IGJvb2xlYW4pIHtcbiAgaWYgKGNvbmRpdGlvbikge1xuICAgIHJldHVybiAxO1xuICB9XG5cbiAgcmV0dXJuIDI7XG59XG4iXSwKICAibWFwcGluZ3MiOiAiQUFBQSxBQUFPO0FBQ0wsU0FBTzs7QUFHRixvQkFBYTtBQUNsQixNQUFJO0FBQ0YsV0FBTzs7QUFHVCxTQUFPOzsiLAogICJuYW1lcyI6IFtdCn0K

The source maps translate to:

1: 0->1:0  0->1:7  
2: 0->2:2  9->2:9  
3: 
4: 0->5:7  20->5:20  
5: 0->6:2  6->6:6  
6: 0->7:4  11->7:11  
7: 
8: 0->10:2  9->10:9 

As you can see line 3 and 7 are missing.

This is the TSC output:

export function foo() {
  return 'bar';
}
export function bar(condition) {
  if (condition) {
    return 1;
  }
  return 2;
}
//# sourceMappingURL=data:application/json;base64,eyJ2ZXJzaW9uIjozLCJmaWxlIjoibXktZWxlbWVudC5qcyIsInNvdXJjZVJvb3QiOiIiLCJzb3VyY2VzIjpbIi4uL215LWVsZW1lbnQudHMiXSwibmFtZXMiOltdLCJtYXBwaW5ncyI6IkFBQUEsTUFBTSxVQUFVLEdBQUc7SUFDakIsT0FBTyxLQUFLLENBQUM7QUFDZixDQUFDO0FBRUQsTUFBTSxVQUFVLEdBQUcsQ0FBQyxTQUFrQjtJQUNwQyxJQUFJLFNBQVMsRUFBRTtRQUNiLE9BQU8sQ0FBQyxDQUFDO0tBQ1Y7SUFFRCxPQUFPLENBQUMsQ0FBQztBQUNYLENBQUMiLCJzb3VyY2VzQ29udGVudCI6WyJleHBvcnQgZnVuY3Rpb24gZm9vKCkge1xuICByZXR1cm4gJ2Jhcic7XG59XG5cbmV4cG9ydCBmdW5jdGlvbiBiYXIoY29uZGl0aW9uOiBib29sZWFuKSB7XG4gIGlmIChjb25kaXRpb24pIHtcbiAgICByZXR1cm4gMTtcbiAgfVxuXG4gIHJldHVybiAyO1xufVxuIl19

The source maps translate to:

0->1:0  6->1:6  16->1:16  19->1:19  
2: 4->2:2  11->2:9  16->2:14  17->2:15  
3: 0->3:0  1->3:1  
4: 0->5:0  6->5:6  16->5:16  19->5:19  20->5:20  29->5:38  
5: 4->6:2  8->6:6  17->6:15  19->6:17  
6: 8->7:4  15->7:11  16->7:12  17->7:13  
7: 5->8:3  
8: 4->10:2  11->10:9  12->10:10  13->10:11  
9: 0->11:0  1->11:1 

Here line 3 and 7 are present.

You can observe the difference between esbuild and TSC in this REPL: https://repl.it/repls/TastyRareLevels

When a line is missing, the source-map library returns null. I found an open issue for this: mozilla/source-map#261

I haven't worked with source maps before, so I'm not entirely certain what the solution here should be. Perhaps when source-map returns null, we should just use the position of the generated code. However this logic is buried inside other libraries I'm using.

Other people might run into this issue as well, so perhaps esbuild could output more detailed source maps?

[evanw]

Wow, that's so weird. Thanks for investigating this.

Clearly my mental model of how that library works is wrong. I assumed that it just binary searches through the mappings in the generated file and finds the closest one that comes before the query. That even seems to be what the source-map library itself does. But if that were true, a query would never return null as long as the file starts with a mapping. So I'm not really sure what's going on now.

FWIW even the source maps generated by TypeScript have dead spots that return null. You can see them by iterating the query code you have over all lines and columns in the generated file. The dead spots appear to be in indents.

If I had to guess, I'd say that Mozilla's library fails to return the previous marker if it's not on the same line as the query. I guess that's just a bug in their implementation (the one you linked to)? If that's the case, an appropriate workaround to avoid these dead spots could be for esbuild to ensure every generated line starts with a mapping. It should be sufficient to just copy the last mapping for the previous line onto the new line if the new line doesn't already start with one.

[LarsDenBakker]

Yeah, I think that's the case. There is a bias option for columns, returning the nearest match if it can't find anything. But that doesn't work for lines.

[evanw]

Ok I think I've fixed the missing lines within an input file. See the release notes for caveats. If bundling is on, some lines that don't correspond to an input file still won't have coverage. Repeating previous mappings to cover those lines seems incorrect to me because those lines aren't actually a part of that file at all, and could lead to confusing debugging situations.

[LarsDenBakker]

I can confirm it's working correctly now! Thanks a lot for the swift action.