Compatibility table, converter

[scripthunter7]

We should detect incompatible rules during the linting process and offer converted rules as fixes, if possible. In order to do this, we need a compatibility table / converter. Theoretically, this could be implemented directly within linter rules, but a standalone API would be more convenient.

This API should be able to:

• Compatibility detection
  • Detecting which adblockers a rule is compatible with. Parser specifies the syntax in most cases, but not the compatibility. For example, parser eats this scriptlet rule without any problems:

    example.org#%#//scriptlet("unknown-scriptlet")

    but in the practice it's not compatible with any adblocker, since it's not a known scriptlet. This is where the compatibility table comes in.
• Conversion: The rule converter allows you to convert from any syntax to any syntax, as long as the destination syntax supports the rule type. If it doesn't support the source rule type, an error will be thrown. For example, you cannot convert a CSS injection to Adblock Plus, since ABP simply doesn't support CSS injections.

Points worth considering:

• Does an adblocker support a rule category? This is partially done at parser level.
  • Examples:
    • ABP doesn't support HTML filtering rules, but ADG/uBO does
    • ADG supports JS injection, but uBO/ABP doesn't
    • OS-level AdGuard supports CSS injection, but the Safari extension doesn't,
    • etc.
• Compatibility table categories:
  • ExtendedCSS elements
  • Scriptlets
  • Redirects
  • HTML filtering rules
• Wiki generation, like in Scriptlets library

Please note that this compatibility table is a quite bit different from the one in the Scriptlets library, since it must be able to convert from any syntax to any syntax, and it must also handle selectors, HTML filtering rules, etc.

@ameshkov I opened this issue so that we can schedule this as part of future developments. What do you think?

[ameshkov]

I think a compatibility table will be VERY useful, but in order to make it "maintainable", we need filters maintainers expertise here.

The idea is that we should have a table in the repo that filters maintainers can contribute to.
This compatibility table should be filled incrementally, at first we may cover only a part of rules.

[scripthunter7]

@ameshkov I haven't specified the details yet, but my idea is to store the data in a TS object / JSON file / YML file. TS/YML file would be best as they allow comments. I think these are clear and easy-to-manage formats.

Of course, we will add a documentation that describes exactly this structure.

[ameshkov]

Makes perfect sense to me

[scripthunter7]

I thought about compatibility tables, here is a draft of what I have in mind.

First of all, there is an important point to keep in mind: we need a maintainable, convenient data structure. Even if it needs to be converted later by a build to be "optimal" for good performance, it should be easy to maintain.

What I was thinking about is the following:

• Store each category in a separate file. Currently, there are 5 categories: modifiers, scriptlets, redirects, extended CSS selectors, and some HTML filtering elements. This means that there will be 5 files: modifiers.yaml, scriptlets.yaml, redirects.yaml, extended_css_selectors.yaml, html_filtering.yaml.
• Each file contains an object, where the keys are the names of each adblocker and the value is the compatibility records for the given category and adblocker.

What should be detailed in the records?

• params: the parameters that the entity accepts, if it accepts any. This is important for scriptlets, modifiers and pseudo-classes. We need some kind of schema for this, but I'm not sure what would be the best one. Maybe jsonschema? This is important because some adblockers may have different parameters for the same entity, and we need to be able to specify that.
• deprecated: whether the entity is deprecated or not. If it is, we need to specify the version in which it was deprecated (if known).
• version_added: the version in which the entity was added. If it is not known, we can use null or skip the field.
• docs: the URL to the documentation of the entity. If it is not known, we can use null or skip the field.
• description: any additional notes that we may want to add. If it is not known, we can use null or skip the field. Later can be used in VSCode extension to show additional information about the entity.

The fields may change depending on the individual categories, so it is advisable to put them in separate files.

Here is a draft of what the modifiers.yaml file could look like:

adg_for_windows:
- name: script
  params: false
  deprecated: false
  version_added: '1.0'
  docs: https://kb.adguard.com/en/general/how-to-create-your-own-ad-filters#script-modifier
  description: The rule corresponds to script requests, e.g. javascript, vbscript.

or modifiers.json:

{
  "adg_for_windows": [
    {
      "name": "script",
      "params": false,
      "deprecated": false,
      "version_added": "1.0",
      "docs": "https://kb.adguard.com/en/general/how-to-create-your-own-ad-filters#script-modifier",
      "description": "The rule corresponds to script requests, e.g. javascript, vbscript."
    }
  ]
}

Points to consider:

• It is worth categorising adblockers in detail, as there may be differences within a brand due to the specificities of each platform.
  • AdGuard: adg_for_windows, adg_for_mac, adg_for_android, adg_for_ios, adg_ext_chromium, adg_ext_firefox, adg_content_blocker
  • uBlock: ubo_ext_chromium, ubo_ext_firefox
  • ABP: abp_ext_chromium, abp_ext_firefox

[ameshkov]

Sounds good to me. I think we should start with something, if we realize that there are any flaws in the chosen structure, converting it to a different one would not be too complicated compared to filling out the table.

[scripthunter7]

I think we should start with something, if we realize that there are any flaws in the chosen structure, converting it to a different one would not be too complicated compared to filling out the table.

Yes, a scope of this size probably cannot be covered perfectly at first, but we will see the edge cases in time.