How to index CoNLL-U sub-features?

[fishfree]

I notcie here, they are commented out. What's you roadmap or any other hacking workarounds? Many thanks!

[jan-niestadt]

You should be able to configure this yourself in your own version of conll-u.blf.yaml. Use process instructions, specifically the replace instruction for regex replace. You can isolate each subfeature this way an index it in a separate annotation.

[fishfree]

@jan-niestadt Could you pls show me an example?

[jan-niestadt]

If your features column contains values like Number=Plur|Person=3|Tense=Pres, you should be able to index Number and Person like this (untested, but I hope you get the idea):

- name: feats
  displayName: Features
  valuePath: 6
  multipleValues: true

- name: number
  valuePath: 6
  process:
  - action: replace
    find: "^.*Number=([^\\|]+).*$"
    replace: "$1"

- name: person
  valuePath: 6
  process:
  - action: replace
    find: "^.*Person=([^\\|]+).*$"
    replace: "$1"

[fishfree]

@jan-niestadt Thank you very much! If some of my corpus CoNLL-U files have no FEATs part ( for the language doesn't support this output), will it be automatically ignored and bypassed? Hence, other corpus files with the FEATs part keep working?

[jan-niestadt]

If column 6 is empty in those files, the regex won't match, so nothing will be replaced, so the original empty value will be indexed for Person. That shouldn't be a problem.

What could be a problem is when FEAT sometimes contains Person and sometimes contains only other features (but not Person). For a value without Person, the replace action would do nothing, so the entire original FEAT value would be indexed.

Maybe you could solve this with an extra replace action that empties any FEAT value where Person doesn't occur (using negative lookahead):

- name: person
  valuePath: 6
  process:
  # If string doesn't contain "Person=", make it empty
  - action: replace
    find: "^(?!.*Person=).*$"   # matches any string that doesn't contain "Person="
    replace: ""
  # Remove everything except the value of the Person FEAT (or leave the string unmodified if regex doesn't match)
  - action: replace
    find: "^.*Person=([^\\|]+).*$"    # match the value for Person in group 1
    replace: "$1"