Feature request: Support keyword/link parsing

[Maxou44]

Hello,

Because of how the lib parse the text, it's complicated to replace a string that contains a space in it (by example user names, emojis or links) 😅

It could be nice to add a list of keywords (that support space character) to the parser and extract these strings as a custom node type. It could be awesome if it can support both case sensitive and case unsensitive and be supported in tags and nested tags ❤️

If we add a "type" on each node, it could be relevant to also parse the "\n" and "\r" as type "eol" and support native http/https link detection as "link" type. 😊

[
  "example",
  " ",
  {
    "type": "keyword",
    "keyword": "my username"
  },
  {
    "type": "tag",
    "tag": "b",
    "attrs": {
      "value": "value"
    },
    "content": [
      " ",
      "my",
      " ",
      "text",
      " "
    ]
  },
  " ",
  {
    "type": "eol"
  },
  {
    "type": "link",
    "link": "https://github.com/JiLiZART/BBob/"
  },
  " ",
  "end"
]

[JiLiZART]

Looks like plugin can solve this problem?

[Maxou44]

Looks like plugin can solve this problem?

I didn't found a lot of documentation about the plugin system, maybe?

Do you think it will be optimized enough?

[JiLiZART]

For example I implement link transformer plugin.

https://codesandbox.io/s/bbob-plugin-example-dmq1bh?file=/src/App.js

Plugin system is simple, you just iterate over ast tree and apply modifications

[Maxou44]

Thanks for the sample, but how can I handle a text that include space? If I'm right I will receive several callback to the function, no?

[JiLiZART]

You need some sort of buffer variable, that collects all string nodes.
And if the next node is not a string node you do some logic (parse all strings to find nickname or whatever). After you drop buffer as set of text and tag nodes.

tree - is a simple array with objects and strings

Some example code

const linkParsePlugin = (tree) => {
  let stringsBuffer = []
  const transformStrings = () => { }

  return tree.reduce((acc, node) => {
   
    if (isStringNode(node)) {
       stringsBuffer.push(node)
       return acc
    } else if (stringsBuffer.length) {
      // do some stuff with all these strings and drop to tree
      
      acc = [...transformStrings(stringsBuffer)]
      stringsBuffer = []
    }
    
    return acc
  }, [])
};

[Maxou44]

Plugins are only supported by the React package, right? Can I use the function on an AST generated by the @bbob/parser?

[JiLiZART]

Yep, but @bbob/core has plugin system

[JiLiZART]

import bbob from '@bbob/core'
import { render } from '@bbob/html'
import presetHTML5 from '@bbob/preset-html5'
const code = `[i]Text[/i]`;

const html = bbob([presetHTML5(), linkParsePlugin]).process(code, { render }).html;

console.log(html); // <span style="font-style: italic;">Text</span>

[Maxou44]

I was able to parse my usernames correctly 🎉
Thanks for your help!

// This plugin merge all string chunks into a single string
const mergeStringsPlugin = (tree) => {
	let stringsBuffer = []
	return tree.reduce((acc, node, i) => {
		// Is a string
		if (isStringNode(node) && i !== tree.length - 1) {
			stringsBuffer.push(node)
			return acc
		}

		// Detect end of concat string
		if (i === tree.length - 1 || !isStringNode(node)) {
			const out = [...acc, stringsBuffer.join(''), node];
			stringsBuffer = [];
			return out;
		}
		return acc
	}, [])
};

// This plugin takes an array of users that contain a "login" field and create "user" node if it match
const usersParsePlugin = (users) => (tree) => {
	let updatedTree = tree;
	for (const user of users) {
		updatedTree = updatedTree.map((e) => {
			if (isStringNode(e)) {
				const out = [];
				const splitted = e.split(new RegExp(`@${user.login}`, 'i'));
				for (let i = 0; i < splitted.length; i++) {
					if (i) {
						out.push(TagNode.create(
							"user", {
							user
						}));
					}
					out.push(splitted[i]);
				}
				return out;
			}
			return [e];
		}).reduce((acc, e) => [...acc, ...e], []);
	}
	return updatedTree;
};

const ast =  bbob([mergeStringsPlugin, usersParsePlugin(myUsers)]).process(value, options);