Syntax highlight vimscript

[NickeZ]

Hi, Really nice project! How would I implement syntax highlighting for vimscript? Could I add a sublime syntax highlight definition file to my project somehow or do I need to modify cobalt or syntect?

[epage]

Hey, sorry for not getting back to you on this.

We use syntect for syntax highlighting. We load the existing dump of syntax files but I don't see any documentation for what languages are included or how to add more.

I think it'd be a good idea for us to add a "syntax folder" so people can add syntax files on top of what is already supported.

For someone wanting the details, we'd need to refactor cobalt.rs/src/syntax_highlight.rs so that we can call

if load_syntax_files {
    syntax_set.load_syntaxes(syntax_path, false);
    syntax_set.link_syntaxes();
}

load_syntax_files is a placeholder for whatever the logic should be. We should avoid doing this unless we have to because the default approach comes pre-linked and syntect's docs say " This operation is idempotent, but takes time even on already linked syntax sets."

I'm leaning towards checking if the directory exists. I'm for minimizing configuration unless we have a good use case so I'd like to avoid a load_syntax_files or syntax_path in the config file.

[epage]

Looks like load_from_folder is how to load themes but not syntaxes. For those we'd want load_syntaxes. We'd then need to link_syntaxes

[boxofrox]

@epage, load_syntaxes requires a mutable SyntaxSet which would require some modification to SETUP in syntax_highlight.rs [1].

[1]: https://github.com/cobalt-org/cobalt.rs/blob/master/src/syntax_highlight.rs#L34

There are a few ways to handle the mutability requirement:

1. std::cell::RefCell

   struct Setup {
       syntax_set: RefCell<SyntaxSet>,
       theme_set: ThemeSet,
   }
   lazy_static!{
       static ref SETUP: Setup = Setup {
           syntax_set: RefCell::new(SyntaxSet::load_defaults_newlines()),
           theme_set: ThemeSet::load_defaults()
       };
   }

   • Con? I'm not sure RefCell is appropriate with the Sync marker on SETUP [2]
   • Con: Not thread-safe. Is cobalt multi-threaded?

2. std::sync::Mutex

   struct Setup {
       syntax_set: Mutex<SyntaxSet>,
       theme_set: ThemeSet,
   }
   lazy_static!{
       static ref SETUP: Setup = Setup {
           syntax_set: Mutex::new(SyntaxSet::load_defaults_newlines()),
           theme_set: ThemeSet::load_defaults()
       };
   }

   • Pro: Simple to implement. Least invasive change to codebase.
   • Pro: Thread-safe.
   • Con: Mutex.unlock().unwrap() when writing and reading from syntax_set.

3. Replace the global SETUP with a locally scoped value, SyntaxHighlight, initiated in cobalt.rs:build() by calling syntax_highlight.rs:SyntaxHighlight::new().

   • Pro: No global vars.
   • Pro: No additional syntax vars in Config.
   • Pro: Single initialization reused by each Documents rendered.
   • Pro: After initialization, immutable reference may be shared across threads.
   • Con: One additional parameter syntax_highlight for Document::render_excerpt and Document::render.
   • Con: Removes single source to Setup for CodeBlock and DecoratedParser. Must dispatch scoped value to each through call-stack. May require std::rc::Rc or std::sync::Arc.
   • Con: #cfg[feature("syntax-highlight")] et. al. in cobalt.rs.
   • Con: Sticky situation in document.rs involving use of SyntaxHighlight as function parameter.

[2]: https://github.com/cobalt-org/cobalt.rs/blob/master/src/syntax_highlight.rs#L31

@epage, which route would you prefer I take with this?

[epage]

I plan to parallelize cobalt.

syntect is not thread safe though. I've heard of it before and am running into it now with my change I'm working on to better define the cross platform semantics of syntax highlighting. I'm trying to throw it in a lock but am running into ownership issues with ThemeSet (in the DecoratedParser).

[epage]

Oh, so I talked about the Mutex opytion, forgot to talk about the dependency injection option.

The main challenge there is being efficient while doing so. We already have to theme.clone() for each liquid parser we create. We'd also need to do the same thing for our SyntaxSet. Granted, I want to refactor things to eventually be reusing liquid parsers across documents so that pain should be reduced.

The markdown formatting is less difficult to efficiently share because the call stack is shallower.