[Discussion] string encoding

[jedwards1211]

Hi there, Ć looks awesome and I really want to use it but for some of my use cases (file parsing) dealing with string encoding would be hard...

I'm not experienced in working with unicode string encodings in C/C++ and I don't know if you are either, but have you had any thoughts about what it would take to make Ć strings unicode (or maybe a pragma to turn on unicode strings)?

It's something I might look into contributing if you're open to it and would like to give me tips on working with the codebase.
In a quest to make cross-language APIs I've determined that Haxe definitely won't work, and SWIG/Emscripten seem like they would be workable, but a huge hassle compared to if I could use Ć.

[pfusik]

Ć is already Unicode-capable. The actual string encoding varies between the target languages. Are you concerned with the C or C++ interface? The C and C++ strings are expected to be UTF-8-encoded, which is the default encoding on modern GNU/Linux and macOS.
UTF-8 is also widely used for text file encoding on Windows. However, Windows API historically uses UTF-16. You can convert between UTF-8 and UTF-16 of course.

[jedwards1211]

Oh I didn't realize that the C/C++ strings would be UTF-8, that's great news! I just assumed because somewhere in the docs you said if you're planning to do a bunch of string manipulation, use perl, and I know string encoding in C/C++ is kind of crazy.

In that case, I'll give Ć a try soon 😃

[jedwards1211]

There's not currently a Ć-native regex that abstracts the differences between target languages is there? Feel free to let me know your thoughts on that.

[pfusik]

I started adding regular expressions today. So far it's just one method, see its test.
Next up: retrieving the position and contents of the match, then the captures.

[jedwards1211]

Wow cool! I'm gonna start playing around with Ć this weekend.

[pfusik]

Match location and captures. It's implemented for C#, Java, JavaScript and Python.
Next up: documentation, Regex object with pre-processed expression for improved performance of repeated searches.

[pfusik]

Unicode capabilities explained. Regexes implemented and documented. Can we close this?

[jedwards1211]

Yup!