Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failure rendering a simple template #252

Open
pavel-esir opened this issue May 31, 2024 · 2 comments
Open

Failure rendering a simple template #252

pavel-esir opened this issue May 31, 2024 · 2 comments

Comments

@pavel-esir
Copy link

pavel-esir commented May 31, 2024

Hi Dear Jinja2cpp developers,

i notices something unexpected when i try to apply a chatbot template.

This "{% for message in messages %}{{ '<start_of_turn>' + message['role'] + message['content'] }}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model'}}{% endif %}" fails with an error:

terminate called after throwing an instance of 'nonstd::expected_lite::bad_expected_access<jinja2::ErrorInfoTpl<char> >'
  what():  bad_expected_access

When i remove the last {% if %} then it's ok. Attached a minimal reproducer. Looks like a bug, but idk maybe my template is wrong. Could you please take a look?

#include <jinja2cpp/template.h>
#include <jinja2cpp/template_env.h>

using namespace std;
using namespace jinja2;

int main() {
    TemplateEnv env;
    env.GetSettings().lstripBlocks = true;
    env.GetSettings().trimBlocks = true;
    Template tpl(&env);
       
    string broken_tpl = "{% for message in messages %}{{ '<start_of_turn>' + message['role'] + message['content'] }}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model'}}{% endif %}";
    string working_tpl = "{% for message in messages %}{{ '<start_of_turn>' + message['role'] + message['content'] }}{% endfor %}";
      
    tpl.Load(broken_tpl);
    
    jinja2::ValuesMap params_1 {{"role", "system"}, {"content", "You are a friendly chatbot who always responds in the style of a pirate"}};
    jinja2::ValuesMap params_2 {{"role", "user"}, {"content", "1+1="}};
    jinja2::ValuesMap params = {
        {"messages", jinja2::ValuesList({params_1, params_2})},
        {"add_generation_prompt", true},
        {"bos_token", "<bos>"},
        {"eos_token", "<eos>"},
        {"unk_token", "<unk>"},
        {"pad_token", "<pad>"}
    };

    cout << tpl.RenderAsString(params).value() << endl;

    return 0;
}

broken_tpl is an abbreviated version, just to reproduce the failure. The full template which i would loke to render is from gemma chatbot:

{{ bos_token }}

{% if messages[0]['role'] == 'system' %}
  {{ raise_exception('System role not supported') }}
{% endif %}

{% for message in messages %}
  {% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}
    {{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}
  {% endif %}

  {% if message['role'] == 'assistant' %}
    {% set role = 'model' %}
  {% else %}
    {% set role = message['role'] %}
  {% endif %}

  {{ '<start_of_turn>' + role + '\n' + message['content'] | trim + '<end_of_turn>\n' }}
{% endfor %}

{% if add_generation_prompt %}
  {{ '<start_of_turn>model\n' }}
{% endif %}

@pavel-esir
Copy link
Author

i localized a bit more, when i replace {{ with { it works on a full template

@pavel-esir
Copy link
Author

I mean replacing {{'<start_of_turn>model'}} with {'<start_of_turn>model'} fixes the problem.

Looks like this bug can be easily fixed. Could you please take a look?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant