[Mistral] Mistral-7B-v0.1 support

[Bam4d]

No description provided.

[WoosukKwon]

This does not work because MistralConfig is not a regular model in HF transformers at the moment (v4.33.3). Could you define this config class just like this?

[WoosukKwon]

@timlacroix Besides this, it seems everything works fine!

[timlacroix]

ok addressed. Will we need to change this back after the next release ?

[WoosukKwon]

@timlacroix Yes. Once a new version of HF transformers is released, we will remove it.

[casper-hansen]

The Mistral model is almost equivalent to llama in terms of quantizing the model, it would be super easy to extend support as I have already added Mistral in AutoAWQ. If you can modify this part below, you will enable AWQ quantized models:

_MODEL_CLASSES_SUPPORT_QUANTIZATION = [
    LlamaForCausalLM,
    MistralForCausalLM,
]

After that, you should be able to run inference with the quantized model that is already available: https://huggingface.co/casperhansen/mistral-7b-instruct-v0.1-awq

from vllm import LLM, SamplingParams

prompts = [
    "The future of AI is",
]
sampling_params = SamplingParams(temperature=0.8, top_p=0.95)

llm = LLM(model="casperhansen/mistral-7b-instruct-v0.1-awq", quantization="awq", dtype="half")

outputs = llm.generate(prompts, sampling_params)

# Print the outputs.
for output in outputs:
    prompt = output.prompt
    generated_text = output.outputs[0].text
    print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")

[WoosukKwon]

LGTM. As this PR is not modifiable, I will fix some miscellaneous issues right after merging this PR.