Fix Mistral model

[WoosukKwon]

Should be merged after #1196

This PR includes a bug fix for MistralConfig and sliding window plus small stylistic changes.

[agrogov]

Seems fix didn't helped:
2023-09-30 21:11:56,076 INFO worker.py:1642 -- Started a local Ray instance.
INFO 09-30 21:11:56 llm_engine.py:72] Initializing an LLM engine with config: model='mistralai/Mistral-7B-Instruct-v0.1', tokenizer='mistralai/Mistral-7B-Instruct-v0.1', tokenizer_mode=auto, trust_remote_code=False, dtype=torch.bfloat16, download_dir=None, load_format=auto, tensor_parallel_size=2, seed=0)
Downloading (…)okenizer_config.json: 100%|███████████████████████████████████████████████| 1.46k/1.46k [00:00<00:00, 10.7MB/s]
Downloading tokenizer.model: 100%|█████████████████████████████████████████████████████████| 493k/493k [00:00<00:00, 7.41MB/s]
Downloading (…)/main/tokenizer.json: 100%|███████████████████████████████████████████████| 1.80M/1.80M [00:00<00:00, 3.95MB/s]
Downloading (…)cial_tokens_map.json: 100%|██████████████████████████████████████████████████| 72.0/72.0 [00:00<00:00, 157kB/s]
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/usr/local/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 624, in
engine = AsyncLLMEngine.from_engine_args(engine_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm/engine/async_llm_engine.py", line 442, in from_engine_args
engine = cls(engine_args.worker_use_ray,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm/engine/async_llm_engine.py", line 250, in init
self.engine = self._init_engine(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm/engine/async_llm_engine.py", line 279, in _init_engine
return engine_class(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm/engine/llm_engine.py", line 100, in init
self._init_workers_ray(placement_group)
File "/usr/local/lib/python3.11/site-packages/vllm/engine/llm_engine.py", line 173, in _init_workers_ray
self._run_workers(
File "/usr/local/lib/python3.11/site-packages/vllm/engine/llm_engine.py", line 682, in _run_workers
all_outputs = ray.get(all_outputs)
^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/ray/_private/auto_init_hook.py", line 24, in auto_init_wrapper
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/ray/_private/client_mode_hook.py", line 103, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/ray/_private/worker.py", line 2547, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(ValueError): ray::RayWorker.execute_method() (pid=597, ip=172.17.0.3, actor_id=4c2e7a57d2cb65354f7eb2b701000000, repr=<vllm.engine.ray_utils.RayWorker object at 0x7f5b58fcdcd0>)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm/engine/ray_utils.py", line 29, in execute_method
return executor(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm/worker/worker.py", line 67, in init_model
self.model = get_model(self.model_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/model_loader.py", line 54, in get_model
model_class = _get_model_architecture(model_config.hf_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/model_loader.py", line 48, in _get_model_architecture
raise ValueError(
ValueError: Model architectures ['MistralForCausalLM'] are not supported for now. Supported architectures: ['AquilaModel', 'BaiChuanForCausalLM', 'BaichuanForCausalLM', 'BloomForCausalLM', 'FalconForCausalLM', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'GPTJForCausalLM', 'GPTNeoXForCausalLM', 'InternLMForCausalLM', 'LlamaForCausalLM', 'LLaMAForCausalLM', 'MPTForCausalLM', 'OPTForCausalLM', 'QWenLMHeadModel', 'RWForCausalLM']