Pydantic 2.x cfg #1239
Pydantic 2.x cfg #1239
Conversation
winglian
force-pushed
the
pydantic-cfg
branch
from
076d50a
to
d464d1b
Compare
winglian
force-pushed
the
pydantic-cfg
branch
from
3b032b8
to
50199ea
Compare
src/axolotl/cli/__init__.py
Outdated
| capabilities = GPUCapabilities( | ||
| bf16=is_torch_bf16_gpu_available(), n_gpu=os.environ.get("WORLD_SIZE", 1) | ||
| ) |
There was a problem hiding this comment.
Should this be moved into validate or normalize config?
There was a problem hiding this comment.
I'm avoiding setting this in the validation as there are downstream use cases where a user might want to make sure their configuration works on a configured GPU cluster before firing off the training.
| @model_validator(mode="before") | ||
| @classmethod | ||
| def check_sample_packing_w_xformers(cls, root): | ||
| if root.get("sample_packing") and root.get("xformers_attention"): |
There was a problem hiding this comment.
Can we have a few methods check for when sample_packing is on but not actually active due to unsupported model type?
Pseudo code
def check_sample_packing_active(cls, root):
if root.get("sample_packing") and not any(llama/mistral/qwen):
raise ValueError(
"sample_packing not compatible with current model type"
)
There was a problem hiding this comment.
I don't know that we can reliably detect the model at this step of the validation to raise a ValueError
winglian
force-pushed
the
pydantic-cfg
branch
from
ea381f0
to
032eced
Compare
| @@ -543,7 +543,7 @@ is_mistral_derived_model: | |||
| is_qwen_derived_model: | |||
|
|
|||
| # optional overrides to the base model configuration | |||
| model_config: | |||
| model_config_overrides: | |||
There was a problem hiding this comment.
I just noticed this, but we would need to deprecate the old name (add valueerror)
There was a problem hiding this comment.
we can't actually deprecate it with Pydantic because model_config is an internal variable name for pydantic models
remember to return in validator missing return add missing relora attributes fix test for DictDefault change fix sys template for mistral from fastchat change in PR 2872 fix test for batch size warning
winglian
force-pushed
the
pydantic-cfg
branch
from
d33a8f7
to
7f688b6
Compare
Description
This PR migrates the validation to use Pydantic validators. We keep all of the existing tests with some modification. I've attempted to capture all the
cfg.*attributes I could find and make sure they are represented in the Pydantic config models. I've also added a GPUCapabilities model to abstract away the underlying hardware checks so that it can be checked offline before sending the config for actual training.For now, we run the validation and convert it back to a DictDefault that we currently use for the
cfg. This is to minimize the blast radius of this change to strictly validation. We can consider down the line how to swap out the various uses ofcfgand how the attributes are accessed and if it's compatible with pydantic models.Motivation and Context
How has this been tested?
Existing unit tests.
Screenshots (if appropriate)
Types of changes
Social Handles (Optional)