Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DocumentSettings ignore extra fields Pydantic Validation #628

Closed
wants to merge 1 commit into from

Conversation

dynalz
Copy link

@dynalz dynalz commented Jul 22, 2023

When setting the default pydantic Config to forbid extra fields:

from pydantic import config, Extra
config.BaseConfig.extra = Extra.forbid

Beanie throws errors at document initialization due to the following:

ValidationError: 4 validation errors for DocumentSettings __dict__
  extra fields not permitted (type=value_error.extra)
__doc__
  extra fields not permitted (type=value_error.extra)
__module__
  extra fields not permitted (type=value_error.extra)
__weakref__
  extra fields not permitted (type=value_error.extra)

The proposed commit fixes the issue.
Not sure if ideal, or if there is any caveats with this solution

When setting the default pydantic Config to forbig extra fields:

```py
from pydantic import config, Extra
config.BaseConfig.extra = Extra.forbid
```

Beanie throws errors at document initialization due to the following:
ValidationError: 4 validation errors for DocumentSettings
__dict__
  extra fields not permitted (type=value_error.extra)
__doc__
  extra fields not permitted (type=value_error.extra)
__module__
  extra fields not permitted (type=value_error.extra)
__weakref__
  extra fields not permitted (type=value_error.extra)


The proposed commit fixes the issue. 
Not sure if ideal, or if there is any caveats with this solution
@roman-right
Copy link
Member

roman-right commented Jul 22, 2023

Hi!

Sorry, I didn't get the use-case. Could you please explain it a bit?

@dynalz
Copy link
Author

dynalz commented Jul 23, 2023

So in our dev environment, we use Pydantic with more enforced rules and Dataclasses on production (for performance issues)

Thus in our dev environment we force all typing and all issues that mimic dataclasses, so that when it goes into production we have no issues. (This is for our service dataclasses, not related to beanie)

This is our use case:

if SETTINGS.is_production() or TYPE_CHECKING:
    # production environment
    from dataclasses import dataclass
else:
    # dev environment
    # pydantic enforced configuration
    from pydantic.dataclasses import dataclass
    ...
    config.BaseConfig.extra = Extra.forbid
    config.BaseConfig.validate_all = True
    config.BaseConfig.validate_assignment = True
    config.BaseConfig.arbitrary_types_allowed = True
    ...

The config.BaseConfig.extra = Extra.forbid is so that it avoids anyone passing extra variables that are not meant for the dataclass as shown below:

from dataclasses import dataclass

@dataclass
class Person:
    name: str

Person(**{"name": "John", "foo": "bear"}) # throws error on dataclasses module but not on pydantic module

this throws error as @dataclass comes from dataclasses module, but not if comes from pydantic.dataclasses module. Thus why the line config.BaseConfig.extra = Extra.forbid when using pydantic to mimic the dataclasses module behaviour, throwing exception on dataclasses initialization when passing arguments which are not valid for that dataclass.

However after we do the config.BaseConfig.extra = Extra.forbid (replacing the default Extra.ignore on pydantic) beanie throws at initializing DocumentSettings since it receives extra fields not defined on its model such as __dict__, __doc__, __module__ and __weakref__

Effectively with this commit the behaviour should remain the same for beanie since the Pydantic Config extra = Extra.ignore is already the default and currently being used on DocumentSettings

Let me know if this made sense

@dynalz
Copy link
Author

dynalz commented Aug 4, 2023

Let me know if some more clarification is needed

@roman-right
Copy link
Member

Hi. Sorry for the late reply. I had a lot of work with Pydantic V2 compatibility. If you still need this feature, could you please resolve the conflict and adapt this to work with Pydantic V2 (and still work with V1)?

@roman-right
Copy link
Member

It looks like this PR is not in work more. If no, feel free to open a new one.

@roman-right roman-right closed this Dec 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants