Skip to content

Commit

Permalink
Those modules had been installed and used on a V14 project.
Browse files Browse the repository at this point in the history
  • Loading branch information
Cyril De Faria authored and isabellerichard committed Mar 5, 2021
1 parent 635b900 commit bf4ad53
Show file tree
Hide file tree
Showing 65 changed files with 3,856 additions and 0 deletions.
81 changes: 81 additions & 0 deletions smile_anonymization/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
======================
Database Anonymization
======================

.. |badge2| image:: https://img.shields.io/badge/licence-AGPL--3-blue.png
:target: http://www.gnu.org/licenses/agpl-3.0-standalone.html
:alt: License: AGPL-3
.. |badge3| image:: https://img.shields.io/badge/github-Smile_SA%2Fodoo_addons-lightgray.png?logo=github
:target: https://github.com/Smile-SA/odoo_addons/tree/13.0/smile_anonymization
:alt: Smile-SA/odoo_addons

|badge2| |badge3|

This module allows to anonymize automatically a database backup.
To do that, you need to define data mask on model fields in Python code or via UI.

**Table of contents**

.. contents::
:local:

Usage
=====

| A data mask is a SQL statement, e.g.:
| ``'partner_' || id::text WHERE is_company IS NOT TRUE``
Add such a mask on the fields containing sensitive data:

* in Python code, e.g.: ``fields.Char(data_mask='NULL')``
* in UI via the menu *Settings > Technical > Database Structure > Fields*

The lock icon allows not to overload data mask at each module update if you defined it in Python code and modify via UI.

Add this module in *server_wide_modules* list in your config file or in the option *--load*.

Go to database manager and backup the desired database. You will download a anonymized backup.

Requirements
============

There are no requirements to use this module.

Bug Tracker
===========

Bugs are tracked on `GitHub Issues <https://github.com/Smile-SA/odoo_addons/issues>`_.
In case of trouble, please check there if your issue has already been reported.
If you spotted it first, help us smashing it by providing a detailed and welcomed
`feedback <https://github.com/Smile-SA/odoo_addons/issues/new?body=module:%smile_anonymization%0Aversion:%2013.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**>`_.

Do not contact contributors directly about support or help with technical issues.

Credits
=======

Authors
~~~~~~~

* Smile SA

Contributors
~~~~~~~~~~~~

* Corentin Pouhet-Brunerie
* Ismail EL BAKKALI

Maintainers
~~~~~~~~~~~

This module is maintained by the Smile SA.

Since 1991 Smile has been a pioneer of technology and also the European expert in open source solutions.

.. image:: https://avatars0.githubusercontent.com/u/572339?s=200&v=4
:alt: Smile SA
:target: http://smile.fr

This module is part of the `odoo-addons <https://github.com/Smile-SA/odoo_addons>`_ project on GitHub.

You are welcome to contribute.
7 changes: 7 additions & 0 deletions smile_anonymization/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# -*- coding: utf-8 -*-
# (C) 2019 Smile (<http://www.smile.fr>)
# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl).

from . import controllers
from . import models
from . import service
26 changes: 26 additions & 0 deletions smile_anonymization/__manifest__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# -*- coding: utf-8 -*-
# (C) 2019 Smile (<http://www.smile.fr>)
# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl).

{
"name": "Database Anonymization",
"version": "0.1",
"depends": [
"mail",
"web",
],
"author": "Smile",
"license": 'AGPL-3',
"description": "",
"summary": "",
"website": "http://www.smile.fr",
"category": 'Tools',
"sequence": 20,
"data": [
"views/ir_model_fields_view.xml",
"views/ir_model_view.xml",
],
"auto_install": True,
"installable": True,
"application": False,
}
5 changes: 5 additions & 0 deletions smile_anonymization/controllers/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# -*- coding: utf-8 -*-
# (C) 2020 Smile (<http://www.smile.fr>)
# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl).

from . import main
22 changes: 22 additions & 0 deletions smile_anonymization/controllers/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# -*- coding: utf-8 -*-
# (C) 2020 Smile (<http://www.smile.fr>)
# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl).

from odoo import http
from odoo.http import request
from odoo.addons.web.controllers.main import Database


class AnoDatabaseController(Database):
@http.route('/web/database/backup', methods=['POST'],
type='http', auth="none", csrf=False)
def backup(self, master_pwd, name, backup_format='zip'):
""" We're later calling odoo.service.db.exp_duplicate_database which
end up closing connections to the current database.
If we don't discard our cursor now, it'll try and commit in
odoo.request.__exit__ and throw an exception. It'd lead to a
corrupt .zip file and not dropping the temp database.
"""
request._cr = None
return super(AnoDatabaseController, self).backup(master_pwd, name,
backup_format)
38 changes: 38 additions & 0 deletions smile_anonymization/i18n/fr.po
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Translation of Odoo Server.
# This file contains the translation of the following modules:
# * smile_anonymization
#
msgid ""
msgstr ""
"Project-Id-Version: Odoo Server 11.0-20180819\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2019-01-09 16:14+0000\n"
"PO-Revision-Date: 2019-01-09 16:14+0000\n"
"Last-Translator: <>\n"
"Language-Team: \n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: \n"
"Plural-Forms: \n"

#. module: smile_anonymization
#: model:ir.model.fields,field_description:smile_anonymization.field_ir_model_fields_data_mask
msgid "Data Mask"
msgstr "Masque de données"

#. module: smile_anonymization
#: model:ir.model,name:smile_anonymization.model_ir_model_fields
msgid "Fields"
msgstr "Champs"

#. module: smile_anonymization
#: code:addons/smile_anonymization/models/ir_model_fields.py:48
#, python-format
msgid "You cannot use '%s' keyword into a data mask"
msgstr "Vous ne pouvez pas utiliser le mot clé '%s' au sein d'un masque"

#. module: smile_anonymization
#: code:addons/smile_anonymization/models/ir_model_fields.py:44
#, python-format
msgid "You cannot use ';' character into a data mask"
msgstr "Vous ne pouvez pas utiliser le caractère ';' au sein d'un masque"
13 changes: 13 additions & 0 deletions smile_anonymization/models/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# -*- coding: utf-8 -*-
# (C) 2019 Smile (<http://www.smile.fr>)
# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl).

from . import ir_attachment
from . import ir_model_fields
from . import mail_mail
from . import mail_message
from . import res_partner_bank
from . import res_partner_title
from . import res_partner
from . import res_users
from . import mail_tracking_value
17 changes: 17 additions & 0 deletions smile_anonymization/models/ir_attachment.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# -*- coding: utf-8 -*-
# (C) 2019 Smile (<http://www.smile.fr>)
# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl).

from odoo import fields, models

_DATA_MASK = "NULL where res_model is not null and res_model != 'ir.ui.view'"


class IrAttachment(models.Model):
_inherit = 'ir.attachment'

name = fields.Char(data_mask="'attachment_' || id::text")
datas_fname = fields.Char(data_mask="'attachment_' || id::text")
description = fields.Text(data_mask="NULL")
res_name = fields.Char(data_mask="'record_' || res_id::text")
db_datas = fields.Binary(data_mask=_DATA_MASK)
111 changes: 111 additions & 0 deletions smile_anonymization/models/ir_model_fields.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
# -*- coding: utf-8 -*-
# (C) 2019 Smile (<http://www.smile.fr>)
# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl).

from odoo import api, fields, models, _
from odoo.exceptions import ValidationError

# PostgreSQL commands list
_UNSAFE_SQL_KEYWORDS = [
'ABORT', 'ALTER',
'BEGIN',
'CALL', 'CHECKPOINT', 'CLOSE', 'CLUSTER', 'COMMIT', 'COPY', 'CREATE',
'DEALLOCATE', 'DECLARE', 'DELETE', 'DISCARD', 'DO', 'DROP',
'END', 'EXECUTE', 'EXPLAIN',
'FETCH',
'GRANT',
'IMPORT', 'INSERT',
'LISTEN', 'LOAD', 'LOCK',
'MOVE',
'PREPARE',
'REASSIGN', 'REFRESH', 'REINDEX', 'RELEASE', 'RESET', 'REVOKE', 'ROLLBACK',
'SAVEPOINT', 'SECURITY', 'SET', 'SHOW', 'START',
'TRUNCATE',
'UNLISTEN', 'UPDATE',
'VACUUM', 'VALUES',
]


class Base(models.AbstractModel):
_inherit = 'base'

def _valid_field_parameter(self, field, name):
return name == 'data_mask' or super()._valid_field_parameter(field, name)


class IrModelFields(models.Model):
_inherit = 'ir.model.fields'

data_mask = fields.Char()
data_mask_locked = fields.Boolean()

@api.constrains('data_mask')
def _check_data_mask(self):

def _format(string):
return " %s " % string.lower()

for field in self:
if field.data_mask:
if ';' in field.data_mask:
raise ValidationError(
_("You cannot use ';' character into a data mask"))
for unsafe_keyword in _UNSAFE_SQL_KEYWORDS:
if _format(unsafe_keyword) in _format(field.data_mask):
raise ValidationError(
_("You cannot use '%s' keyword into a data mask")
% unsafe_keyword)

def _reflect_field_params(self, field, model_id):
vals = super(IrModelFields, self)._reflect_field_params(field, model_id)
vals['data_mask'] = getattr(field, 'data_mask', None)
return vals

def _instanciate_attrs(self, field_data):
attrs = super(IrModelFields, self)._instanciate_attrs(field_data)
if attrs and field_data.get('data_mask'):
attrs['data_mask'] = field_data['data_mask']
return attrs

def toggle_data_mask_locked(self):
for field in self:
field.data_mask_locked = not field.data_mask_locked

_safe_attributes = ['data_mask', 'data_mask_locked']

def write(self, vals):
for attribute in self._safe_attributes:
if attribute in vals:
fields_to_update = self
if attribute == 'data_mask':
fields_to_update = self.filtered(
lambda field: not field.data_mask_locked)
fields_to_update._write({attribute: vals[attribute]})
del vals[attribute]
return super(IrModelFields, self).write(vals)

@api.model
def get_anonymization_query(self):
return self.search([
('data_mask', '!=', False),
('store', '=', True),
])._get_anonymization_query()

def _get_anonymization_query(self):
query = "DELETE FROM ir_attachment WHERE name ilike '/web/content/%'" \
"OR name ilike '%/static/%';\n"
data = {}
for field in self:
if field.data_mask:
if self.env[field.model]._table not in data.keys():
data[self.env[field.model]._table] = [
"UPDATE %s SET %s = %s" % (self.env[field.model]._table, field.name, field.data_mask)]
else:
if 'where'.lower() in field.data_mask.lower():
data[self.env[field.model]._table].append(
"UPDATE %s SET %s = %s" % (self.env[field.model]._table, field.name, field.data_mask))
else:
data[self.env[field.model]._table][0] += ",%s = %s" % (field.name, field.data_mask)
for val in data.values():
query += ";\n".join(val) + ";\n"
return query
13 changes: 13 additions & 0 deletions smile_anonymization/models/mail_mail.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# -*- coding: utf-8 -*-
# (C) 2019 Smile (<http://www.smile.fr>)
# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl).

from odoo import fields, models


class MailMail(models.Model):
_inherit = 'mail.mail'

body_html = fields.Text(data_mask="'mail_' || id::text")
email_to = fields.Text(data_mask="NULL")
email_cc = fields.Char(data_mask="NULL")
16 changes: 16 additions & 0 deletions smile_anonymization/models/mail_message.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# -*- coding: utf-8 -*-
# (C) 2019 Smile (<http://www.smile.fr>)
# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl).

from odoo import fields, models


class MailMessage(models.Model):
_inherit = 'mail.message'

subject = fields.Char(data_mask="'message_' || id::text")
body = fields.Html(data_mask="'message_' || id::text")
record_name = fields.Char(data_mask="model || ',' || res_id::text")
email_from = fields.Char(data_mask="NULL")
reply_to = fields.Char(data_mask="NULL")
message_id = fields.Char(data_mask="NULL")
23 changes: 23 additions & 0 deletions smile_anonymization/models/mail_tracking_value.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# -*- coding: utf-8 -*-
# (C) 2019 Smile (<http://www.smile.fr>)
# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl).

from odoo import fields, models


class MailTracking(models.Model):
_inherit = 'mail.tracking.value'

old_value_integer = fields.Integer(data_mask="NULL")
old_value_float = fields.Float(data_mask="NULL")
old_value_monetary = fields.Float(data_mask="NULL")
old_value_char = fields.Char(data_mask="NULL")
old_value_text = fields.Text(data_mask="NULL")
old_value_datetime = fields.Datetime(data_mask="NULL")

new_value_integer = fields.Integer(data_mask="NULL")
new_value_float = fields.Float(data_mask="NULL")
new_value_monetary = fields.Float(data_mask="NULL")
new_value_char = fields.Char(data_mask="NULL")
new_value_text = fields.Text(data_mask="NULL")
new_value_datetime = fields.Datetime(data_mask="NULL")
Loading

0 comments on commit bf4ad53

Please sign in to comment.