Browse Source

Merge pull request #1190 from Tecnativa/9.0-module_auto_update-refactor

[REF] module_auto_update: Deprecate buggy features, add new API
pull/1199/head
Pedro M. Baeza 7 years ago
committed by GitHub
parent
commit
7d10384bdd
No known key found for this signature in database GPG Key ID: 4AEE18F83AFDEB23
  1. 64
      module_auto_update/README.rst
  2. 2
      module_auto_update/__init__.py
  3. 15
      module_auto_update/__openerp__.py
  4. 45
      module_auto_update/addon_hash.py
  5. 2
      module_auto_update/data/cron_data_deprecated.xml
  6. 16
      module_auto_update/hooks.py
  7. 23
      module_auto_update/migrations/9.0.2.0.0/pre-migrate.py
  8. 1
      module_auto_update/models/__init__.py
  9. 178
      module_auto_update/models/module.py
  10. 69
      module_auto_update/models/module_deprecated.py
  11. 4
      module_auto_update/tests/__init__.py
  12. 1
      module_auto_update/tests/sample_module/README.rst
  13. 1
      module_auto_update/tests/sample_module/data/f1.xml
  14. 1
      module_auto_update/tests/sample_module/data/f2.xml
  15. 1
      module_auto_update/tests/sample_module/i18n/en.po
  16. 1
      module_auto_update/tests/sample_module/i18n/en_US.po
  17. 1
      module_auto_update/tests/sample_module/i18n/fr.po
  18. 1
      module_auto_update/tests/sample_module/i18n/fr_BE.po
  19. 1
      module_auto_update/tests/sample_module/i18n/test.pot
  20. 1
      module_auto_update/tests/sample_module/i18n_extra/en.po
  21. 1
      module_auto_update/tests/sample_module/i18n_extra/fr.po
  22. 1
      module_auto_update/tests/sample_module/i18n_extra/nl_NL.po
  23. 1
      module_auto_update/tests/sample_module/models/stuff.py
  24. BIN
      module_auto_update/tests/sample_module/models/stuff.pyc
  25. 1
      module_auto_update/tests/sample_module/static/src/some.js
  26. 67
      module_auto_update/tests/test_addon_hash.py
  27. 302
      module_auto_update/tests/test_module.py
  28. 172
      module_auto_update/tests/test_module_deprecated.py
  29. 3
      module_auto_update/tests/test_module_upgrade_deprecated.py
  30. 49
      module_auto_update/views/module_views.xml
  31. 2
      module_auto_update/wizards/__init__.py
  32. 51
      module_auto_update/wizards/module_upgrade.py
  33. 85
      module_auto_update/wizards/module_upgrade_deprecated.py
  34. 1
      requirements.txt

64
module_auto_update/README.rst

@ -6,36 +6,69 @@
Module Auto Update
==================
This module will automatically check for and apply module upgrades on a schedule.
Upgrade checking is accomplished by comparing the SHA1 checksums of currently-installed modules to the checksums of corresponding modules in the addons directories.
Installation
============
Prior to installing this module, you need to:
#. Install checksumdir with `pip install checksumdir`
#. Ensure all installed modules are up-to-date. When installed, this module will assume the versions found in the addons directories are currently installed.
This addon provides mechanisms to compute sha1 hashes of installed addons,
and save them in the database. It also provides a method that exploits these
mechanisms to update a database by upgrading only the modules for which the
hash has changed since the last successful upgrade.
Configuration
=============
The default time for checking and applying upgrades is 3:00 AM (UTC). To change this schedule, modify the "Perform Module Upgrades" scheduled action.
This module supports the following system parameters:
This module will ignore .pyc and .pyo file extensions by default. To modify this, create a module_auto_update.checksum_excluded_extensions system parameter with the desired extensions listed as comma-separated values.
* ``module_auto_update.exclude_patterns``: comma-separated list of file
name patterns to ignore when computing addon checksums. Defaults to
``*.pyc,*.pyo,i18n/*.pot,i18n_extra/*.pot,static/*``.
Filename patterns must be compatible with the python ``fnmatch`` function.
In addition to the above pattern, .po files corresponding to languages that
are not installed in the Odoo database are ignored when computing checksums.
Usage
=====
Modules scheduled for upgrade can be viewed by clicking the "Updates" menu item in the Apps sidebar.
The main method provided by this module is ``upgrade_changed_checksum``
on ``ir.module.module``. It runs a database upgrade for all installed
modules for which the hash has changed since the last successful
run of this method. On success it saves the hashes in the database.
The first time this method is invoked after installing the module, it
runs an upgrade of all modules, because it has not saved the hashes yet.
This is by design, priviledging safety. Should this be an issue,
the method ``_save_installed_checksums`` can be invoked in a situation
where one is sure all modules on disk are installed and up-to-date in the
database.
An easy way to invoke this upgrade mechanism is by issuing the following
in an Odoo shell session::
To perform upgrades manually, click the "Apply Scheduled Upgrades" menu item in the Apps sidebar.
env['ir.module.module'].upgrade_changed_checksum()
.. image:: https://odoo-community.org/website/image/ir.attachment/5784_f2813bd/datas
:alt: Try me on Runbot
:target: https://runbot.odoo-community.org/runbot/149/9.0
Known issues / Roadmap
======================
* Since version ``2.0.0``, some features have been deprecated.
When you upgrade from previous versions, these features will be kept for
backwards compatibility, but beware! They are buggy!
If you install this addon from scratch, these features are disabled by
default.
To force enabling or disabling the deprecated features, set a configuration
parameter called ``module_auto_update.enable_deprecated`` to either ``1``
or ``0``. It is recommended that you disable them.
Keep in mind that from this version, all upgrades are assumed to run in a
separate odoo instance, dedicated exclusively to upgrade Odoo.
* When migrating the addon to new versions, the deprecated features should be
removed. To make it simple all deprecated features are found in files
suffixed with ``_deprecated``.
Bug Tracker
===========
@ -58,6 +91,7 @@ Contributors
* Brent Hughes <brent.hughes@laslabs.com>
* Juan José Scarafía <jjs@adhoc.com.ar>
* Jairo Llopis <jairo.llopis@tecnativa.com>
* Stéphane Bidoul <stephane.bidoul@acsone.eu> (https://acsone.eu)
Do not contact contributors directly about support or help with technical issues.

2
module_auto_update/__init__.py

@ -4,4 +4,4 @@
from . import models
from . import wizards
from .hooks import post_init_hook
from .hooks import uninstall_hook

15
module_auto_update/__openerp__.py

@ -5,27 +5,22 @@
{
'name': 'Module Auto Update',
'summary': 'Automatically update Odoo modules',
'version': '9.0.1.0.2',
'version': '9.0.2.0.0',
'category': 'Extra Tools',
'website': 'https://odoo-community.org/',
'website': 'https://github.com/OCA/server-tools',
'author': 'LasLabs, '
'Juan José Scarafía, '
'Tecnativa, '
'ACSONE SA/NV, '
'Odoo Community Association (OCA)',
'license': 'LGPL-3',
'application': False,
'installable': True,
'post_init_hook': 'post_init_hook',
'external_dependencies': {
'python': [
'checksumdir',
],
},
'uninstall_hook': 'uninstall_hook',
'depends': [
'base',
],
'data': [
'views/module_views.xml',
'data/cron_data.xml',
'data/cron_data_deprecated.xml',
],
}

45
module_auto_update/addon_hash.py

@ -0,0 +1,45 @@
# -*- coding: utf-8 -*-
# Copyright 2018 ACSONE SA/NV.
# License LGPL-3.0 or later (http://www.gnu.org/licenses/lgpl).
from fnmatch import fnmatch
import hashlib
import os
def _fnmatch(filename, patterns):
for pattern in patterns:
if fnmatch(filename, pattern):
return True
return False
def _walk(top, exclude_patterns, keep_langs):
keep_langs = {l.split('_')[0] for l in keep_langs}
for dirpath, dirnames, filenames in os.walk(top):
dirnames.sort()
reldir = os.path.relpath(dirpath, top)
if reldir == '.':
reldir = ''
for filename in sorted(filenames):
filepath = os.path.join(reldir, filename)
if _fnmatch(filepath, exclude_patterns):
continue
if keep_langs and reldir in {'i18n', 'i18n_extra'}:
basename, ext = os.path.splitext(filename)
if ext == '.po':
if basename.split('_')[0] not in keep_langs:
continue
yield filepath
def addon_hash(top, exclude_patterns, keep_langs):
"""Compute a sha1 digest of file contents."""
m = hashlib.sha1()
for filepath in _walk(top, exclude_patterns, keep_langs):
# hash filename so empty files influence the hash
m.update(filepath.encode('utf-8'))
# hash file content
with open(os.path.join(top, filepath), 'rb') as f:
m.update(f.read())
return m.hexdigest()

2
module_auto_update/data/cron_data.xml → module_auto_update/data/cron_data_deprecated.xml

@ -2,7 +2,7 @@
<odoo noupdate="1">
<record model="ir.cron" id="module_check_upgrades_cron">
<field name="name">Perform Module Upgrades</field>
<field name="active" eval="True"/>
<field name="active" eval="False"/>
<field name="user_id" ref="base.user_root"/>
<field name="interval_number">1</field>
<field name="interval_type">days</field>

16
module_auto_update/hooks.py

@ -4,11 +4,15 @@
from openerp import SUPERUSER_ID, api
from .models.module import PARAM_INSTALLED_CHECKSUMS
from .models.module_deprecated import PARAM_DEPRECATED
def post_init_hook(cr, registry):
def uninstall_hook(cr, registry):
env = api.Environment(cr, SUPERUSER_ID, {})
installed_modules = env['ir.module.module'].search([
('state', '=', 'installed'),
])
for r in installed_modules:
r.checksum_installed = r.checksum_dir
env["ir.config_parameter"].set_param(PARAM_INSTALLED_CHECKSUMS, False)
# TODO Remove from here when removing deprecated features
env["ir.config_parameter"].set_param(PARAM_DEPRECATED, False)
prefix = "module_auto_update.field_ir_module_module_checksum_%s"
fields = env.ref(prefix % "dir") | env.ref(prefix % "installed")
fields.with_context(_force_unlink=True).unlink()

23
module_auto_update/migrations/9.0.2.0.0/pre-migrate.py

@ -0,0 +1,23 @@
# -*- coding: utf-8 -*-
# Copyright 2018 Tecnativa - Jairo Llopis
# License LGPL-3.0 or later (https://www.gnu.org/licenses/lgpl).
import logging
from psycopg2 import IntegrityError
from openerp.addons.module_auto_update.models.module_deprecated import \
PARAM_DEPRECATED
_logger = logging.getLogger(__name__)
def migrate(cr, version):
"""Autoenable deprecated behavior."""
try:
cr.execute(
"INSERT INTO ir_config_parameter (key, value) VALUES (%s, '1')",
(PARAM_DEPRECATED,)
)
_logger.warn("Deprecated features have been autoenabled, see "
"addon's README to know how to upgrade to the new "
"supported autoupdate mechanism.")
except IntegrityError:
_logger.info("Deprecated features setting exists, not autoenabling")

1
module_auto_update/models/__init__.py

@ -3,3 +3,4 @@
# License LGPL-3.0 or later (http://www.gnu.org/licenses/lgpl).
from . import module
from . import module_deprecated

178
module_auto_update/models/module.py

@ -1,70 +1,148 @@
# -*- coding: utf-8 -*-
# Copyright 2017 LasLabs Inc.
# Copyright 2018 ACSONE SA/NV.
# License LGPL-3.0 or later (http://www.gnu.org/licenses/lgpl).
import json
import logging
import os
from openerp import api, fields, models
from openerp import api, exceptions, models, tools
from openerp.modules.module import get_module_path
from ..addon_hash import addon_hash
PARAM_INSTALLED_CHECKSUMS = \
'module_auto_update.installed_checksums'
PARAM_EXCLUDE_PATTERNS = \
'module_auto_update.exclude_patterns'
DEFAULT_EXCLUDE_PATTERNS = \
'*.pyc,*.pyo,i18n/*.pot,i18n_extra/*.pot,static/*'
_logger = logging.getLogger(__name__)
try:
from checksumdir import dirhash
except ImportError:
_logger.debug('Cannot `import checksumdir`.')
class IncompleteUpgradeError(exceptions.UserError):
pass
class Module(models.Model):
_inherit = 'ir.module.module'
checksum_dir = fields.Char(
compute='_compute_checksum_dir',
)
checksum_installed = fields.Char()
@api.depends('name')
def _compute_checksum_dir(self):
exclude = self.env["ir.config_parameter"].get_param(
"module_auto_update.checksum_excluded_extensions",
"pyc,pyo",
).split(",")
for r in self:
try:
r.checksum_dir = dirhash(
get_module_path(r.name),
'sha1',
excluded_extensions=exclude,
@api.multi
def _get_checksum_dir(self):
self.ensure_one()
exclude_patterns = self.env["ir.config_parameter"].get_param(
PARAM_EXCLUDE_PATTERNS,
DEFAULT_EXCLUDE_PATTERNS,
)
except TypeError:
_logger.debug(
"Cannot compute dir hash for %s, module not found",
r.display_name)
exclude_patterns = [p.strip() for p in exclude_patterns.split(',')]
keep_langs = self.env['res.lang'].search([]).mapped('code')
@api.multi
def _store_checksum_installed(self, vals):
"""Store the right installed checksum, if addon is installed."""
if 'checksum_installed' not in vals:
try:
version = vals["latest_version"]
except KeyError:
return # Not [un]installing/updating any addon
if version is False:
# Uninstalling
self.write({'checksum_installed': False})
module_path = get_module_path(self.name)
if module_path and os.path.isdir(module_path):
checksum_dir = addon_hash(
module_path,
exclude_patterns,
keep_langs,
)
else:
# Installing or updating
for one in self:
one.checksum_installed = one.checksum_dir
checksum_dir = False
return checksum_dir
@api.model
def create(self, vals):
res = super(Module, self).create(vals)
res._store_checksum_installed(vals)
return res
def _get_saved_checksums(self):
Icp = self.env['ir.config_parameter']
return json.loads(Icp.get_param(PARAM_INSTALLED_CHECKSUMS, '{}'))
@api.multi
def write(self, vals):
res = super(Module, self).write(vals)
self._store_checksum_installed(vals)
return res
@api.model
def _save_checksums(self, checksums):
Icp = self.env['ir.config_parameter']
Icp.set_param(PARAM_INSTALLED_CHECKSUMS, json.dumps(checksums))
@api.model
def _save_installed_checksums(self):
checksums = {}
installed_modules = self.search([('state', '=', 'installed')])
for module in installed_modules:
checksums[module.name] = module._get_checksum_dir()
self._save_checksums(checksums)
@api.model
def _get_modules_partially_installed(self):
return self.search([
('state', 'in', ('to install', 'to remove', 'to upgrade')),
])
@api.model
def _get_modules_with_changed_checksum(self):
saved_checksums = self._get_saved_checksums()
installed_modules = self.search([('state', '=', 'installed')])
return installed_modules.filtered(
lambda r: r._get_checksum_dir() != saved_checksums.get(r.name),
)
@api.model
def upgrade_changed_checksum(self, overwrite_existing_translations=False):
"""Run an upgrade of the database, upgrading only changed modules.
Installed modules for which the checksum has changed since the
last successful run of this method are marked "to upgrade",
then the normal Odoo scheduled upgrade process
is launched.
If there is no module with a changed checksum, and no module in state
other than installed, uninstalled, uninstallable, this method does
nothing, otherwise the normal Odoo upgrade process is launched.
After a successful upgrade, the checksums of installed modules are
saved.
In case of error during the upgrade, an exception is raised.
If any module remains to upgrade or to uninstall after the upgrade
process, an exception is raised as well.
Note: this method commits the current transaction at each important
step, it is therefore not intended to be run as part of a
larger transaction.
"""
_logger.info(
"Checksum upgrade starting (i18n-overwrite=%s)...",
overwrite_existing_translations
)
tools.config['overwrite_existing_translations'] = \
overwrite_existing_translations
_logger.info("Updating modules list...")
self.update_list()
changed_modules = self._get_modules_with_changed_checksum()
if not changed_modules and not self._get_modules_partially_installed():
_logger.info("No checksum change detected in installed modules "
"and all modules installed, nothing to do.")
return
_logger.info("Marking the following modules to upgrade, "
"for their checksums changed: %s...",
','.join(changed_modules.mapped('name')))
changed_modules.button_upgrade()
self.env.cr.commit() # pylint: disable=invalid-commit
_logger.info("Upgrading...")
self.env['base.module.upgrade'].upgrade_module()
self.env.cr.commit() # pylint: disable=invalid-commit
_logger.info("Upgrade successful, updating checksums...")
self._save_installed_checksums()
self.env.cr.commit() # pylint: disable=invalid-commit
partial_modules = self._get_modules_partially_installed()
if partial_modules:
raise IncompleteUpgradeError(
"Checksum upgrade successful "
"but incomplete for the following modules: %s" %
','.join(partial_modules.mapped('name'))
)
_logger.info("Checksum upgrade complete.")

69
module_auto_update/models/module_deprecated.py

@ -0,0 +1,69 @@
# -*- coding: utf-8 -*-
# Copyright 2017 LasLabs Inc.
# License LGPL-3.0 or later (http://www.gnu.org/licenses/lgpl).
from openerp import api, fields, models
PARAM_DEPRECATED = "module_auto_update.enable_deprecated"
class Module(models.Model):
_inherit = 'ir.module.module'
checksum_dir = fields.Char(
deprecated=True,
compute='_compute_checksum_dir',
)
checksum_installed = fields.Char(
deprecated=True,
compute='_compute_checksum_installed',
inverse='_inverse_checksum_installed',
store=False,
)
@api.depends('name')
def _compute_checksum_dir(self):
for rec in self:
rec.checksum_dir = rec._get_checksum_dir()
def _compute_checksum_installed(self):
saved_checksums = self._get_saved_checksums()
for rec in self:
rec.checksum_installed = saved_checksums.get(rec.name, False)
def _inverse_checksum_installed(self):
checksums = self._get_saved_checksums()
for rec in self:
checksums[rec.name] = rec.checksum_installed
self._save_checksums(checksums)
@api.multi
def _store_checksum_installed(self, vals):
"""Store the right installed checksum, if addon is installed."""
if not self.env["base.module.upgrade"]._autoupdate_deprecated():
# Skip if deprecated features are disabled
return
if 'checksum_installed' not in vals:
try:
version = vals["latest_version"]
except KeyError:
return # Not [un]installing/updating any addon
if version is False:
# Uninstalling
self.write({'checksum_installed': False})
else:
# Installing or updating
for one in self:
one.checksum_installed = one.checksum_dir
@api.model
def create(self, vals):
res = super(Module, self).create(vals)
res._store_checksum_installed(vals)
return res
@api.multi
def write(self, vals):
res = super(Module, self).write(vals)
self._store_checksum_installed(vals)
return res

4
module_auto_update/tests/__init__.py

@ -2,5 +2,7 @@
# Copyright 2017 LasLabs Inc.
# License LGPL-3.0 or later (http://www.gnu.org/licenses/lgpl).
from . import test_addon_hash
from . import test_module
from . import test_module_upgrade
from . import test_module_deprecated
from . import test_module_upgrade_deprecated

1
module_auto_update/tests/sample_module/README.rst

@ -0,0 +1 @@
Test data for addon_hash module.

1
module_auto_update/tests/sample_module/data/f1.xml

@ -0,0 +1 @@
<odoo/>

1
module_auto_update/tests/sample_module/data/f2.xml

@ -0,0 +1 @@
<odoo/>

1
module_auto_update/tests/sample_module/i18n/en.po

@ -0,0 +1 @@
en text

1
module_auto_update/tests/sample_module/i18n/en_US.po

@ -0,0 +1 @@
en_US

1
module_auto_update/tests/sample_module/i18n/fr.po

@ -0,0 +1 @@
fr

1
module_auto_update/tests/sample_module/i18n/fr_BE.po

@ -0,0 +1 @@
fr_BE

1
module_auto_update/tests/sample_module/i18n/test.pot

@ -0,0 +1 @@
...

1
module_auto_update/tests/sample_module/i18n_extra/en.po

@ -0,0 +1 @@
en

1
module_auto_update/tests/sample_module/i18n_extra/fr.po

@ -0,0 +1 @@
fr

1
module_auto_update/tests/sample_module/i18n_extra/nl_NL.po

@ -0,0 +1 @@
nl_NL

1
module_auto_update/tests/sample_module/models/stuff.py

@ -0,0 +1 @@
1+1

BIN
module_auto_update/tests/sample_module/models/stuff.pyc

1
module_auto_update/tests/sample_module/static/src/some.js

@ -0,0 +1 @@
/* javascript */

67
module_auto_update/tests/test_addon_hash.py

@ -0,0 +1,67 @@
# -*- coding: utf-8 -*-
# Copyright 2018 ACSONE SA/NV.
# License LGPL-3.0 or later (http://www.gnu.org/licenses/lgpl).
import os
import unittest
from .. import addon_hash
from ..models.module import DEFAULT_EXCLUDE_PATTERNS
class TestAddonHash(unittest.TestCase):
def setUp(self):
super(TestAddonHash, self).setUp()
self.sample_dir = os.path.join(
os.path.dirname(__file__),
'sample_module',
)
def test_basic(self):
files = list(addon_hash._walk(
self.sample_dir,
exclude_patterns=[],
keep_langs=[],
))
self.assertEqual(files, [
'README.rst',
'data/f1.xml',
'data/f2.xml',
'i18n/en.po',
'i18n/en_US.po',
'i18n/fr.po',
'i18n/fr_BE.po',
'i18n/test.pot',
'i18n_extra/en.po',
'i18n_extra/fr.po',
'i18n_extra/nl_NL.po',
'models/stuff.py',
'models/stuff.pyc',
'static/src/some.js',
])
def test_exclude(self):
files = list(addon_hash._walk(
self.sample_dir,
exclude_patterns=DEFAULT_EXCLUDE_PATTERNS.split(','),
keep_langs=['fr_FR', 'nl'],
))
self.assertEqual(files, [
'README.rst',
'data/f1.xml',
'data/f2.xml',
'i18n/fr.po',
'i18n/fr_BE.po',
'i18n_extra/fr.po',
'i18n_extra/nl_NL.po',
'models/stuff.py',
])
def test2(self):
checksum = addon_hash.addon_hash(
self.sample_dir,
exclude_patterns=['*.pyc', '*.pyo', '*.pot', 'static/*'],
keep_langs=['fr_FR', 'nl'],
)
self.assertEqual(checksum, 'fecb89486c8a29d1f760cbd01c1950f6e8421b14')

302
module_auto_update/tests/test_module.py

@ -1,54 +1,43 @@
# -*- coding: utf-8 -*-
# Copyright 2017 LasLabs Inc.
# Copyright 2018 ACSONE SA/NV.
# License LGPL-3.0 or later (http://www.gnu.org/licenses/lgpl).
import logging
import os
import tempfile
import mock
from openerp.modules import get_module_path
from openerp.tests import common
from openerp.tests.common import TransactionCase
from openerp.tools import mute_logger
from .. import post_init_hook
from ..addon_hash import addon_hash
from ..models.module import IncompleteUpgradeError, DEFAULT_EXCLUDE_PATTERNS
_logger = logging.getLogger(__name__)
try:
from checksumdir import dirhash
except ImportError:
_logger.debug('Cannot `import checksumdir`.')
model = 'openerp.addons.module_auto_update.models.module'
MODULE_NAME = 'module_auto_update'
class TestModule(TransactionCase):
def setUp(self):
super(TestModule, self).setUp()
module_name = 'module_auto_update'
self.own_module = self.env['ir.module.module'].search([
('name', '=', module_name),
('name', '=', MODULE_NAME),
])
self.own_dir_path = get_module_path(module_name)
self.own_checksum = dirhash(
self.own_dir_path = get_module_path(MODULE_NAME)
keep_langs = self.env['res.lang'].search([]).mapped('code')
self.own_checksum = addon_hash(
self.own_dir_path,
'sha1',
excluded_extensions=['pyc', 'pyo'],
exclude_patterns=DEFAULT_EXCLUDE_PATTERNS.split(','),
keep_langs=keep_langs,
)
self.own_writeable = os.access(self.own_dir_path, os.W_OK)
@mock.patch('%s.get_module_path' % model)
def create_test_module(self, vals, get_module_path_mock):
get_module_path_mock.return_value = self.own_dir_path
test_module = self.env['ir.module.module'].create(vals)
return test_module
def test_compute_checksum_dir(self):
"""It should compute the directory's SHA-1 hash"""
self.assertEqual(
self.own_module.checksum_dir, self.own_checksum,
self.own_module._get_checksum_dir(), self.own_checksum,
'Module directory checksum not computed properly',
)
@ -57,10 +46,9 @@ class TestModule(TransactionCase):
calculations"""
if not self.own_writeable:
self.skipTest("Own directory not writeable")
with tempfile.NamedTemporaryFile(
suffix='.pyc', dir=self.own_dir_path):
with tempfile.NamedTemporaryFile(suffix='.pyc', dir=self.own_dir_path):
self.assertEqual(
self.own_module.checksum_dir, self.own_checksum,
self.own_module._get_checksum_dir(), self.own_checksum,
'SHA1 checksum does not ignore excluded extensions',
)
@ -69,149 +57,147 @@ class TestModule(TransactionCase):
added to the module directory"""
if not self.own_writeable:
self.skipTest("Own directory not writeable")
with tempfile.NamedTemporaryFile(
suffix='.py', dir=self.own_dir_path):
with tempfile.NamedTemporaryFile(suffix='.py', dir=self.own_dir_path):
self.assertNotEqual(
self.own_module.checksum_dir, self.own_checksum,
self.own_module._get_checksum_dir(), self.own_checksum,
'SHA1 checksum not recomputed',
)
def test_store_checksum_installed_state_installed(self):
"""It should set the module's checksum_installed equal to
checksum_dir when vals contain a ``latest_version`` str."""
self.own_module.checksum_installed = 'test'
self.own_module._store_checksum_installed({'latest_version': '1.0'})
self.assertEqual(
self.own_module.checksum_installed, self.own_module.checksum_dir,
)
def test_saved_checksums(self):
Imm = self.env['ir.module.module']
base_module = Imm.search([('name', '=', 'base')])
self.assertEqual(base_module.state, 'installed')
self.assertFalse(Imm._get_saved_checksums())
Imm._save_installed_checksums()
saved_checksums = Imm._get_saved_checksums()
self.assertTrue(saved_checksums)
self.assertTrue(saved_checksums['base'])
def test_store_checksum_installed_state_uninstalled(self):
"""It should clear the module's checksum_installed when vals
contain ``"latest_version": False``"""
self.own_module.checksum_installed = 'test'
self.own_module._store_checksum_installed({'latest_version': False})
self.assertIs(self.own_module.checksum_installed, False)
def test_store_checksum_installed_vals_contain_checksum_installed(self):
"""It should not set checksum_installed to False or checksum_dir when
a checksum_installed is included in vals"""
self.own_module.checksum_installed = 'test'
self.own_module._store_checksum_installed({
'state': 'installed',
'checksum_installed': 'test',
})
self.assertEqual(
self.own_module.checksum_installed, 'test',
'Providing checksum_installed in vals did not prevent overwrite',
)
def test_get_modules_with_changed_checksum(self):
Imm = self.env['ir.module.module']
self.assertTrue(Imm._get_modules_with_changed_checksum())
Imm._save_installed_checksums()
self.assertFalse(Imm._get_modules_with_changed_checksum())
def test_store_checksum_installed_with_retain_context(self):
"""It should not set checksum_installed to False or checksum_dir when
self has context retain_checksum_installed=True"""
self.own_module.checksum_installed = 'test'
self.own_module.with_context(
retain_checksum_installed=True,
)._store_checksum_installed({'state': 'installed'})
self.assertEqual(
self.own_module.checksum_installed, 'test',
'Providing retain_checksum_installed context did not prevent '
'overwrite',
)
def test_button_uninstall_cancel(self):
"""It should preserve checksum_installed when cancelling uninstall"""
self.own_module.write({'state': 'to remove'})
self.own_module.checksum_installed = 'test'
self.own_module.button_uninstall_cancel()
self.assertEqual(
self.own_module.checksum_installed, 'test',
'Uninstall cancellation does not preserve checksum_installed',
)
@common.at_install(False)
@common.post_install(True)
class TestModuleAfterInstall(TransactionCase):
def test_button_upgrade_cancel(self):
"""It should preserve checksum_installed when cancelling upgrades"""
self.own_module.write({'state': 'to upgrade'})
self.own_module.checksum_installed = 'test'
def setUp(self):
super(TestModuleAfterInstall, self).setUp()
Imm = self.env['ir.module.module']
self.own_module = Imm.search([('name', '=', MODULE_NAME)])
self.base_module = Imm.search([('name', '=', 'base')])
def test_get_modules_partially_installed(self):
Imm = self.env['ir.module.module']
self.assertTrue(
self.own_module not in Imm._get_modules_partially_installed())
self.own_module.button_upgrade()
self.assertTrue(
self.own_module in Imm._get_modules_partially_installed())
self.own_module.button_upgrade_cancel()
self.assertEqual(
self.own_module.checksum_installed, 'test',
'Upgrade cancellation does not preserve checksum_installed',
)
self.assertTrue(
self.own_module not in Imm._get_modules_partially_installed())
def test_upgrade_changed_checksum(self):
Imm = self.env['ir.module.module']
Bmu = self.env['base.module.upgrade']
# check modules are in installed state
installed_modules = Imm.search([('state', '=', 'installed')])
self.assertTrue(self.own_module in installed_modules)
self.assertTrue(self.base_module in installed_modules)
self.assertTrue(len(installed_modules) > 2)
# change the checksum of 'base'
Imm._save_installed_checksums()
saved_checksums = Imm._get_saved_checksums()
saved_checksums['base'] = False
Imm._save_checksums(saved_checksums)
changed_modules = Imm._get_modules_with_changed_checksum()
self.assertEqual(len(changed_modules), 1)
self.assertTrue(self.base_module in changed_modules)
def upgrade_module_mock(self_model):
upgrade_module_mock.call_count += 1
# since we are upgrading base, all installed module
# must have been marked to upgrade at this stage
self.assertEqual(self.base_module.state, 'to upgrade')
self.assertEqual(self.own_module.state, 'to upgrade')
installed_modules.write({'state': 'installed'})
upgrade_module_mock.call_count = 0
# upgrade_changed_checksum commits, so mock that
with mock.patch.object(self.env.cr, 'commit'):
# we simulate an install by setting module states
Bmu._patch_method('upgrade_module', upgrade_module_mock)
try:
Imm.upgrade_changed_checksum()
self.assertEqual(upgrade_module_mock.call_count, 1)
self.assertEqual(self.base_module.state, 'installed')
self.assertEqual(self.own_module.state, 'installed')
saved_checksums = Imm._get_saved_checksums()
self.assertTrue(saved_checksums['base'])
self.assertTrue(saved_checksums[MODULE_NAME])
finally:
Bmu._revert_method('upgrade_module')
def test_incomplete_upgrade(self):
Imm = self.env['ir.module.module']
Bmu = self.env['base.module.upgrade']
installed_modules = Imm.search([('state', '=', 'installed')])
# change the checksum of 'base'
Imm._save_installed_checksums()
saved_checksums = Imm._get_saved_checksums()
saved_checksums['base'] = False
Imm._save_checksums(saved_checksums)
def upgrade_module_mock(self_model):
upgrade_module_mock.call_count += 1
# since we are upgrading base, all installed module
# must have been marked to upgrade at this stage
self.assertEqual(self.base_module.state, 'to upgrade')
self.assertEqual(self.own_module.state, 'to upgrade')
installed_modules.write({'state': 'installed'})
# simulate partial upgrade
self.own_module.write({'state': 'to upgrade'})
def test_create(self):
"""It should call _store_checksum_installed method"""
_store_checksum_installed_mock = mock.MagicMock()
self.env['ir.module.module']._patch_method(
'_store_checksum_installed',
_store_checksum_installed_mock,
)
vals = {
'name': 'module_auto_update_test_module',
'state': 'installed',
}
self.create_test_module(vals)
_store_checksum_installed_mock.assert_called_once_with(vals)
self.env['ir.module.module']._revert_method(
'_store_checksum_installed',
)
upgrade_module_mock.call_count = 0
@mute_logger("openerp.modules.module")
@mock.patch('%s.get_module_path' % model)
def test_get_module_list(self, module_path_mock):
"""It should change the state of modules with different
checksum_dir and checksum_installed to 'to upgrade'"""
module_path_mock.return_value = self.own_dir_path
vals = {
'name': 'module_auto_update_test_module',
'state': 'installed',
}
test_module = self.create_test_module(vals)
test_module.checksum_installed = 'test'
self.env['base.module.upgrade'].get_module_list()
self.assertEqual(
test_module.state, 'to upgrade',
'List update does not mark upgradeable modules "to upgrade"',
)
# upgrade_changed_checksum commits, so mock that
with mock.patch.object(self.env.cr, 'commit'):
@mock.patch('%s.get_module_path' % model)
def test_get_module_list_only_changes_installed(self, module_path_mock):
"""It should not change the state of a module with a former state
other than 'installed' to 'to upgrade'"""
module_path_mock.return_value = self.own_dir_path
vals = {
'name': 'module_auto_update_test_module',
'state': 'uninstalled',
}
test_module = self.create_test_module(vals)
self.env['base.module.upgrade'].get_module_list()
self.assertNotEqual(
test_module.state, 'to upgrade',
'List update changed state of an uninstalled module',
)
# we simulate an install by setting module states
Bmu._patch_method('upgrade_module', upgrade_module_mock)
try:
with self.assertRaises(IncompleteUpgradeError):
Imm.upgrade_changed_checksum()
self.assertEqual(upgrade_module_mock.call_count, 1)
finally:
Bmu._revert_method('upgrade_module')
def test_write(self):
"""It should call _store_checksum_installed method"""
_store_checksum_installed_mock = mock.MagicMock()
self.env['ir.module.module']._patch_method(
'_store_checksum_installed',
_store_checksum_installed_mock,
)
vals = {'state': 'installed'}
self.own_module.write(vals)
_store_checksum_installed_mock.assert_called_once_with(vals)
self.env['ir.module.module']._revert_method(
'_store_checksum_installed',
)
def test_nothing_to_upgrade(self):
Imm = self.env['ir.module.module']
Bmu = self.env['base.module.upgrade']
def test_post_init_hook(self):
"""It should set checksum_installed equal to checksum_dir for all
installed modules"""
installed_modules = self.env['ir.module.module'].search([
('state', '=', 'installed'),
])
post_init_hook(self.env.cr, None)
self.assertListEqual(
installed_modules.mapped('checksum_dir'),
installed_modules.mapped('checksum_installed'),
'Installed modules did not have checksum_installed stored',
)
Imm._save_installed_checksums()
def upgrade_module_mock(self_model):
upgrade_module_mock.call_count += 1
upgrade_module_mock.call_count = 0
# upgrade_changed_checksum commits, so mock that
with mock.patch.object(self.env.cr, 'commit'):
# we simulate an install by setting module states
Bmu._patch_method('upgrade_module', upgrade_module_mock)
try:
Imm.upgrade_changed_checksum()
self.assertEqual(upgrade_module_mock.call_count, 0)
finally:
Bmu._revert_method('upgrade_module')

172
module_auto_update/tests/test_module_deprecated.py

@ -0,0 +1,172 @@
# -*- coding: utf-8 -*-
# Copyright 2017 LasLabs Inc.
# License LGPL-3.0 or later (http://www.gnu.org/licenses/lgpl).
import os
import mock
from openerp.modules import get_module_path
from openerp.tests.common import TransactionCase
from openerp.tools import mute_logger
from openerp.addons.module_auto_update.addon_hash import addon_hash
from ..models.module_deprecated import PARAM_DEPRECATED
model = 'openerp.addons.module_auto_update.models.module'
class TestModule(TransactionCase):
def setUp(self):
super(TestModule, self).setUp()
module_name = 'module_auto_update'
self.env["ir.config_parameter"].set_param(PARAM_DEPRECATED, "1")
self.own_module = self.env['ir.module.module'].search([
('name', '=', module_name),
])
self.own_dir_path = get_module_path(module_name)
keep_langs = self.env['res.lang'].search([]).mapped('code')
self.own_checksum = addon_hash(
self.own_dir_path,
exclude_patterns=['*.pyc', '*.pyo', '*.pot', 'static/*'],
keep_langs=keep_langs,
)
self.own_writeable = os.access(self.own_dir_path, os.W_OK)
@mock.patch('%s.get_module_path' % model)
def create_test_module(self, vals, get_module_path_mock):
get_module_path_mock.return_value = self.own_dir_path
test_module = self.env['ir.module.module'].create(vals)
return test_module
def test_store_checksum_installed_state_installed(self):
"""It should set the module's checksum_installed equal to
checksum_dir when vals contain a ``latest_version`` str."""
self.own_module.checksum_installed = 'test'
self.own_module._store_checksum_installed({'latest_version': '1.0'})
self.assertEqual(
self.own_module.checksum_installed, self.own_module.checksum_dir,
)
def test_store_checksum_installed_state_uninstalled(self):
"""It should clear the module's checksum_installed when vals
contain ``"latest_version": False``"""
self.own_module.checksum_installed = 'test'
self.own_module._store_checksum_installed({'latest_version': False})
self.assertIs(self.own_module.checksum_installed, False)
def test_store_checksum_installed_vals_contain_checksum_installed(self):
"""It should not set checksum_installed to False or checksum_dir when
a checksum_installed is included in vals"""
self.own_module.checksum_installed = 'test'
self.own_module._store_checksum_installed({
'state': 'installed',
'checksum_installed': 'test',
})
self.assertEqual(
self.own_module.checksum_installed, 'test',
'Providing checksum_installed in vals did not prevent overwrite',
)
def test_store_checksum_installed_with_retain_context(self):
"""It should not set checksum_installed to False or checksum_dir when
self has context retain_checksum_installed=True"""
self.own_module.checksum_installed = 'test'
self.own_module.with_context(
retain_checksum_installed=True,
)._store_checksum_installed({'state': 'installed'})
self.assertEqual(
self.own_module.checksum_installed, 'test',
'Providing retain_checksum_installed context did not prevent '
'overwrite',
)
def test_button_uninstall_cancel(self):
"""It should preserve checksum_installed when cancelling uninstall"""
self.own_module.write({'state': 'to remove'})
self.own_module.checksum_installed = 'test'
self.own_module.button_uninstall_cancel()
self.assertEqual(
self.own_module.checksum_installed, 'test',
'Uninstall cancellation does not preserve checksum_installed',
)
def test_button_upgrade_cancel(self):
"""It should preserve checksum_installed when cancelling upgrades"""
self.own_module.write({'state': 'to upgrade'})
self.own_module.checksum_installed = 'test'
self.own_module.button_upgrade_cancel()
self.assertEqual(
self.own_module.checksum_installed, 'test',
'Upgrade cancellation does not preserve checksum_installed',
)
def test_create(self):
"""It should call _store_checksum_installed method"""
_store_checksum_installed_mock = mock.MagicMock()
try:
self.env['ir.module.module']._patch_method(
'_store_checksum_installed',
_store_checksum_installed_mock,
)
vals = {
'name': 'module_auto_update_test_module',
'state': 'installed',
}
self.create_test_module(vals)
_store_checksum_installed_mock.assert_called_once_with(vals)
finally:
self.env['ir.module.module']._revert_method(
'_store_checksum_installed',
)
@mute_logger("openerp.modules.module")
@mock.patch('%s.get_module_path' % model)
def test_get_module_list(self, module_path_mock):
"""It should change the state of modules with different
checksum_dir and checksum_installed to 'to upgrade'"""
module_path_mock.return_value = self.own_dir_path
vals = {
'name': 'module_auto_update_test_module',
'state': 'installed',
}
test_module = self.create_test_module(vals)
test_module.checksum_installed = 'test'
self.env['base.module.upgrade'].get_module_list()
self.assertEqual(
test_module.state, 'to upgrade',
'List update does not mark upgradeable modules "to upgrade"',
)
@mock.patch('%s.get_module_path' % model)
def test_get_module_list_only_changes_installed(self, module_path_mock):
"""It should not change the state of a module with a former state
other than 'installed' to 'to upgrade'"""
module_path_mock.return_value = self.own_dir_path
vals = {
'name': 'module_auto_update_test_module',
'state': 'uninstalled',
}
test_module = self.create_test_module(vals)
self.env['base.module.upgrade'].get_module_list()
self.assertNotEqual(
test_module.state, 'to upgrade',
'List update changed state of an uninstalled module',
)
def test_write(self):
"""It should call _store_checksum_installed method"""
_store_checksum_installed_mock = mock.MagicMock()
self.env['ir.module.module']._patch_method(
'_store_checksum_installed',
_store_checksum_installed_mock,
)
vals = {'state': 'installed'}
self.own_module.write(vals)
_store_checksum_installed_mock.assert_called_once_with(vals)
self.env['ir.module.module']._revert_method(
'_store_checksum_installed',
)

3
module_auto_update/tests/test_module_upgrade.py → module_auto_update/tests/test_module_upgrade_deprecated.py

@ -8,12 +8,15 @@ from openerp.modules import get_module_path
from openerp.modules.registry import RegistryManager
from openerp.tests.common import TransactionCase
from ..models.module_deprecated import PARAM_DEPRECATED
class TestModuleUpgrade(TransactionCase):
def setUp(self):
super(TestModuleUpgrade, self).setUp()
module_name = 'module_auto_update'
self.env["ir.config_parameter"].set_param(PARAM_DEPRECATED, "1")
self.own_module = self.env['ir.module.module'].search([
('name', '=', module_name),
])

49
module_auto_update/views/module_views.xml

@ -1,49 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<odoo>
<!-- Module Search View -->
<record id="module_view_search" model="ir.ui.view">
<field name="name">updates.module.search</field>
<field name="model">ir.module.module</field>
<field name="inherit_id" ref="base.view_module_filter"/>
<field name="arch" type="xml">
<field name="category_id" position="after">
<filter name="scheduled_upgrades" string="Scheduled Upgrades" domain="[('state', '=', 'to upgrade')]"/>
</field>
</field>
</record>
<!--Open Updates Action (updates apps list first)-->
<record id="module_action_open_updates" model="ir.actions.server">
<field name="name">Open Updates and Update Apps List Server Action</field>
<field name="model_id" ref="model_ir_module_module"/>
<field name="code">
if model.update_list():
action = {
'name': 'Updates',
'type': 'ir.actions.act_window',
'res_model': 'ir.module.module',
'view_type': 'form',
'view_mode': 'tree,form',
'target': 'main',
'context': '{"search_default_scheduled_upgrades": 1}',
}
</field>
</record>
<!--Apps / Updates menu item-->
<menuitem
name="Updates"
action="module_action_open_updates"
id="module_menu_updates"
groups="base.group_no_one"
parent="base.menu_management"
sequence="20"/>
<!-- Menu in Settings > Technical for standard Updates link -->
<menuitem parent="base.menu_custom" sequence="27" name="Modules" id="menu_default_modules"/>
<!-- Moved standard Updates link -->
<record model="ir.ui.menu" id="base.menu_module_updates">
<field name="parent_id" ref="menu_default_modules"/>
</record>
</odoo>

2
module_auto_update/wizards/__init__.py

@ -2,4 +2,4 @@
# Copyright 2017 LasLabs Inc.
# License LGPL-3.0 or later (http://www.gnu.org/licenses/lgpl).
from . import module_upgrade
from . import module_upgrade_deprecated

51
module_auto_update/wizards/module_upgrade.py

@ -1,51 +0,0 @@
# -*- coding: utf-8 -*-
# Copyright 2017 LasLabs Inc.
# License LGPL-3.0 or later (http://www.gnu.org/licenses/lgpl).
from openerp import api, models
class ModuleUpgrade(models.TransientModel):
_inherit = 'base.module.upgrade'
@api.model
def get_module_list(self):
"""Set modules to upgrade searching by their dir checksum."""
Module = self.env["ir.module.module"]
installed_modules = Module.search([('state', '=', 'installed')])
upgradeable_modules = installed_modules.filtered(
lambda r: r.checksum_dir != r.checksum_installed,
)
upgradeable_modules.button_upgrade()
return super(ModuleUpgrade, self).get_module_list()
@api.multi
def upgrade_module(self):
"""Make a fully automated addon upgrade."""
# Compute updates by checksum when called in @api.model fashion
if not self:
self.get_module_list()
Module = self.env["ir.module.module"]
# Get every addon state before updating
pre_states = {addon["name"]: addon["state"]
for addon in Module.search_read([], ["name", "state"])}
# Perform upgrades, possibly in a limited graph that excludes me
self.env.cr.autocommit(True) # Avoid transaction lock
result = super(ModuleUpgrade, self).upgrade_module()
self.env.cr.autocommit(False)
# Reload environments, anything may have changed
self.env.clear()
# Update addons checksum if state changed and I wasn't uninstalled
own = Module.search_read(
[("name", "=", "module_auto_update")],
["state"],
limit=1)
if own and own[0]["state"] != "uninstalled":
for addon in Module.search([]):
if addon.state != pre_states.get(addon.name):
# Trigger the write hook that should have been
# triggered when the module was [un]installed/updated in
# the limited module graph inside above call to super(),
# and updates its dir checksum as needed
addon.latest_version = addon.latest_version
return result

85
module_auto_update/wizards/module_upgrade_deprecated.py

@ -0,0 +1,85 @@
# -*- coding: utf-8 -*-
# Copyright 2017 LasLabs Inc.
# License LGPL-3.0 or later (http://www.gnu.org/licenses/lgpl).
import logging
from openerp import api, models
from ..models.module_deprecated import PARAM_DEPRECATED
_logger = logging.getLogger(__name__)
class ModuleUpgrade(models.TransientModel):
_inherit = 'base.module.upgrade'
@api.model
def _autoupdate_deprecated(self):
"""Know if we should enable deprecated features."""
deprecated = (
self.env["ir.config_parameter"].get_param(PARAM_DEPRECATED))
if deprecated is False:
# Enable deprecated features if this is the 1st automated update
# after the version that deprecated them (X.Y.2.0.0)
own_module = self.env["ir.module.module"].search([
("name", "=", "module_auto_update"),
])
try:
if own_module.latest_version.split(".")[2] == "1":
deprecated = "1"
except AttributeError:
pass # 1st install, there's no latest_version
return deprecated == "1"
@api.model
def get_module_list(self):
"""Set modules to upgrade searching by their dir checksum."""
if self._autoupdate_deprecated():
Module = self.env["ir.module.module"]
installed_modules = Module.search([('state', '=', 'installed')])
upgradeable_modules = installed_modules.filtered(
lambda r: r.checksum_dir != r.checksum_installed,
)
upgradeable_modules.button_upgrade()
return super(ModuleUpgrade, self).get_module_list()
@api.multi
def upgrade_module(self):
"""Make a fully automated addon upgrade."""
if self._autoupdate_deprecated():
_logger.warning(
"You are possibly using an unsupported upgrade system; "
"set '%s' system parameter to '0' and start calling "
"`env['ir.module.module'].upgrade_changed_checksum()` from "
"now on to get rid of this message. See module's README's "
"Known Issues section for further information on the matter."
)
# Compute updates by checksum when called in @api.model fashion
self.env.cr.autocommit(True) # Avoid transaction lock
if not self:
self.get_module_list()
Module = self.env["ir.module.module"]
# Get every addon state before updating
pre_states = {addon["name"]: addon["state"] for addon
in Module.search_read([], ["name", "state"])}
# Perform upgrades, possibly in a limited graph that excludes me
result = super(ModuleUpgrade, self).upgrade_module()
if self._autoupdate_deprecated():
self.env.cr.autocommit(False)
# Reload environments, anything may have changed
self.env.clear()
# Update addons checksum if state changed and I wasn't uninstalled
own = Module.search_read(
[("name", "=", "module_auto_update")],
["state"],
limit=1)
if own and own[0]["state"] != "uninstalled":
for addon in Module.search([]):
if addon.state != pre_states.get(addon.name):
# Trigger the write hook that should have been
# triggered when the module was [un]installed/updated
# in the limited module graph inside above call to
# super(), and updates its dir checksum as needed
addon.latest_version = addon.latest_version
return result

1
requirements.txt

@ -1,4 +1,3 @@
checksumdir
python-ldap
unidecode
acme_tiny

Loading…
Cancel
Save