[REF] profiler: Many changes (see list below)
- Use OCA README template - Fix lint, flake8, rst, js and OCA guidelines - IMP supports pgbadger profiling - Add testspull/375/head
-
14.hgignore
-
68README.rst
-
95profiler/README.rst
-
38profiler/__init__.py
-
51profiler/__openerp__.py
-
70profiler/controllers.py
-
5profiler/controllers/__init__.py
-
228profiler/controllers/main.py
-
20profiler/core.py
-
16profiler/data/profiler_excluding.xml
-
103profiler/hooks.py
-
4profiler/security/group.xml
-
0profiler/static/description/clear_stats.png
-
0profiler/static/description/dump_stats.png
-
0profiler/static/description/player.png
-
0profiler/static/description/start_profiling.png
-
0profiler/static/description/stop_profiling.png
-
14profiler/static/src/css/player.css
-
BINprofiler/static/src/img/icon.png
-
7profiler/static/src/js/player.js
-
52profiler/static/src/js/test_profiler.js
-
24profiler/static/src/less/player.less
-
36profiler/static/src/xml/player.xml
-
5profiler/tests/__init__.py
-
17profiler/tests/test_profiler.py
-
12profiler/views/assets.xml
-
7profiler/views/profiler.xml
-
1requirements.txt
@ -1,14 +0,0 @@ |
|||||
syntax: glob |
|
||||
|
|
||||
*.swp |
|
||||
*.orig |
|
||||
*.pyc |
|
||||
*.pyo |
|
||||
*.log |
|
||||
*\# |
|
||||
*.\#* |
|
||||
*~ |
|
||||
|
|
||||
.pydevproject |
|
||||
.project |
|
||||
.installed.cfg |
|
@ -1,68 +0,0 @@ |
|||||
cProfile integration for OpenERP |
|
||||
================================ |
|
||||
|
|
||||
The module ``profiler`` provides a very basic integration of |
|
||||
the standard ``cProfile`` into OpenERP/Odoo. |
|
||||
|
|
||||
Basic usage |
|
||||
----------- |
|
||||
|
|
||||
After installation, a player is add on the header bar, with |
|
||||
four items: |
|
||||
|
|
||||
|player| |
|
||||
|
|
||||
* Start profiling |start_profiling| |
|
||||
* Stop profiling |stop_profiling| |
|
||||
* Download stats: download stats file |dump_stats| |
|
||||
* Clear stats |clear_stats| |
|
||||
|
|
||||
Advantages |
|
||||
---------- |
|
||||
|
|
||||
Executing Python code under the profiler is not really hard, but this |
|
||||
module allows to do it in OpenERP context such that: |
|
||||
|
|
||||
* no direct modification of main server Python code or addons is needed |
|
||||
(although it could be pretty simple depending on the need) |
|
||||
* subtleties about threads are taken care of. In particular, the |
|
||||
accumulation of stats over several requests is correct. |
|
||||
* Quick access UI to avoid statistics pollution |
|
||||
* Use the standard cProfile format, see Python documentation and performance |
|
||||
wiki page for exploitation tips. Also do not miss `RunSnakeRun |
|
||||
<http://www.vrplumber.com/programming/runsnakerun/>`_ GUI tool to help you to |
|
||||
interpret it easly. |
|
||||
|
|
||||
Caveats |
|
||||
------- |
|
||||
|
|
||||
* enabling the profile in one database actually does it for the whole |
|
||||
instance |
|
||||
* multiprocessing (``--workers``) is *not* taken into account |
|
||||
* currently developped and tested with OpenERP 7.0 only |
|
||||
* no special care for uninstallion : currently a restart is needed to |
|
||||
finish uninstalling. |
|
||||
* requests not going through web controllers are currently not taken |
|
||||
into account |
|
||||
|
|
||||
|
|
||||
Credit |
|
||||
------ |
|
||||
|
|
||||
Remotely inspired from ZopeProfiler, although there is no online |
|
||||
visualisation and there may never be one. |
|
||||
|
|
||||
.. |player| image:: https://bytebucket.org/anybox/odoo_profiler/raw/default/doc/static/player.png |
|
||||
:alt: Player to manage profiler |
|
||||
.. |start_profiling| image:: https://bytebucket.org/anybox/odoo_profiler/raw/default/doc/static/start_profiling.png |
|
||||
:alt: Start profiling |
|
||||
:height: 35px |
|
||||
.. |stop_profiling| image:: https://bytebucket.org/anybox/odoo_profiler/raw/default/doc/static/stop_profiling.png |
|
||||
:alt: Stop profiling |
|
||||
:height: 35px |
|
||||
.. |dump_stats| image:: https://bytebucket.org/anybox/odoo_profiler/raw/default/doc/static/dump_stats.png |
|
||||
:alt: Download cprofile stats file |
|
||||
:height: 35px |
|
||||
.. |clear_stats| image:: https://bytebucket.org/anybox/odoo_profiler/raw/default/doc/static/clear_stats.png |
|
||||
:alt: Clear and remove stats file |
|
||||
:height: 35px |
|
@ -0,0 +1,95 @@ |
|||||
|
.. image:: https://img.shields.io/badge/licence-AGPL--3-blue.svg |
||||
|
:target: http://www.gnu.org/licenses/agpl-3.0-standalone.html |
||||
|
:alt: License: AGPL-3 |
||||
|
|
||||
|
======== |
||||
|
Profiler |
||||
|
======== |
||||
|
|
||||
|
Integration of cProfile and PgBadger. |
||||
|
|
||||
|
Installation |
||||
|
============ |
||||
|
|
||||
|
To install this module, you need the following requirements: |
||||
|
|
||||
|
* Install `pgbadger <http://dalibo.github.io/pgbadger/>`_ binary package. |
||||
|
* Install `pstats_print2list <https://pypi.python.org/pypi/pstats_print2list>`_ python package. |
||||
|
* Set `PG_LOG_PATH` environment variable to know location of the `postgresql.log` file by default is `/var/lib/postgresql/9.X/main/pg_log/postgresql.log` |
||||
|
* Enable postgresql logs from postgresql's configuration file (Default location for Linux Debian is `/etc/postgresql/*/main/postgresql.conf`) |
||||
|
- Add the following lines at final (A postgresql restart is required `/etc/init.d/postgresql restart`) |
||||
|
|
||||
|
.. code-block:: text |
||||
|
|
||||
|
logging_collector=on |
||||
|
log_destination='stderr' |
||||
|
log_directory='pg_log' |
||||
|
log_filename='postgresql.log' |
||||
|
log_rotation_age=0 |
||||
|
log_checkpoints=on |
||||
|
log_hostname=on |
||||
|
log_line_prefix='%t [%p]: [%l-1] db=%d,user=%u ' |
||||
|
|
||||
|
|
||||
|
Configuration |
||||
|
============= |
||||
|
|
||||
|
By default profiler module adds two system parameters |
||||
|
- exclude_fnames > '/.repo_requirements,~/odoo-8.0,/usr/,>' |
||||
|
- exclude_query > 'ir_translation'. |
||||
|
|
||||
|
These parameters can be configurated in order to exclude some outputs from |
||||
|
profiling stats or pgbadger output. |
||||
|
|
||||
|
Usage |
||||
|
===== |
||||
|
|
||||
|
After installation, a player is add on the header bar, with following buttons: |
||||
|
|
||||
|
- .. figure:: static/description/player.png |
||||
|
:alt: Player to manage profiler |
||||
|
|
||||
|
|
||||
|
* Start profiling |
||||
|
- .. figure:: static/description/start_profiling.png |
||||
|
:alt: Start profiling |
||||
|
:height: 35px |
||||
|
* Stop profiling |
||||
|
- .. figure:: static/description/stop_profiling.png |
||||
|
:alt: Stop profiling |
||||
|
:height: 35px |
||||
|
* Download stats: download stats file |
||||
|
- .. figure:: static/description/dump_stats.png |
||||
|
:alt: Download cprofile stats file |
||||
|
:height: 35px |
||||
|
* Clear stats |
||||
|
- .. figure:: static/description/clear_stats.png |
||||
|
:alt: Clear and remove stats file |
||||
|
:height: 35px |
||||
|
|
||||
|
|
||||
|
Credits |
||||
|
======= |
||||
|
|
||||
|
Contributors |
||||
|
------------ |
||||
|
|
||||
|
* Georges Racinet <gracinet@anybox.fr> |
||||
|
- Remotely inspired from ZopeProfiler, although there is no online visualisation and there may never be one. |
||||
|
* Moisés López <moylop260@vauxoo.com> |
||||
|
* Hugo Adan <hugo@vauxoo.com> |
||||
|
|
||||
|
Maintainer |
||||
|
---------- |
||||
|
|
||||
|
.. image:: https://odoo-community.org/logo.png |
||||
|
:alt: Odoo Community Association |
||||
|
:target: https://odoo-community.org |
||||
|
|
||||
|
This module is maintained by the OCA. |
||||
|
|
||||
|
OCA, or the Odoo Community Association, is a nonprofit organization whose |
||||
|
mission is to support the collaborative development of Odoo features and |
||||
|
promote its widespread use. |
||||
|
|
||||
|
To contribute to this module, please visit https://odoo-community.org. |
@ -1,32 +1,6 @@ |
|||||
import controllers # noqa |
|
||||
from cProfile import Profile |
|
||||
|
|
||||
|
|
||||
def patch_openerp(): |
|
||||
"""Modify OpenERP/Odoo entry points so that profile can record. |
|
||||
|
|
||||
Odoo is a multi-threaded program. Therefore, the :data:`profile` object |
|
||||
needs to be enabled/disabled each in each thread to capture all the |
|
||||
execution. |
|
||||
|
|
||||
For instance, OpenERP 7 spawns a new thread for each request. |
|
||||
""" |
|
||||
from openerp.addons.web.http import JsonRequest |
|
||||
from .core import profiling |
|
||||
orig_dispatch = JsonRequest.dispatch |
|
||||
|
|
||||
def dispatch(*args, **kwargs): |
|
||||
with profiling(): |
|
||||
return orig_dispatch(*args, **kwargs) |
|
||||
JsonRequest.dispatch = dispatch |
|
||||
|
|
||||
|
|
||||
def create_profile(): |
|
||||
"""Create the global, shared profile object.""" |
|
||||
from . import core |
|
||||
core.profile = Profile() |
|
||||
|
|
||||
|
|
||||
def post_load(): |
|
||||
create_profile() |
|
||||
patch_openerp() |
|
||||
|
# coding: utf-8 |
||||
|
# License AGPL-3 or later (http://www.gnu.org/licenses/lgpl). |
||||
|
# Copyright 2014 Anybox <http://anybox.fr> |
||||
|
# Copyright 2016 Vauxoo (https://www.vauxoo.com) <info@vauxoo.com> |
||||
|
from . import controllers |
||||
|
from .hooks import post_load |
@ -1,45 +1,32 @@ |
|||||
#============================================================================== |
|
||||
# = |
|
||||
# profiler module for OpenERP, cProfile integration for Odoo/OpenERP |
|
||||
# Copyright (C) 2014 Anybox <http://anybox.fr> |
|
||||
# = |
|
||||
# This file is a part of profiler |
|
||||
# = |
|
||||
# profiler is free software: you can redistribute it and/or modify |
|
||||
# it under the terms of the GNU Affero General Public License v3 or later |
|
||||
# as published by the Free Software Foundation, either version 3 of the |
|
||||
# License, or (at your option) any later version. |
|
||||
# = |
|
||||
# profiler is distributed in the hope that it will be useful, |
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of |
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
|
||||
# GNU Affero General Public License v3 or later for more details. |
|
||||
# = |
|
||||
# You should have received a copy of the GNU Affero General Public License |
|
||||
# v3 or later along with this program. |
|
||||
# If not, see <http://www.gnu.org/licenses/>. |
|
||||
# = |
|
||||
#============================================================================== |
|
||||
|
# coding: utf-8 |
||||
|
# License AGPL-3 or later (http://www.gnu.org/licenses/lgpl). |
||||
|
# Copyright 2014 Anybox <http://anybox.fr> |
||||
|
# Copyright 2016 Vauxoo (https://www.vauxoo.com) <info@vauxoo.com> |
||||
|
|
||||
{ |
{ |
||||
'name': 'profiler', |
'name': 'profiler', |
||||
'version': '8.0.0.1', |
|
||||
'category': 'devtools', |
|
||||
'description': """ |
|
||||
cprofile integration for Odoo/OpenERP. Check the Profiler menu in admin menu |
|
||||
""", |
|
||||
'author': 'Georges Racinet', |
|
||||
'website': 'http://anybox.fr', |
|
||||
'depends': ['base', 'web'], |
|
||||
|
'version': '8.0.1.0.0', |
||||
|
'category': 'Tools', |
||||
|
'license': 'AGPL-3', |
||||
|
'author': 'Anybox, Vauxoo, Odoo Community Association (OCA)', |
||||
|
'website': 'https://odoo-community.org', |
||||
|
'depends': ['website'], |
||||
'data': [ |
'data': [ |
||||
|
'data/profiler_excluding.xml', |
||||
'security/group.xml', |
'security/group.xml', |
||||
'views/profiler.xml' |
|
||||
|
'views/profiler.xml', |
||||
|
'views/assets.xml', |
||||
|
], |
||||
|
'external_dependencies': { |
||||
|
'python': [ |
||||
|
'pstats_print2list', |
||||
], |
], |
||||
|
}, |
||||
'qweb': [ |
'qweb': [ |
||||
'static/src/xml/player.xml', |
'static/src/xml/player.xml', |
||||
], |
], |
||||
'installable': True, |
'installable': True, |
||||
'application': False, |
'application': False, |
||||
'auto_install': False, |
'auto_install': False, |
||||
'license': 'AGPL-3', |
|
||||
'post_load': 'post_load', |
'post_load': 'post_load', |
||||
} |
} |
@ -1,70 +0,0 @@ |
|||||
import os |
|
||||
import logging |
|
||||
from datetime import datetime |
|
||||
from tempfile import mkstemp |
|
||||
import openerp.addons.web.http as openerpweb |
|
||||
from . import core |
|
||||
|
|
||||
logger = logging.getLogger(__name__) |
|
||||
|
|
||||
|
|
||||
class ProfilerController(openerpweb.Controller): |
|
||||
|
|
||||
_cp_path = '/web/profiler' |
|
||||
|
|
||||
player_state = 'profiler_player_clear' |
|
||||
"""Indicate the state(css class) of the player: |
|
||||
|
|
||||
* profiler_player_clear |
|
||||
* profiler_player_enabled |
|
||||
* profiler_player_disabled |
|
||||
""" |
|
||||
|
|
||||
@openerpweb.jsonrequest |
|
||||
def enable(self, request): |
|
||||
logger.info("Enabling") |
|
||||
core.enabled = True |
|
||||
ProfilerController.player_state = 'profiler_player_enabled' |
|
||||
|
|
||||
@openerpweb.jsonrequest |
|
||||
def disable(self, request): |
|
||||
logger.info("Disabling") |
|
||||
core.enabled = False |
|
||||
ProfilerController.player_state = 'profiler_player_disabled' |
|
||||
|
|
||||
@openerpweb.jsonrequest |
|
||||
def clear(self, request): |
|
||||
core.profile.clear() |
|
||||
logger.info("Cleared stats") |
|
||||
ProfilerController.player_state = 'profiler_player_clear' |
|
||||
|
|
||||
@openerpweb.httprequest |
|
||||
def dump(self, request, token): |
|
||||
"""Provide the stats as a file download. |
|
||||
|
|
||||
Uses a temporary file, because apparently there's no API to |
|
||||
dump stats in a stream directly. |
|
||||
""" |
|
||||
handle, path = mkstemp(prefix='profiling') |
|
||||
core.profile.dump_stats(path) |
|
||||
stream = os.fdopen(handle) |
|
||||
os.unlink(path) # TODO POSIX only ? |
|
||||
stream.seek(0) |
|
||||
filename = 'openerp_%s.stats' % datetime.now().isoformat() |
|
||||
# can't close the stream even in a context manager: it'll be needed |
|
||||
# after the return from this method, we'll let Python's GC do its job |
|
||||
return request.make_response( |
|
||||
stream, |
|
||||
headers=[ |
|
||||
('Content-Disposition', |
|
||||
'attachment; filename="%s"' % filename), |
|
||||
('Content-Type', 'application/octet-stream') |
|
||||
], cookies={'fileToken': token}) |
|
||||
|
|
||||
@openerpweb.jsonrequest |
|
||||
def initial_state(self, request): |
|
||||
user = request.session.model('res.users') |
|
||||
return { |
|
||||
'has_player_group': user.has_group('profiler.group_profiler_player'), |
|
||||
'player_state': ProfilerController.player_state, |
|
||||
} |
|
@ -0,0 +1,5 @@ |
|||||
|
# coding: utf-8 |
||||
|
# License AGPL-3 or later (http://www.gnu.org/licenses/lgpl). |
||||
|
# Copyright 2014 Anybox <http://anybox.fr> |
||||
|
# Copyright 2016 Vauxoo (https://www.vauxoo.com) <info@vauxoo.com> |
||||
|
from . import main |
@ -0,0 +1,228 @@ |
|||||
|
# coding: utf-8 |
||||
|
# License AGPL-3 or later (http://www.gnu.org/licenses/lgpl). |
||||
|
# Copyright 2014 Anybox <http://anybox.fr> |
||||
|
# Copyright 2016 Vauxoo (https://www.vauxoo.com) <info@vauxoo.com> |
||||
|
import errno |
||||
|
import logging |
||||
|
import os |
||||
|
import sys |
||||
|
import tempfile |
||||
|
from cStringIO import StringIO |
||||
|
from datetime import datetime |
||||
|
|
||||
|
from openerp import http, sql_db, tools |
||||
|
from openerp.addons.web.controllers.main import content_disposition |
||||
|
from openerp.http import request |
||||
|
from openerp.service.db import dump_db_manifest |
||||
|
from openerp.tools.misc import find_in_path |
||||
|
|
||||
|
from ..hooks import CoreProfile as core |
||||
|
|
||||
|
_logger = logging.getLogger(__name__) |
||||
|
|
||||
|
try: |
||||
|
from pstats_print2list import get_pstats_print2list, print_pstats_list |
||||
|
except ImportError as err: # pragma: no cover |
||||
|
_logger.debug(err) |
||||
|
|
||||
|
DFTL_LOG_PATH = '/var/lib/postgresql/%s/main/pg_log/postgresql.log' |
||||
|
|
||||
|
PGOPTIONS = ( |
||||
|
'-c client_min_messages=notice -c log_min_messages=warning ' |
||||
|
'-c log_min_error_statement=error ' |
||||
|
'-c log_min_duration_statement=0 -c log_connections=on ' |
||||
|
'-c log_disconnections=on -c log_duration=off ' |
||||
|
'-c log_error_verbosity=verbose -c log_lock_waits=on ' |
||||
|
'-c log_statement=none -c log_temp_files=0 ' |
||||
|
) |
||||
|
|
||||
|
|
||||
|
class Capturing(list): |
||||
|
def __enter__(self): |
||||
|
self._stdout = sys.stdout |
||||
|
sys.stdout = self._stringio = StringIO() |
||||
|
return self |
||||
|
|
||||
|
def __exit__(self, *args): |
||||
|
self.extend(self._stringio.getvalue().splitlines()) |
||||
|
del self._stringio # free up some memory |
||||
|
sys.stdout = self._stdout |
||||
|
|
||||
|
|
||||
|
class ProfilerController(http.Controller): |
||||
|
|
||||
|
_cp_path = '/web/profiler' |
||||
|
|
||||
|
player_state = 'profiler_player_clear' |
||||
|
begin_date = '' |
||||
|
end_date = '' |
||||
|
"""Indicate the state(css class) of the player: |
||||
|
|
||||
|
* profiler_player_clear |
||||
|
* profiler_player_enabled |
||||
|
* profiler_player_disabled |
||||
|
""" |
||||
|
|
||||
|
@http.route(['/web/profiler/enable'], type='json', auth="user") |
||||
|
def enable(self): |
||||
|
_logger.info("Enabling") |
||||
|
core.enabled = True |
||||
|
ProfilerController.begin_date = datetime.now().strftime( |
||||
|
"%Y-%m-%d %H:%M:%S") |
||||
|
ProfilerController.player_state = 'profiler_player_enabled' |
||||
|
os.environ['PGOPTIONS'] = PGOPTIONS |
||||
|
self.empty_cursor_pool() |
||||
|
|
||||
|
@http.route(['/web/profiler/disable'], type='json', auth="user") |
||||
|
def disable(self, **post): |
||||
|
_logger.info("Disabling") |
||||
|
core.enabled = False |
||||
|
ProfilerController.end_date = datetime.now().strftime( |
||||
|
"%Y-%m-%d %H:%M:%S") |
||||
|
ProfilerController.player_state = 'profiler_player_disabled' |
||||
|
os.environ.pop("PGOPTIONS", None) |
||||
|
self.empty_cursor_pool() |
||||
|
|
||||
|
@http.route(['/web/profiler/clear'], type='json', auth="user") |
||||
|
def clear(self, **post): |
||||
|
core.profile.clear() |
||||
|
_logger.info("Cleared stats") |
||||
|
ProfilerController.player_state = 'profiler_player_clear' |
||||
|
ProfilerController.end_date = '' |
||||
|
ProfilerController.begin_date = '' |
||||
|
|
||||
|
@http.route(['/web/profiler/dump'], type='http', auth="user") |
||||
|
def dump(self, token, **post): |
||||
|
"""Provide the stats as a file download. |
||||
|
|
||||
|
Uses a temporary file, because apparently there's no API to |
||||
|
dump stats in a stream directly. |
||||
|
""" |
||||
|
exclude_fname = self.get_exclude_fname() |
||||
|
with tools.osutil.tempdir() as dump_dir: |
||||
|
ts = datetime.now().strftime("%Y-%m-%d_%H-%M-%S") |
||||
|
filename = 'openerp_%s' % ts |
||||
|
stats_path = os.path.join(dump_dir, '%s.stats' % filename) |
||||
|
core.profile.dump_stats(stats_path) |
||||
|
_logger.info("Pstats Command:") |
||||
|
params = {'fnames': stats_path, 'sort': 'cumulative', 'limit': 45, |
||||
|
'exclude_fnames': exclude_fname} |
||||
|
_logger.info( |
||||
|
"fnames=%(fnames)s, sort=%(sort)s," |
||||
|
" limit=%(limit)s, exclude_fnames=%(exclude_fnames)s", params) |
||||
|
pstats_list = get_pstats_print2list(**params) |
||||
|
with Capturing() as output: |
||||
|
print_pstats_list(pstats_list) |
||||
|
result_path = os.path.join(dump_dir, '%s.txt' % filename) |
||||
|
with open(result_path, "a") as res_file: |
||||
|
for line in output: |
||||
|
res_file.write('%s\n' % line) |
||||
|
# PG_BADGER |
||||
|
self.dump_pgbadger(dump_dir, 'pgbadger_output.txt', request.cr) |
||||
|
t_zip = tempfile.TemporaryFile() |
||||
|
tools.osutil.zip_dir(dump_dir, t_zip, include_dir=False) |
||||
|
t_zip.seek(0) |
||||
|
headers = [ |
||||
|
('Content-Type', 'application/octet-stream; charset=binary'), |
||||
|
('Content-Disposition', content_disposition( |
||||
|
'%s.zip' % filename))] |
||||
|
_logger.info('Download Profiler zip: %s', t_zip.name) |
||||
|
return request.make_response( |
||||
|
t_zip, headers=headers, cookies={'fileToken': token}) |
||||
|
|
||||
|
@http.route(['/web/profiler/initial_state'], type='json', auth="user") |
||||
|
def initial_state(self, **post): |
||||
|
user = request.env['res.users'].browse(request.uid) |
||||
|
return { |
||||
|
'has_player_group': user.has_group( |
||||
|
'profiler.group_profiler_player'), |
||||
|
'player_state': ProfilerController.player_state, |
||||
|
} |
||||
|
|
||||
|
def dump_pgbadger(self, dir_dump, output, cursor): |
||||
|
pgbadger = find_in_path("pgbadger") |
||||
|
if not pgbadger: |
||||
|
_logger.error("Pgbadger not found") |
||||
|
return |
||||
|
filename = os.path.join(dir_dump, output) |
||||
|
pg_version = dump_db_manifest(cursor)['pg_version'] |
||||
|
log_path = os.environ.get('PG_LOG_PATH', DFTL_LOG_PATH % pg_version) |
||||
|
if not os.path.exists(os.path.dirname(filename)): |
||||
|
try: |
||||
|
os.makedirs(os.path.dirname(filename)) |
||||
|
except OSError as exc: |
||||
|
# error is different than File exists |
||||
|
if exc.errno != errno.EEXIST: |
||||
|
_logger.error("Folder %s can not be created", |
||||
|
os.path.dirname(filename)) |
||||
|
return |
||||
|
_logger.info("Generating PG Badger report.") |
||||
|
exclude_query = self.get_exclude_query() |
||||
|
dbname = cursor.dbname |
||||
|
command = [ |
||||
|
pgbadger, '-f', 'stderr', '-T', 'Odoo-Profiler', |
||||
|
'-o', '-', '-d', dbname, '-b', ProfilerController.begin_date, |
||||
|
'-e', ProfilerController.end_date, '--sample', '2', |
||||
|
'--disable-type', '--disable-error', '--disable-hourly', |
||||
|
'--disable-session', '--disable-connection', |
||||
|
'--disable-temporary', '--quiet'] |
||||
|
command.extend(exclude_query) |
||||
|
command.append(log_path) |
||||
|
|
||||
|
_logger.info("Pgbadger Command:") |
||||
|
_logger.info(command) |
||||
|
result = tools.exec_command_pipe(*command) |
||||
|
with open(filename, 'w') as fw: |
||||
|
fw.write(result[1].read()) |
||||
|
_logger.info("Done") |
||||
|
|
||||
|
def get_exclude_fname(self): |
||||
|
efnameid = request.env.ref( |
||||
|
'profiler.default_exclude_fnames_pstas', raise_if_not_found=False) |
||||
|
if not efnameid: |
||||
|
return [] |
||||
|
return [os.path.expanduser(path) |
||||
|
for path in efnameid and efnameid.value.strip(',').split(',') |
||||
|
if path] |
||||
|
|
||||
|
def get_exclude_query(self): |
||||
|
"""Example '^(COPY|COMMIT)' |
||||
|
""" |
||||
|
equeryid = request.env.ref( |
||||
|
'profiler.default_exclude_query_pgbadger', |
||||
|
raise_if_not_found=False) |
||||
|
if not equeryid: |
||||
|
return [] |
||||
|
exclude_queries = [] |
||||
|
for path in equeryid and equeryid.value.strip(',').split(','): |
||||
|
exclude_queries.extend( |
||||
|
['--exclude-query', '"^(%s)" ' % path.encode('UTF-8')]) |
||||
|
return exclude_queries |
||||
|
|
||||
|
def empty_cursor_pool(self): |
||||
|
"""This method cleans (rollback) all current transactions over actual |
||||
|
cursor in order to avoid errors with waiting transactions. |
||||
|
- request.cr.rollback() |
||||
|
|
||||
|
Also connections on current database's only are closed by the next |
||||
|
statement |
||||
|
- dsn = openerp.sql_db.dsn(request.cr.dbname) |
||||
|
- openerp.sql_db._Pool.close_all(dsn[1]) |
||||
|
Otherwise next error will be trigger |
||||
|
'InterfaceError: connection already closed' |
||||
|
|
||||
|
Finally new cursor is assigned to the request object, this cursor will |
||||
|
take the os.environ setted. In this case the os.environ is setted with |
||||
|
all 'PGOPTIONS' required to log all sql transactions in postgres.log |
||||
|
file. |
||||
|
|
||||
|
If this method is called one more time, it will create a new cursor and |
||||
|
take the os.environ again, this is usefully if we want to reset |
||||
|
'PGOPTIONS' |
||||
|
|
||||
|
""" |
||||
|
request.cr._cnx.reset() |
||||
|
dsn = sql_db.dsn(request.cr.dbname) |
||||
|
sql_db._Pool.close_all(dsn[1]) |
||||
|
db = sql_db.db_connect(request.cr.dbname) |
||||
|
request._cr = db.cursor() |
@ -1,20 +0,0 @@ |
|||||
from contextlib import contextmanager |
|
||||
profile = None |
|
||||
"""The thread-shared profile object. |
|
||||
""" |
|
||||
|
|
||||
enabled = False |
|
||||
"""Indicates if the whole profiling functionality is globally active or not. |
|
||||
""" |
|
||||
|
|
||||
|
|
||||
@contextmanager |
|
||||
def profiling(): |
|
||||
"""Thread local profile management, according to the shared :data:`enabled` |
|
||||
""" |
|
||||
if enabled: |
|
||||
profile.enable() |
|
||||
yield |
|
||||
|
|
||||
if enabled: |
|
||||
profile.disable() |
|
@ -0,0 +1,16 @@ |
|||||
|
<?xml version="1.0" encoding="utf-8"?> |
||||
|
<openerp> |
||||
|
<data> |
||||
|
<!-- exclude_fnames de pstats_print2list --> |
||||
|
<record id="default_exclude_fnames_pstas" model="ir.config_parameter"> |
||||
|
<field name="key">exclude_fnames</field> |
||||
|
<field name="value">/.repo_requirements,~/odoo-8.0,/usr/,></field> |
||||
|
</record> |
||||
|
|
||||
|
<!-- exclude-query de pgbader --> |
||||
|
<record id="default_exclude_query_pgbadger" model="ir.config_parameter"> |
||||
|
<field name="key">exclude_query</field> |
||||
|
<field name="value">ir_translation</field> |
||||
|
</record> |
||||
|
</data> |
||||
|
</openerp> |
@ -0,0 +1,103 @@ |
|||||
|
# coding: utf-8 |
||||
|
# License AGPL-3 or later (http://www.gnu.org/licenses/lgpl). |
||||
|
# Copyright 2014 Anybox <http://anybox.fr> |
||||
|
# Copyright 2016 Vauxoo (https://www.vauxoo.com) <info@vauxoo.com> |
||||
|
import logging |
||||
|
import os |
||||
|
from contextlib import contextmanager |
||||
|
from cProfile import Profile |
||||
|
|
||||
|
import openerp |
||||
|
from openerp import sql_db |
||||
|
from openerp.http import WebRequest |
||||
|
from openerp.service.server import ThreadedServer |
||||
|
|
||||
|
_logger = logging.getLogger(__name__) |
||||
|
|
||||
|
|
||||
|
class CoreProfile(object): |
||||
|
"""The thread-shared profile object""" |
||||
|
profile = None |
||||
|
# Indicates if the whole profiling functionality is globally active or not. |
||||
|
enabled = False |
||||
|
|
||||
|
|
||||
|
@contextmanager |
||||
|
def profiling(): |
||||
|
"""Thread local profile management, according to the shared :data:`enabled` |
||||
|
""" |
||||
|
if CoreProfile.enabled: |
||||
|
CoreProfile.profile.enable() |
||||
|
yield |
||||
|
|
||||
|
if CoreProfile.enabled: |
||||
|
CoreProfile.profile.disable() |
||||
|
|
||||
|
|
||||
|
def patch_odoo(): |
||||
|
"""Modify Odoo entry points so that profile can record. |
||||
|
|
||||
|
Odoo is a multi-threaded program. Therefore, the :data:`profile` object |
||||
|
needs to be enabled/disabled each in each thread to capture all the |
||||
|
execution. |
||||
|
|
||||
|
For instance, Odoo spawns a new thread for each request. |
||||
|
|
||||
|
Modify database connect method to add options to enable postgresql logging |
||||
|
based on PGOPTIONS environment variable |
||||
|
""" |
||||
|
_logger.info('Patching openerp.http.WebRequest._call_function') |
||||
|
webreq_f_origin = WebRequest._call_function |
||||
|
|
||||
|
def webreq_f(*args, **kwargs): |
||||
|
with profiling(): |
||||
|
return webreq_f_origin(*args, **kwargs) |
||||
|
WebRequest._call_function = webreq_f |
||||
|
|
||||
|
_logger.info('Patching openerp.sql_db.db_connect') |
||||
|
db_connect_origin = sql_db.db_connect |
||||
|
|
||||
|
def dbconnect_f(to, *args, **kwargs): |
||||
|
try: |
||||
|
to += " options='%s' " % (os.environ['PGOPTIONS'] or '') |
||||
|
except KeyError: |
||||
|
pass |
||||
|
return db_connect_origin(to, *args, **kwargs) |
||||
|
sql_db.db_connect = dbconnect_f |
||||
|
|
||||
|
|
||||
|
def dump_stats(): |
||||
|
"""Dump stats to standard file""" |
||||
|
_logger.info('Dump stats') |
||||
|
CoreProfile.profile.dump_stats( |
||||
|
os.path.expanduser('~/.openerp_server.stats')) |
||||
|
|
||||
|
|
||||
|
def create_profile(): |
||||
|
"""Create the global, shared profile object.""" |
||||
|
_logger.info('Create profile') |
||||
|
CoreProfile.profile = Profile() |
||||
|
|
||||
|
|
||||
|
def patch_stop(): |
||||
|
"""When the server is stopped then save the result of cProfile stats""" |
||||
|
origin_stop = ThreadedServer.stop |
||||
|
|
||||
|
_logger.info('Patching openerp.service.server.ThreadedServer.stop') |
||||
|
|
||||
|
def stop(*args, **kwargs): |
||||
|
if openerp.tools.config['test_enable']: |
||||
|
dump_stats() |
||||
|
return origin_stop(*args, **kwargs) |
||||
|
ThreadedServer.stop = stop |
||||
|
|
||||
|
|
||||
|
def post_load(): |
||||
|
_logger.info('Post load') |
||||
|
create_profile() |
||||
|
patch_odoo() |
||||
|
if openerp.tools.config['test_enable']: |
||||
|
# Enable profile in test mode for orm methods. |
||||
|
_logger.info('Enabling profiler and apply patch') |
||||
|
CoreProfile.enabled = True |
||||
|
patch_stop() |
@ -1,11 +1,11 @@ |
|||||
<?xml version="1.0" encoding="UTF-8"?> |
<?xml version="1.0" encoding="UTF-8"?> |
||||
<openerp> |
<openerp> |
||||
<data noupdate="1"> |
|
||||
|
<data noupdate="1"> |
||||
<record id="group_profiler_player" model="res.groups"> |
<record id="group_profiler_player" model="res.groups"> |
||||
<field name="name">Profiling capability</field> |
<field name="name">Profiling capability</field> |
||||
</record> |
</record> |
||||
<record model="res.users" id="base.user_root"> |
<record model="res.users" id="base.user_root"> |
||||
<field name="groups_id" eval="[(4, ref('profiler.group_profiler_player'))]"/> |
<field name="groups_id" eval="[(4, ref('profiler.group_profiler_player'))]"/> |
||||
</record> |
</record> |
||||
</data> |
|
||||
|
</data> |
||||
</openerp> |
</openerp> |
Before Width: 277 | Height: 58 | Size: 8.2 KiB After Width: 277 | Height: 58 | Size: 8.2 KiB |
Before Width: 279 | Height: 52 | Size: 8.1 KiB After Width: 279 | Height: 52 | Size: 8.1 KiB |
Before Width: 76 | Height: 30 | Size: 1.5 KiB After Width: 76 | Height: 30 | Size: 1.5 KiB |
Before Width: 280 | Height: 49 | Size: 7.1 KiB After Width: 280 | Height: 49 | Size: 7.1 KiB |
Before Width: 275 | Height: 57 | Size: 7.2 KiB After Width: 275 | Height: 57 | Size: 7.2 KiB |
@ -1,21 +1,21 @@ |
|||||
.openerp .oe_topbar_item.profiler_player img { |
|
||||
|
.oe_topbar_item.profiler_player i { |
||||
cursor: pointer; |
cursor: pointer; |
||||
} |
} |
||||
.openerp .oe_topbar_item.profiler_player.profiler_player_enabled img.profiler_enable { |
|
||||
|
.oe_topbar_item.profiler_player.profiler_player_enabled a.profiler_enable { |
||||
display: None; |
display: None; |
||||
} |
} |
||||
.openerp .oe_topbar_item.profiler_player.profiler_player_enabled img.profiler_dump { |
|
||||
|
.oe_topbar_item.profiler_player.profiler_player_enabled a.profiler_dump { |
||||
display: None; |
display: None; |
||||
} |
} |
||||
.openerp .oe_topbar_item.profiler_player.profiler_player_disabled img.profiler_disable { |
|
||||
|
.oe_topbar_item.profiler_player.profiler_player_disabled a.profiler_disable { |
||||
display: None; |
display: None; |
||||
} |
} |
||||
.openerp .oe_topbar_item.profiler_player.profiler_player_clear img.profiler_disable { |
|
||||
|
.oe_topbar_item.profiler_player.profiler_player_clear a.profiler_disable { |
||||
display: None; |
display: None; |
||||
} |
} |
||||
.openerp .oe_topbar_item.profiler_player.profiler_player_clear img.profiler_clear { |
|
||||
|
.oe_topbar_item.profiler_player.profiler_player_clear a.profiler_clear { |
||||
display: None; |
display: None; |
||||
} |
} |
||||
.openerp .oe_topbar_item.profiler_player.profiler_player_clear img.profiler_dump { |
|
||||
|
.oe_topbar_item.profiler_player.profiler_player_clear a.profiler_dump { |
||||
display: None; |
display: None; |
||||
} |
} |
Before Width: 80 | Height: 80 | Size: 2.1 KiB |
@ -0,0 +1,52 @@ |
|||||
|
(function(){ |
||||
|
'use_strict'; |
||||
|
openerp.Tour.register({ |
||||
|
id: 'profile_run', |
||||
|
name: 'Profile run', |
||||
|
path: '/web', |
||||
|
mode: 'test', |
||||
|
steps: [ |
||||
|
{ |
||||
|
title: 'Check if is cleared', |
||||
|
waitFor: 'li.oe_topbar_item.profiler_player.profiler_player_clear' |
||||
|
}, |
||||
|
{ |
||||
|
title: 'Start profiling', |
||||
|
onload: function () { |
||||
|
$('a.profiler_enable').trigger('click'); |
||||
|
} |
||||
|
}, |
||||
|
{ |
||||
|
title: 'Check if is enabled', |
||||
|
waitFor: 'li.oe_topbar_item.profiler_player.profiler_player_enabled' |
||||
|
}, |
||||
|
{ |
||||
|
title: 'Stop profiling', |
||||
|
onload: function () { |
||||
|
$('a.profiler_disable').trigger('click'); |
||||
|
} |
||||
|
}, |
||||
|
{ |
||||
|
title: 'Check if is disabled', |
||||
|
waitFor: 'li.oe_topbar_item.profiler_player.profiler_player_disabled' |
||||
|
}, |
||||
|
{ |
||||
|
title: 'Dump profiling', |
||||
|
onload: function () { |
||||
|
$('a.profiler_dump').trigger('click'); |
||||
|
} |
||||
|
}, |
||||
|
{ |
||||
|
title: 'Clear profiling', |
||||
|
onload: function () { |
||||
|
$('a.profiler_clear').trigger('click'); |
||||
|
} |
||||
|
}, |
||||
|
{ |
||||
|
title: 'Check if is cleared again', |
||||
|
waitFor: 'li.oe_topbar_item.profiler_player.profiler_player_clear' |
||||
|
}, |
||||
|
|
||||
|
] |
||||
|
}); |
||||
|
}()); |
@ -0,0 +1,24 @@ |
|||||
|
#oe_main_menu_navbar .o_menu_systray { |
||||
|
|
||||
|
.oe_topbar_item.profiler_player i { |
||||
|
cursor: pointer; |
||||
|
} |
||||
|
.oe_topbar_item.profiler_player.profiler_player_enabled a.profiler_enable { |
||||
|
display: None; |
||||
|
} |
||||
|
.oe_topbar_item.profiler_player.profiler_player_enabled a.profiler_dump { |
||||
|
display: None; |
||||
|
} |
||||
|
.oe_topbar_item.profiler_player.profiler_player_disabled a.profiler_disable { |
||||
|
display: None; |
||||
|
} |
||||
|
.oe_topbar_item.profiler_player.profiler_player_clear a.profiler_disable { |
||||
|
display: None; |
||||
|
} |
||||
|
.oe_topbar_item.profiler_player.profiler_player_clear a.profiler_clear { |
||||
|
display: None; |
||||
|
} |
||||
|
.oe_topbar_item.profiler_player.profiler_player_clear a.profiler_dump { |
||||
|
display: None; |
||||
|
} |
||||
|
} |
@ -1,18 +1,26 @@ |
|||||
<templates> |
<templates> |
||||
<t t-name="profiler.player"> |
<t t-name="profiler.player"> |
||||
<div class="oe_topbar_item profiler_player"> |
|
||||
<img class="profiler_enable" |
|
||||
title="Start profiling" |
|
||||
src="/web/static/src/img/icons/gtk-media-record.png"/> |
|
||||
<img class="profiler_disable" |
|
||||
title="Stop profiling" |
|
||||
src="/web/static/src/img/icons/gtk-media-pause.png"/> |
|
||||
<img class="profiler_clear" |
|
||||
title="Clear profiling" |
|
||||
src="/web/static/src/img/icons/gtk-media-stop.png"/> |
|
||||
<img class="profiler_dump" |
|
||||
title="Download stats" |
|
||||
src="/web/static/src/img/icons/gtk-floppy.png"/> |
|
||||
</div> |
|
||||
|
<li class="oe_topbar_item profiler_player"> |
||||
|
<a href="#" class="profiler_enable"> |
||||
|
<i class="fa fa-play" title="Start profiling"/> |
||||
|
</a> |
||||
|
</li> |
||||
|
<li class="oe_topbar_item profiler_player"> |
||||
|
<a href="#" class="profiler_disable"> |
||||
|
<!-- <i class="fa fa-pause" title="Stop profiling"/> --> |
||||
|
<i class="fa fa-stop" title="Stop profiling"/> |
||||
|
</a> |
||||
|
</li> |
||||
|
<li class="oe_topbar_item profiler_player"> |
||||
|
<a href="#" class="profiler_clear"> |
||||
|
<!-- <i class="fa fa-stop" title="Clear profiling"/> --> |
||||
|
<i class="fa fa-refresh" title="Clear profiling"/> |
||||
|
</a> |
||||
|
</li> |
||||
|
<li class="oe_topbar_item profiler_player"> |
||||
|
<a href="#" class="profiler_dump"> |
||||
|
<i class="fa fa-floppy-o" title="Download stats"/> |
||||
|
</a> |
||||
|
</li> |
||||
</t> |
</t> |
||||
</templates> |
</templates> |
@ -0,0 +1,5 @@ |
|||||
|
# coding: utf-8 |
||||
|
# License AGPL-3 or later (http://www.gnu.org/licenses/lgpl). |
||||
|
# Copyright 2014 Anybox <http://anybox.fr> |
||||
|
# Copyright 2016 Vauxoo (https://www.vauxoo.com) <info@vauxoo.com> |
||||
|
from . import test_profiler |
@ -0,0 +1,17 @@ |
|||||
|
# coding: utf-8 |
||||
|
# License AGPL-3 or later (http://www.gnu.org/licenses/lgpl). |
||||
|
# Copyright 2016 Vauxoo (https://www.vauxoo.com) <info@vauxoo.com> |
||||
|
|
||||
|
import unittest |
||||
|
|
||||
|
from openerp import tests |
||||
|
|
||||
|
|
||||
|
@tests.at_install(False) |
||||
|
@tests.post_install(True) |
||||
|
class TestProfiler(tests.HttpCase): |
||||
|
|
||||
|
@unittest.skip("phantomjs tests async for 8.0 are so flaky") |
||||
|
def test_profiler_tour(self): |
||||
|
self.phantom_js('/web', "openerp.Tour.run('profile_run', 'test')", |
||||
|
'openerp.Tour.tours.profile_run', login='admin') |
@ -0,0 +1,12 @@ |
|||||
|
<?xml version="1.0" encoding="utf-8"?> |
||||
|
<openerp> |
||||
|
<data> |
||||
|
<template id="assets_backend_test_profiler" name="Profiler Tests" |
||||
|
inherit_id="website.assets_backend"> |
||||
|
<xpath expr="//script[last()]" position="after"> |
||||
|
<script src="/profiler/static/src/js/test_profiler.js" |
||||
|
type="text/javascript"/> |
||||
|
</xpath> |
||||
|
</template> |
||||
|
</data> |
||||
|
</openerp> |
@ -1,14 +1,11 @@ |
|||||
<?xml version="1.0" encoding="utf-8"?> |
<?xml version="1.0" encoding="utf-8"?> |
||||
<!-- vim:fdn=3: --> |
|
||||
<openerp> |
<openerp> |
||||
<data> |
|
||||
|
|
||||
|
<data> |
||||
<template id="profiler_assets_backend" name="profiler assets" inherit_id="web.assets_backend"> |
<template id="profiler_assets_backend" name="profiler assets" inherit_id="web.assets_backend"> |
||||
<xpath expr="." position="inside"> |
<xpath expr="." position="inside"> |
||||
<link rel="stylesheet" href="/profiler/static/src/css/player.css" /> |
<link rel="stylesheet" href="/profiler/static/src/css/player.css" /> |
||||
<script type="text/javascript" src="/profiler/static/src/js/player.js" /> |
<script type="text/javascript" src="/profiler/static/src/js/player.js" /> |
||||
</xpath> |
</xpath> |
||||
</template> |
</template> |
||||
|
|
||||
</data> |
|
||||
|
</data> |
||||
</openerp> |
</openerp> |