[REF] profiler: Many changes (see list below)
- Use OCA README template - Fix lint, flake8, rst, js and OCA guidelines - IMP supports pgbadger profiling - Add testspull/375/head
-
14.hgignore
-
68README.rst
-
95profiler/README.rst
-
38profiler/__init__.py
-
51profiler/__openerp__.py
-
70profiler/controllers.py
-
5profiler/controllers/__init__.py
-
228profiler/controllers/main.py
-
20profiler/core.py
-
16profiler/data/profiler_excluding.xml
-
103profiler/hooks.py
-
4profiler/security/group.xml
-
0profiler/static/description/clear_stats.png
-
0profiler/static/description/dump_stats.png
-
0profiler/static/description/player.png
-
0profiler/static/description/start_profiling.png
-
0profiler/static/description/stop_profiling.png
-
14profiler/static/src/css/player.css
-
BINprofiler/static/src/img/icon.png
-
7profiler/static/src/js/player.js
-
52profiler/static/src/js/test_profiler.js
-
24profiler/static/src/less/player.less
-
36profiler/static/src/xml/player.xml
-
5profiler/tests/__init__.py
-
17profiler/tests/test_profiler.py
-
12profiler/views/assets.xml
-
7profiler/views/profiler.xml
-
1requirements.txt
@ -1,14 +0,0 @@ |
|||
syntax: glob |
|||
|
|||
*.swp |
|||
*.orig |
|||
*.pyc |
|||
*.pyo |
|||
*.log |
|||
*\# |
|||
*.\#* |
|||
*~ |
|||
|
|||
.pydevproject |
|||
.project |
|||
.installed.cfg |
@ -1,68 +0,0 @@ |
|||
cProfile integration for OpenERP |
|||
================================ |
|||
|
|||
The module ``profiler`` provides a very basic integration of |
|||
the standard ``cProfile`` into OpenERP/Odoo. |
|||
|
|||
Basic usage |
|||
----------- |
|||
|
|||
After installation, a player is add on the header bar, with |
|||
four items: |
|||
|
|||
|player| |
|||
|
|||
* Start profiling |start_profiling| |
|||
* Stop profiling |stop_profiling| |
|||
* Download stats: download stats file |dump_stats| |
|||
* Clear stats |clear_stats| |
|||
|
|||
Advantages |
|||
---------- |
|||
|
|||
Executing Python code under the profiler is not really hard, but this |
|||
module allows to do it in OpenERP context such that: |
|||
|
|||
* no direct modification of main server Python code or addons is needed |
|||
(although it could be pretty simple depending on the need) |
|||
* subtleties about threads are taken care of. In particular, the |
|||
accumulation of stats over several requests is correct. |
|||
* Quick access UI to avoid statistics pollution |
|||
* Use the standard cProfile format, see Python documentation and performance |
|||
wiki page for exploitation tips. Also do not miss `RunSnakeRun |
|||
<http://www.vrplumber.com/programming/runsnakerun/>`_ GUI tool to help you to |
|||
interpret it easly. |
|||
|
|||
Caveats |
|||
------- |
|||
|
|||
* enabling the profile in one database actually does it for the whole |
|||
instance |
|||
* multiprocessing (``--workers``) is *not* taken into account |
|||
* currently developped and tested with OpenERP 7.0 only |
|||
* no special care for uninstallion : currently a restart is needed to |
|||
finish uninstalling. |
|||
* requests not going through web controllers are currently not taken |
|||
into account |
|||
|
|||
|
|||
Credit |
|||
------ |
|||
|
|||
Remotely inspired from ZopeProfiler, although there is no online |
|||
visualisation and there may never be one. |
|||
|
|||
.. |player| image:: https://bytebucket.org/anybox/odoo_profiler/raw/default/doc/static/player.png |
|||
:alt: Player to manage profiler |
|||
.. |start_profiling| image:: https://bytebucket.org/anybox/odoo_profiler/raw/default/doc/static/start_profiling.png |
|||
:alt: Start profiling |
|||
:height: 35px |
|||
.. |stop_profiling| image:: https://bytebucket.org/anybox/odoo_profiler/raw/default/doc/static/stop_profiling.png |
|||
:alt: Stop profiling |
|||
:height: 35px |
|||
.. |dump_stats| image:: https://bytebucket.org/anybox/odoo_profiler/raw/default/doc/static/dump_stats.png |
|||
:alt: Download cprofile stats file |
|||
:height: 35px |
|||
.. |clear_stats| image:: https://bytebucket.org/anybox/odoo_profiler/raw/default/doc/static/clear_stats.png |
|||
:alt: Clear and remove stats file |
|||
:height: 35px |
@ -0,0 +1,95 @@ |
|||
.. image:: https://img.shields.io/badge/licence-AGPL--3-blue.svg |
|||
:target: http://www.gnu.org/licenses/agpl-3.0-standalone.html |
|||
:alt: License: AGPL-3 |
|||
|
|||
======== |
|||
Profiler |
|||
======== |
|||
|
|||
Integration of cProfile and PgBadger. |
|||
|
|||
Installation |
|||
============ |
|||
|
|||
To install this module, you need the following requirements: |
|||
|
|||
* Install `pgbadger <http://dalibo.github.io/pgbadger/>`_ binary package. |
|||
* Install `pstats_print2list <https://pypi.python.org/pypi/pstats_print2list>`_ python package. |
|||
* Set `PG_LOG_PATH` environment variable to know location of the `postgresql.log` file by default is `/var/lib/postgresql/9.X/main/pg_log/postgresql.log` |
|||
* Enable postgresql logs from postgresql's configuration file (Default location for Linux Debian is `/etc/postgresql/*/main/postgresql.conf`) |
|||
- Add the following lines at final (A postgresql restart is required `/etc/init.d/postgresql restart`) |
|||
|
|||
.. code-block:: text |
|||
|
|||
logging_collector=on |
|||
log_destination='stderr' |
|||
log_directory='pg_log' |
|||
log_filename='postgresql.log' |
|||
log_rotation_age=0 |
|||
log_checkpoints=on |
|||
log_hostname=on |
|||
log_line_prefix='%t [%p]: [%l-1] db=%d,user=%u ' |
|||
|
|||
|
|||
Configuration |
|||
============= |
|||
|
|||
By default profiler module adds two system parameters |
|||
- exclude_fnames > '/.repo_requirements,~/odoo-8.0,/usr/,>' |
|||
- exclude_query > 'ir_translation'. |
|||
|
|||
These parameters can be configurated in order to exclude some outputs from |
|||
profiling stats or pgbadger output. |
|||
|
|||
Usage |
|||
===== |
|||
|
|||
After installation, a player is add on the header bar, with following buttons: |
|||
|
|||
- .. figure:: static/description/player.png |
|||
:alt: Player to manage profiler |
|||
|
|||
|
|||
* Start profiling |
|||
- .. figure:: static/description/start_profiling.png |
|||
:alt: Start profiling |
|||
:height: 35px |
|||
* Stop profiling |
|||
- .. figure:: static/description/stop_profiling.png |
|||
:alt: Stop profiling |
|||
:height: 35px |
|||
* Download stats: download stats file |
|||
- .. figure:: static/description/dump_stats.png |
|||
:alt: Download cprofile stats file |
|||
:height: 35px |
|||
* Clear stats |
|||
- .. figure:: static/description/clear_stats.png |
|||
:alt: Clear and remove stats file |
|||
:height: 35px |
|||
|
|||
|
|||
Credits |
|||
======= |
|||
|
|||
Contributors |
|||
------------ |
|||
|
|||
* Georges Racinet <gracinet@anybox.fr> |
|||
- Remotely inspired from ZopeProfiler, although there is no online visualisation and there may never be one. |
|||
* Moisés López <moylop260@vauxoo.com> |
|||
* Hugo Adan <hugo@vauxoo.com> |
|||
|
|||
Maintainer |
|||
---------- |
|||
|
|||
.. image:: https://odoo-community.org/logo.png |
|||
:alt: Odoo Community Association |
|||
:target: https://odoo-community.org |
|||
|
|||
This module is maintained by the OCA. |
|||
|
|||
OCA, or the Odoo Community Association, is a nonprofit organization whose |
|||
mission is to support the collaborative development of Odoo features and |
|||
promote its widespread use. |
|||
|
|||
To contribute to this module, please visit https://odoo-community.org. |
@ -1,32 +1,6 @@ |
|||
import controllers # noqa |
|||
from cProfile import Profile |
|||
|
|||
|
|||
def patch_openerp(): |
|||
"""Modify OpenERP/Odoo entry points so that profile can record. |
|||
|
|||
Odoo is a multi-threaded program. Therefore, the :data:`profile` object |
|||
needs to be enabled/disabled each in each thread to capture all the |
|||
execution. |
|||
|
|||
For instance, OpenERP 7 spawns a new thread for each request. |
|||
""" |
|||
from openerp.addons.web.http import JsonRequest |
|||
from .core import profiling |
|||
orig_dispatch = JsonRequest.dispatch |
|||
|
|||
def dispatch(*args, **kwargs): |
|||
with profiling(): |
|||
return orig_dispatch(*args, **kwargs) |
|||
JsonRequest.dispatch = dispatch |
|||
|
|||
|
|||
def create_profile(): |
|||
"""Create the global, shared profile object.""" |
|||
from . import core |
|||
core.profile = Profile() |
|||
|
|||
|
|||
def post_load(): |
|||
create_profile() |
|||
patch_openerp() |
|||
# coding: utf-8 |
|||
# License AGPL-3 or later (http://www.gnu.org/licenses/lgpl). |
|||
# Copyright 2014 Anybox <http://anybox.fr> |
|||
# Copyright 2016 Vauxoo (https://www.vauxoo.com) <info@vauxoo.com> |
|||
from . import controllers |
|||
from .hooks import post_load |
@ -1,45 +1,32 @@ |
|||
#============================================================================== |
|||
# = |
|||
# profiler module for OpenERP, cProfile integration for Odoo/OpenERP |
|||
# Copyright (C) 2014 Anybox <http://anybox.fr> |
|||
# = |
|||
# This file is a part of profiler |
|||
# = |
|||
# profiler is free software: you can redistribute it and/or modify |
|||
# it under the terms of the GNU Affero General Public License v3 or later |
|||
# as published by the Free Software Foundation, either version 3 of the |
|||
# License, or (at your option) any later version. |
|||
# = |
|||
# profiler is distributed in the hope that it will be useful, |
|||
# but WITHOUT ANY WARRANTY; without even the implied warranty of |
|||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
|||
# GNU Affero General Public License v3 or later for more details. |
|||
# = |
|||
# You should have received a copy of the GNU Affero General Public License |
|||
# v3 or later along with this program. |
|||
# If not, see <http://www.gnu.org/licenses/>. |
|||
# = |
|||
#============================================================================== |
|||
# coding: utf-8 |
|||
# License AGPL-3 or later (http://www.gnu.org/licenses/lgpl). |
|||
# Copyright 2014 Anybox <http://anybox.fr> |
|||
# Copyright 2016 Vauxoo (https://www.vauxoo.com) <info@vauxoo.com> |
|||
|
|||
{ |
|||
'name': 'profiler', |
|||
'version': '8.0.0.1', |
|||
'category': 'devtools', |
|||
'description': """ |
|||
cprofile integration for Odoo/OpenERP. Check the Profiler menu in admin menu |
|||
""", |
|||
'author': 'Georges Racinet', |
|||
'website': 'http://anybox.fr', |
|||
'depends': ['base', 'web'], |
|||
'version': '8.0.1.0.0', |
|||
'category': 'Tools', |
|||
'license': 'AGPL-3', |
|||
'author': 'Anybox, Vauxoo, Odoo Community Association (OCA)', |
|||
'website': 'https://odoo-community.org', |
|||
'depends': ['website'], |
|||
'data': [ |
|||
'data/profiler_excluding.xml', |
|||
'security/group.xml', |
|||
'views/profiler.xml' |
|||
'views/profiler.xml', |
|||
'views/assets.xml', |
|||
], |
|||
'external_dependencies': { |
|||
'python': [ |
|||
'pstats_print2list', |
|||
], |
|||
}, |
|||
'qweb': [ |
|||
'static/src/xml/player.xml', |
|||
], |
|||
'installable': True, |
|||
'application': False, |
|||
'auto_install': False, |
|||
'license': 'AGPL-3', |
|||
'post_load': 'post_load', |
|||
} |
@ -1,70 +0,0 @@ |
|||
import os |
|||
import logging |
|||
from datetime import datetime |
|||
from tempfile import mkstemp |
|||
import openerp.addons.web.http as openerpweb |
|||
from . import core |
|||
|
|||
logger = logging.getLogger(__name__) |
|||
|
|||
|
|||
class ProfilerController(openerpweb.Controller): |
|||
|
|||
_cp_path = '/web/profiler' |
|||
|
|||
player_state = 'profiler_player_clear' |
|||
"""Indicate the state(css class) of the player: |
|||
|
|||
* profiler_player_clear |
|||
* profiler_player_enabled |
|||
* profiler_player_disabled |
|||
""" |
|||
|
|||
@openerpweb.jsonrequest |
|||
def enable(self, request): |
|||
logger.info("Enabling") |
|||
core.enabled = True |
|||
ProfilerController.player_state = 'profiler_player_enabled' |
|||
|
|||
@openerpweb.jsonrequest |
|||
def disable(self, request): |
|||
logger.info("Disabling") |
|||
core.enabled = False |
|||
ProfilerController.player_state = 'profiler_player_disabled' |
|||
|
|||
@openerpweb.jsonrequest |
|||
def clear(self, request): |
|||
core.profile.clear() |
|||
logger.info("Cleared stats") |
|||
ProfilerController.player_state = 'profiler_player_clear' |
|||
|
|||
@openerpweb.httprequest |
|||
def dump(self, request, token): |
|||
"""Provide the stats as a file download. |
|||
|
|||
Uses a temporary file, because apparently there's no API to |
|||
dump stats in a stream directly. |
|||
""" |
|||
handle, path = mkstemp(prefix='profiling') |
|||
core.profile.dump_stats(path) |
|||
stream = os.fdopen(handle) |
|||
os.unlink(path) # TODO POSIX only ? |
|||
stream.seek(0) |
|||
filename = 'openerp_%s.stats' % datetime.now().isoformat() |
|||
# can't close the stream even in a context manager: it'll be needed |
|||
# after the return from this method, we'll let Python's GC do its job |
|||
return request.make_response( |
|||
stream, |
|||
headers=[ |
|||
('Content-Disposition', |
|||
'attachment; filename="%s"' % filename), |
|||
('Content-Type', 'application/octet-stream') |
|||
], cookies={'fileToken': token}) |
|||
|
|||
@openerpweb.jsonrequest |
|||
def initial_state(self, request): |
|||
user = request.session.model('res.users') |
|||
return { |
|||
'has_player_group': user.has_group('profiler.group_profiler_player'), |
|||
'player_state': ProfilerController.player_state, |
|||
} |
@ -0,0 +1,5 @@ |
|||
# coding: utf-8 |
|||
# License AGPL-3 or later (http://www.gnu.org/licenses/lgpl). |
|||
# Copyright 2014 Anybox <http://anybox.fr> |
|||
# Copyright 2016 Vauxoo (https://www.vauxoo.com) <info@vauxoo.com> |
|||
from . import main |
@ -0,0 +1,228 @@ |
|||
# coding: utf-8 |
|||
# License AGPL-3 or later (http://www.gnu.org/licenses/lgpl). |
|||
# Copyright 2014 Anybox <http://anybox.fr> |
|||
# Copyright 2016 Vauxoo (https://www.vauxoo.com) <info@vauxoo.com> |
|||
import errno |
|||
import logging |
|||
import os |
|||
import sys |
|||
import tempfile |
|||
from cStringIO import StringIO |
|||
from datetime import datetime |
|||
|
|||
from openerp import http, sql_db, tools |
|||
from openerp.addons.web.controllers.main import content_disposition |
|||
from openerp.http import request |
|||
from openerp.service.db import dump_db_manifest |
|||
from openerp.tools.misc import find_in_path |
|||
|
|||
from ..hooks import CoreProfile as core |
|||
|
|||
_logger = logging.getLogger(__name__) |
|||
|
|||
try: |
|||
from pstats_print2list import get_pstats_print2list, print_pstats_list |
|||
except ImportError as err: # pragma: no cover |
|||
_logger.debug(err) |
|||
|
|||
DFTL_LOG_PATH = '/var/lib/postgresql/%s/main/pg_log/postgresql.log' |
|||
|
|||
PGOPTIONS = ( |
|||
'-c client_min_messages=notice -c log_min_messages=warning ' |
|||
'-c log_min_error_statement=error ' |
|||
'-c log_min_duration_statement=0 -c log_connections=on ' |
|||
'-c log_disconnections=on -c log_duration=off ' |
|||
'-c log_error_verbosity=verbose -c log_lock_waits=on ' |
|||
'-c log_statement=none -c log_temp_files=0 ' |
|||
) |
|||
|
|||
|
|||
class Capturing(list): |
|||
def __enter__(self): |
|||
self._stdout = sys.stdout |
|||
sys.stdout = self._stringio = StringIO() |
|||
return self |
|||
|
|||
def __exit__(self, *args): |
|||
self.extend(self._stringio.getvalue().splitlines()) |
|||
del self._stringio # free up some memory |
|||
sys.stdout = self._stdout |
|||
|
|||
|
|||
class ProfilerController(http.Controller): |
|||
|
|||
_cp_path = '/web/profiler' |
|||
|
|||
player_state = 'profiler_player_clear' |
|||
begin_date = '' |
|||
end_date = '' |
|||
"""Indicate the state(css class) of the player: |
|||
|
|||
* profiler_player_clear |
|||
* profiler_player_enabled |
|||
* profiler_player_disabled |
|||
""" |
|||
|
|||
@http.route(['/web/profiler/enable'], type='json', auth="user") |
|||
def enable(self): |
|||
_logger.info("Enabling") |
|||
core.enabled = True |
|||
ProfilerController.begin_date = datetime.now().strftime( |
|||
"%Y-%m-%d %H:%M:%S") |
|||
ProfilerController.player_state = 'profiler_player_enabled' |
|||
os.environ['PGOPTIONS'] = PGOPTIONS |
|||
self.empty_cursor_pool() |
|||
|
|||
@http.route(['/web/profiler/disable'], type='json', auth="user") |
|||
def disable(self, **post): |
|||
_logger.info("Disabling") |
|||
core.enabled = False |
|||
ProfilerController.end_date = datetime.now().strftime( |
|||
"%Y-%m-%d %H:%M:%S") |
|||
ProfilerController.player_state = 'profiler_player_disabled' |
|||
os.environ.pop("PGOPTIONS", None) |
|||
self.empty_cursor_pool() |
|||
|
|||
@http.route(['/web/profiler/clear'], type='json', auth="user") |
|||
def clear(self, **post): |
|||
core.profile.clear() |
|||
_logger.info("Cleared stats") |
|||
ProfilerController.player_state = 'profiler_player_clear' |
|||
ProfilerController.end_date = '' |
|||
ProfilerController.begin_date = '' |
|||
|
|||
@http.route(['/web/profiler/dump'], type='http', auth="user") |
|||
def dump(self, token, **post): |
|||
"""Provide the stats as a file download. |
|||
|
|||
Uses a temporary file, because apparently there's no API to |
|||
dump stats in a stream directly. |
|||
""" |
|||
exclude_fname = self.get_exclude_fname() |
|||
with tools.osutil.tempdir() as dump_dir: |
|||
ts = datetime.now().strftime("%Y-%m-%d_%H-%M-%S") |
|||
filename = 'openerp_%s' % ts |
|||
stats_path = os.path.join(dump_dir, '%s.stats' % filename) |
|||
core.profile.dump_stats(stats_path) |
|||
_logger.info("Pstats Command:") |
|||
params = {'fnames': stats_path, 'sort': 'cumulative', 'limit': 45, |
|||
'exclude_fnames': exclude_fname} |
|||
_logger.info( |
|||
"fnames=%(fnames)s, sort=%(sort)s," |
|||
" limit=%(limit)s, exclude_fnames=%(exclude_fnames)s", params) |
|||
pstats_list = get_pstats_print2list(**params) |
|||
with Capturing() as output: |
|||
print_pstats_list(pstats_list) |
|||
result_path = os.path.join(dump_dir, '%s.txt' % filename) |
|||
with open(result_path, "a") as res_file: |
|||
for line in output: |
|||
res_file.write('%s\n' % line) |
|||
# PG_BADGER |
|||
self.dump_pgbadger(dump_dir, 'pgbadger_output.txt', request.cr) |
|||
t_zip = tempfile.TemporaryFile() |
|||
tools.osutil.zip_dir(dump_dir, t_zip, include_dir=False) |
|||
t_zip.seek(0) |
|||
headers = [ |
|||
('Content-Type', 'application/octet-stream; charset=binary'), |
|||
('Content-Disposition', content_disposition( |
|||
'%s.zip' % filename))] |
|||
_logger.info('Download Profiler zip: %s', t_zip.name) |
|||
return request.make_response( |
|||
t_zip, headers=headers, cookies={'fileToken': token}) |
|||
|
|||
@http.route(['/web/profiler/initial_state'], type='json', auth="user") |
|||
def initial_state(self, **post): |
|||
user = request.env['res.users'].browse(request.uid) |
|||
return { |
|||
'has_player_group': user.has_group( |
|||
'profiler.group_profiler_player'), |
|||
'player_state': ProfilerController.player_state, |
|||
} |
|||
|
|||
def dump_pgbadger(self, dir_dump, output, cursor): |
|||
pgbadger = find_in_path("pgbadger") |
|||
if not pgbadger: |
|||
_logger.error("Pgbadger not found") |
|||
return |
|||
filename = os.path.join(dir_dump, output) |
|||
pg_version = dump_db_manifest(cursor)['pg_version'] |
|||
log_path = os.environ.get('PG_LOG_PATH', DFTL_LOG_PATH % pg_version) |
|||
if not os.path.exists(os.path.dirname(filename)): |
|||
try: |
|||
os.makedirs(os.path.dirname(filename)) |
|||
except OSError as exc: |
|||
# error is different than File exists |
|||
if exc.errno != errno.EEXIST: |
|||
_logger.error("Folder %s can not be created", |
|||
os.path.dirname(filename)) |
|||
return |
|||
_logger.info("Generating PG Badger report.") |
|||
exclude_query = self.get_exclude_query() |
|||
dbname = cursor.dbname |
|||
command = [ |
|||
pgbadger, '-f', 'stderr', '-T', 'Odoo-Profiler', |
|||
'-o', '-', '-d', dbname, '-b', ProfilerController.begin_date, |
|||
'-e', ProfilerController.end_date, '--sample', '2', |
|||
'--disable-type', '--disable-error', '--disable-hourly', |
|||
'--disable-session', '--disable-connection', |
|||
'--disable-temporary', '--quiet'] |
|||
command.extend(exclude_query) |
|||
command.append(log_path) |
|||
|
|||
_logger.info("Pgbadger Command:") |
|||
_logger.info(command) |
|||
result = tools.exec_command_pipe(*command) |
|||
with open(filename, 'w') as fw: |
|||
fw.write(result[1].read()) |
|||
_logger.info("Done") |
|||
|
|||
def get_exclude_fname(self): |
|||
efnameid = request.env.ref( |
|||
'profiler.default_exclude_fnames_pstas', raise_if_not_found=False) |
|||
if not efnameid: |
|||
return [] |
|||
return [os.path.expanduser(path) |
|||
for path in efnameid and efnameid.value.strip(',').split(',') |
|||
if path] |
|||
|
|||
def get_exclude_query(self): |
|||
"""Example '^(COPY|COMMIT)' |
|||
""" |
|||
equeryid = request.env.ref( |
|||
'profiler.default_exclude_query_pgbadger', |
|||
raise_if_not_found=False) |
|||
if not equeryid: |
|||
return [] |
|||
exclude_queries = [] |
|||
for path in equeryid and equeryid.value.strip(',').split(','): |
|||
exclude_queries.extend( |
|||
['--exclude-query', '"^(%s)" ' % path.encode('UTF-8')]) |
|||
return exclude_queries |
|||
|
|||
def empty_cursor_pool(self): |
|||
"""This method cleans (rollback) all current transactions over actual |
|||
cursor in order to avoid errors with waiting transactions. |
|||
- request.cr.rollback() |
|||
|
|||
Also connections on current database's only are closed by the next |
|||
statement |
|||
- dsn = openerp.sql_db.dsn(request.cr.dbname) |
|||
- openerp.sql_db._Pool.close_all(dsn[1]) |
|||
Otherwise next error will be trigger |
|||
'InterfaceError: connection already closed' |
|||
|
|||
Finally new cursor is assigned to the request object, this cursor will |
|||
take the os.environ setted. In this case the os.environ is setted with |
|||
all 'PGOPTIONS' required to log all sql transactions in postgres.log |
|||
file. |
|||
|
|||
If this method is called one more time, it will create a new cursor and |
|||
take the os.environ again, this is usefully if we want to reset |
|||
'PGOPTIONS' |
|||
|
|||
""" |
|||
request.cr._cnx.reset() |
|||
dsn = sql_db.dsn(request.cr.dbname) |
|||
sql_db._Pool.close_all(dsn[1]) |
|||
db = sql_db.db_connect(request.cr.dbname) |
|||
request._cr = db.cursor() |
@ -1,20 +0,0 @@ |
|||
from contextlib import contextmanager |
|||
profile = None |
|||
"""The thread-shared profile object. |
|||
""" |
|||
|
|||
enabled = False |
|||
"""Indicates if the whole profiling functionality is globally active or not. |
|||
""" |
|||
|
|||
|
|||
@contextmanager |
|||
def profiling(): |
|||
"""Thread local profile management, according to the shared :data:`enabled` |
|||
""" |
|||
if enabled: |
|||
profile.enable() |
|||
yield |
|||
|
|||
if enabled: |
|||
profile.disable() |
@ -0,0 +1,16 @@ |
|||
<?xml version="1.0" encoding="utf-8"?> |
|||
<openerp> |
|||
<data> |
|||
<!-- exclude_fnames de pstats_print2list --> |
|||
<record id="default_exclude_fnames_pstas" model="ir.config_parameter"> |
|||
<field name="key">exclude_fnames</field> |
|||
<field name="value">/.repo_requirements,~/odoo-8.0,/usr/,></field> |
|||
</record> |
|||
|
|||
<!-- exclude-query de pgbader --> |
|||
<record id="default_exclude_query_pgbadger" model="ir.config_parameter"> |
|||
<field name="key">exclude_query</field> |
|||
<field name="value">ir_translation</field> |
|||
</record> |
|||
</data> |
|||
</openerp> |
@ -0,0 +1,103 @@ |
|||
# coding: utf-8 |
|||
# License AGPL-3 or later (http://www.gnu.org/licenses/lgpl). |
|||
# Copyright 2014 Anybox <http://anybox.fr> |
|||
# Copyright 2016 Vauxoo (https://www.vauxoo.com) <info@vauxoo.com> |
|||
import logging |
|||
import os |
|||
from contextlib import contextmanager |
|||
from cProfile import Profile |
|||
|
|||
import openerp |
|||
from openerp import sql_db |
|||
from openerp.http import WebRequest |
|||
from openerp.service.server import ThreadedServer |
|||
|
|||
_logger = logging.getLogger(__name__) |
|||
|
|||
|
|||
class CoreProfile(object): |
|||
"""The thread-shared profile object""" |
|||
profile = None |
|||
# Indicates if the whole profiling functionality is globally active or not. |
|||
enabled = False |
|||
|
|||
|
|||
@contextmanager |
|||
def profiling(): |
|||
"""Thread local profile management, according to the shared :data:`enabled` |
|||
""" |
|||
if CoreProfile.enabled: |
|||
CoreProfile.profile.enable() |
|||
yield |
|||
|
|||
if CoreProfile.enabled: |
|||
CoreProfile.profile.disable() |
|||
|
|||
|
|||
def patch_odoo(): |
|||
"""Modify Odoo entry points so that profile can record. |
|||
|
|||
Odoo is a multi-threaded program. Therefore, the :data:`profile` object |
|||
needs to be enabled/disabled each in each thread to capture all the |
|||
execution. |
|||
|
|||
For instance, Odoo spawns a new thread for each request. |
|||
|
|||
Modify database connect method to add options to enable postgresql logging |
|||
based on PGOPTIONS environment variable |
|||
""" |
|||
_logger.info('Patching openerp.http.WebRequest._call_function') |
|||
webreq_f_origin = WebRequest._call_function |
|||
|
|||
def webreq_f(*args, **kwargs): |
|||
with profiling(): |
|||
return webreq_f_origin(*args, **kwargs) |
|||
WebRequest._call_function = webreq_f |
|||
|
|||
_logger.info('Patching openerp.sql_db.db_connect') |
|||
db_connect_origin = sql_db.db_connect |
|||
|
|||
def dbconnect_f(to, *args, **kwargs): |
|||
try: |
|||
to += " options='%s' " % (os.environ['PGOPTIONS'] or '') |
|||
except KeyError: |
|||
pass |
|||
return db_connect_origin(to, *args, **kwargs) |
|||
sql_db.db_connect = dbconnect_f |
|||
|
|||
|
|||
def dump_stats(): |
|||
"""Dump stats to standard file""" |
|||
_logger.info('Dump stats') |
|||
CoreProfile.profile.dump_stats( |
|||
os.path.expanduser('~/.openerp_server.stats')) |
|||
|
|||
|
|||
def create_profile(): |
|||
"""Create the global, shared profile object.""" |
|||
_logger.info('Create profile') |
|||
CoreProfile.profile = Profile() |
|||
|
|||
|
|||
def patch_stop(): |
|||
"""When the server is stopped then save the result of cProfile stats""" |
|||
origin_stop = ThreadedServer.stop |
|||
|
|||
_logger.info('Patching openerp.service.server.ThreadedServer.stop') |
|||
|
|||
def stop(*args, **kwargs): |
|||
if openerp.tools.config['test_enable']: |
|||
dump_stats() |
|||
return origin_stop(*args, **kwargs) |
|||
ThreadedServer.stop = stop |
|||
|
|||
|
|||
def post_load(): |
|||
_logger.info('Post load') |
|||
create_profile() |
|||
patch_odoo() |
|||
if openerp.tools.config['test_enable']: |
|||
# Enable profile in test mode for orm methods. |
|||
_logger.info('Enabling profiler and apply patch') |
|||
CoreProfile.enabled = True |
|||
patch_stop() |
@ -1,11 +1,11 @@ |
|||
<?xml version="1.0" encoding="UTF-8"?> |
|||
<openerp> |
|||
<data noupdate="1"> |
|||
<data noupdate="1"> |
|||
<record id="group_profiler_player" model="res.groups"> |
|||
<field name="name">Profiling capability</field> |
|||
</record> |
|||
<record model="res.users" id="base.user_root"> |
|||
<field name="groups_id" eval="[(4, ref('profiler.group_profiler_player'))]"/> |
|||
</record> |
|||
</data> |
|||
</data> |
|||
</openerp> |
Before Width: 277 | Height: 58 | Size: 8.2 KiB After Width: 277 | Height: 58 | Size: 8.2 KiB |
Before Width: 279 | Height: 52 | Size: 8.1 KiB After Width: 279 | Height: 52 | Size: 8.1 KiB |
Before Width: 76 | Height: 30 | Size: 1.5 KiB After Width: 76 | Height: 30 | Size: 1.5 KiB |
Before Width: 280 | Height: 49 | Size: 7.1 KiB After Width: 280 | Height: 49 | Size: 7.1 KiB |
Before Width: 275 | Height: 57 | Size: 7.2 KiB After Width: 275 | Height: 57 | Size: 7.2 KiB |
@ -1,21 +1,21 @@ |
|||
.openerp .oe_topbar_item.profiler_player img { |
|||
.oe_topbar_item.profiler_player i { |
|||
cursor: pointer; |
|||
} |
|||
.openerp .oe_topbar_item.profiler_player.profiler_player_enabled img.profiler_enable { |
|||
.oe_topbar_item.profiler_player.profiler_player_enabled a.profiler_enable { |
|||
display: None; |
|||
} |
|||
.openerp .oe_topbar_item.profiler_player.profiler_player_enabled img.profiler_dump { |
|||
.oe_topbar_item.profiler_player.profiler_player_enabled a.profiler_dump { |
|||
display: None; |
|||
} |
|||
.openerp .oe_topbar_item.profiler_player.profiler_player_disabled img.profiler_disable { |
|||
.oe_topbar_item.profiler_player.profiler_player_disabled a.profiler_disable { |
|||
display: None; |
|||
} |
|||
.openerp .oe_topbar_item.profiler_player.profiler_player_clear img.profiler_disable { |
|||
.oe_topbar_item.profiler_player.profiler_player_clear a.profiler_disable { |
|||
display: None; |
|||
} |
|||
.openerp .oe_topbar_item.profiler_player.profiler_player_clear img.profiler_clear { |
|||
.oe_topbar_item.profiler_player.profiler_player_clear a.profiler_clear { |
|||
display: None; |
|||
} |
|||
.openerp .oe_topbar_item.profiler_player.profiler_player_clear img.profiler_dump { |
|||
.oe_topbar_item.profiler_player.profiler_player_clear a.profiler_dump { |
|||
display: None; |
|||
} |
Before Width: 80 | Height: 80 | Size: 2.1 KiB |
@ -0,0 +1,52 @@ |
|||
(function(){ |
|||
'use_strict'; |
|||
openerp.Tour.register({ |
|||
id: 'profile_run', |
|||
name: 'Profile run', |
|||
path: '/web', |
|||
mode: 'test', |
|||
steps: [ |
|||
{ |
|||
title: 'Check if is cleared', |
|||
waitFor: 'li.oe_topbar_item.profiler_player.profiler_player_clear' |
|||
}, |
|||
{ |
|||
title: 'Start profiling', |
|||
onload: function () { |
|||
$('a.profiler_enable').trigger('click'); |
|||
} |
|||
}, |
|||
{ |
|||
title: 'Check if is enabled', |
|||
waitFor: 'li.oe_topbar_item.profiler_player.profiler_player_enabled' |
|||
}, |
|||
{ |
|||
title: 'Stop profiling', |
|||
onload: function () { |
|||
$('a.profiler_disable').trigger('click'); |
|||
} |
|||
}, |
|||
{ |
|||
title: 'Check if is disabled', |
|||
waitFor: 'li.oe_topbar_item.profiler_player.profiler_player_disabled' |
|||
}, |
|||
{ |
|||
title: 'Dump profiling', |
|||
onload: function () { |
|||
$('a.profiler_dump').trigger('click'); |
|||
} |
|||
}, |
|||
{ |
|||
title: 'Clear profiling', |
|||
onload: function () { |
|||
$('a.profiler_clear').trigger('click'); |
|||
} |
|||
}, |
|||
{ |
|||
title: 'Check if is cleared again', |
|||
waitFor: 'li.oe_topbar_item.profiler_player.profiler_player_clear' |
|||
}, |
|||
|
|||
] |
|||
}); |
|||
}()); |
@ -0,0 +1,24 @@ |
|||
#oe_main_menu_navbar .o_menu_systray { |
|||
|
|||
.oe_topbar_item.profiler_player i { |
|||
cursor: pointer; |
|||
} |
|||
.oe_topbar_item.profiler_player.profiler_player_enabled a.profiler_enable { |
|||
display: None; |
|||
} |
|||
.oe_topbar_item.profiler_player.profiler_player_enabled a.profiler_dump { |
|||
display: None; |
|||
} |
|||
.oe_topbar_item.profiler_player.profiler_player_disabled a.profiler_disable { |
|||
display: None; |
|||
} |
|||
.oe_topbar_item.profiler_player.profiler_player_clear a.profiler_disable { |
|||
display: None; |
|||
} |
|||
.oe_topbar_item.profiler_player.profiler_player_clear a.profiler_clear { |
|||
display: None; |
|||
} |
|||
.oe_topbar_item.profiler_player.profiler_player_clear a.profiler_dump { |
|||
display: None; |
|||
} |
|||
} |
@ -1,18 +1,26 @@ |
|||
<templates> |
|||
<t t-name="profiler.player"> |
|||
<div class="oe_topbar_item profiler_player"> |
|||
<img class="profiler_enable" |
|||
title="Start profiling" |
|||
src="/web/static/src/img/icons/gtk-media-record.png"/> |
|||
<img class="profiler_disable" |
|||
title="Stop profiling" |
|||
src="/web/static/src/img/icons/gtk-media-pause.png"/> |
|||
<img class="profiler_clear" |
|||
title="Clear profiling" |
|||
src="/web/static/src/img/icons/gtk-media-stop.png"/> |
|||
<img class="profiler_dump" |
|||
title="Download stats" |
|||
src="/web/static/src/img/icons/gtk-floppy.png"/> |
|||
</div> |
|||
<li class="oe_topbar_item profiler_player"> |
|||
<a href="#" class="profiler_enable"> |
|||
<i class="fa fa-play" title="Start profiling"/> |
|||
</a> |
|||
</li> |
|||
<li class="oe_topbar_item profiler_player"> |
|||
<a href="#" class="profiler_disable"> |
|||
<!-- <i class="fa fa-pause" title="Stop profiling"/> --> |
|||
<i class="fa fa-stop" title="Stop profiling"/> |
|||
</a> |
|||
</li> |
|||
<li class="oe_topbar_item profiler_player"> |
|||
<a href="#" class="profiler_clear"> |
|||
<!-- <i class="fa fa-stop" title="Clear profiling"/> --> |
|||
<i class="fa fa-refresh" title="Clear profiling"/> |
|||
</a> |
|||
</li> |
|||
<li class="oe_topbar_item profiler_player"> |
|||
<a href="#" class="profiler_dump"> |
|||
<i class="fa fa-floppy-o" title="Download stats"/> |
|||
</a> |
|||
</li> |
|||
</t> |
|||
</templates> |
@ -0,0 +1,5 @@ |
|||
# coding: utf-8 |
|||
# License AGPL-3 or later (http://www.gnu.org/licenses/lgpl). |
|||
# Copyright 2014 Anybox <http://anybox.fr> |
|||
# Copyright 2016 Vauxoo (https://www.vauxoo.com) <info@vauxoo.com> |
|||
from . import test_profiler |
@ -0,0 +1,17 @@ |
|||
# coding: utf-8 |
|||
# License AGPL-3 or later (http://www.gnu.org/licenses/lgpl). |
|||
# Copyright 2016 Vauxoo (https://www.vauxoo.com) <info@vauxoo.com> |
|||
|
|||
import unittest |
|||
|
|||
from openerp import tests |
|||
|
|||
|
|||
@tests.at_install(False) |
|||
@tests.post_install(True) |
|||
class TestProfiler(tests.HttpCase): |
|||
|
|||
@unittest.skip("phantomjs tests async for 8.0 are so flaky") |
|||
def test_profiler_tour(self): |
|||
self.phantom_js('/web', "openerp.Tour.run('profile_run', 'test')", |
|||
'openerp.Tour.tours.profile_run', login='admin') |
@ -0,0 +1,12 @@ |
|||
<?xml version="1.0" encoding="utf-8"?> |
|||
<openerp> |
|||
<data> |
|||
<template id="assets_backend_test_profiler" name="Profiler Tests" |
|||
inherit_id="website.assets_backend"> |
|||
<xpath expr="//script[last()]" position="after"> |
|||
<script src="/profiler/static/src/js/test_profiler.js" |
|||
type="text/javascript"/> |
|||
</xpath> |
|||
</template> |
|||
</data> |
|||
</openerp> |
@ -1,14 +1,11 @@ |
|||
<?xml version="1.0" encoding="utf-8"?> |
|||
<!-- vim:fdn=3: --> |
|||
<openerp> |
|||
<data> |
|||
|
|||
<data> |
|||
<template id="profiler_assets_backend" name="profiler assets" inherit_id="web.assets_backend"> |
|||
<xpath expr="." position="inside"> |
|||
<link rel="stylesheet" href="/profiler/static/src/css/player.css" /> |
|||
<script type="text/javascript" src="/profiler/static/src/js/player.js" /> |
|||
</xpath> |
|||
</template> |
|||
|
|||
</data> |
|||
</data> |
|||
</openerp> |