From e6e75bd8148d2abd4cef80f164d83d7bd6c218ce Mon Sep 17 00:00:00 2001 From: Nils Hamerlinck Date: Sun, 4 Aug 2019 17:56:01 +0700 Subject: [PATCH] [ADD] bus_alt_connection --- bus_alt_connection/README.rst | 187 +++++++ bus_alt_connection/__init__.py | 3 + bus_alt_connection/__manifest__.py | 15 + bus_alt_connection/models/__init__.py | 3 + bus_alt_connection/models/bus.py | 77 +++ bus_alt_connection/readme/CONFIGURE.rst | 15 + bus_alt_connection/readme/CONTRIBUTORS.rst | 1 + bus_alt_connection/readme/DESCRIPTION.rst | 79 +++ bus_alt_connection/readme/INSTALL.rst | 13 + .../static/description/icon.png | Bin 0 -> 9455 bytes .../static/description/index.html | 498 ++++++++++++++++++ 11 files changed, 891 insertions(+) create mode 100644 bus_alt_connection/README.rst create mode 100644 bus_alt_connection/__init__.py create mode 100644 bus_alt_connection/__manifest__.py create mode 100644 bus_alt_connection/models/__init__.py create mode 100644 bus_alt_connection/models/bus.py create mode 100644 bus_alt_connection/readme/CONFIGURE.rst create mode 100644 bus_alt_connection/readme/CONTRIBUTORS.rst create mode 100644 bus_alt_connection/readme/DESCRIPTION.rst create mode 100644 bus_alt_connection/readme/INSTALL.rst create mode 100644 bus_alt_connection/static/description/icon.png create mode 100644 bus_alt_connection/static/description/index.html diff --git a/bus_alt_connection/README.rst b/bus_alt_connection/README.rst new file mode 100644 index 000000000..f219115b1 --- /dev/null +++ b/bus_alt_connection/README.rst @@ -0,0 +1,187 @@ +================== +Bus Alt Connection +================== + +.. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! + !! This file is generated by oca-gen-addon-readme !! + !! changes will be overwritten. !! + !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! + +.. |badge1| image:: https://img.shields.io/badge/maturity-Beta-yellow.png + :target: https://odoo-community.org/page/development-status + :alt: Beta +.. |badge2| image:: https://img.shields.io/badge/licence-AGPL--3-blue.png + :target: http://www.gnu.org/licenses/agpl-3.0-standalone.html + :alt: License: AGPL-3 +.. |badge3| image:: https://img.shields.io/badge/github-OCA%2Fserver--tools-lightgray.png?logo=github + :target: https://github.com/OCA/server-tools/tree/12.0/bus_alt_connection + :alt: OCA/server-tools +.. |badge4| image:: https://img.shields.io/badge/weblate-Translate%20me-F47D42.png + :target: https://translation.odoo-community.org/projects/server-tools-12-0/server-tools-12-0-bus_alt_connection + :alt: Translate me on Weblate +.. |badge5| image:: https://img.shields.io/badge/runbot-Try%20me-875A7B.png + :target: https://runbot.odoo-community.org/runbot/149/12.0 + :alt: Try me on Runbot + +|badge1| |badge2| |badge3| |badge4| |badge5| + +This module makes it possible to use PgBouncer_ as a connection pooler +for odoo. + +.. _PgBouncer: https://pgbouncer.github.io/ + +Why isn't odoo's connection pooling good enough? +================================================ + +Odoo's builtin connection pooling works at process level: each Odoo process +has its own ConnectionPool_, limited to ``db_maxconn``. + +It does the job of re-using open connections available in the pool. +But it never closes these connections, `unless reaching db_maxconn`_. + +.. _ConnectionPool: https://github.com/odoo/odoo/blob/12.0/odoo/sql_db.py#L525 +.. _`unless reaching db_maxconn`: https://github.com/odoo/odoo/blob/12.0/odoo/sql_db.py#L593 + +In practice, we observe that each odoo worker will end up +with up to 3 open connection in its pool. +With 10 http workers, that's up to 30 connection continuously open just +for one single instance. + +Here comes PgBouncer +==================== + +PgBouncer will help to limit this number of open connections, +by sharing a pool of connections at the instance level, between +all workers. Odoo workers will still have up to 3 open connections, +but these will be connections to PgBouncer, that on its side will +close unnecessary connections to pg. + +This has proven to help performances on Odoo deployments with +multiple instances. + +It allows you to define how resources should be shared, +according to your priorities, e.g. : + +* key odoo instance on host A can open up to 30 connections +* while odoo instance on host B, dedicated to reports, + can open up to 10 connections only + +And most importantly, it helps you to ensure that +``max_connections`` will never be reached on pg server side. + + +Why is this module needed? +========================== + +When configuring PgBouncer, you can choose between 2 transaction pooling modes: + +* `pool_mode = session` +* `pool_mode = transaction` + +If we choose `pool_mode = session`, then one server connection will be tied +to a given odoo process until its death, which is exactly what we're trying +to change. Thus, to release the server connection once the transaction is +complete, we use `pool_mode = transaction`. + +This works fine, except for Odoo's longpolling features that relies +on the `LISTEN/NOTIFY`_ mechanism from pg, which is `not compatible`_ with that +mode. + +.. _`LISTEN/NOTIFY`: https://www.postgresql.org/docs/9.6/static/sql-notify.html +.. _`not compatible`: https://wiki.postgresql.org/wiki/PgBouncer + + +To be more precise, `NOTIFY` statements are properly transfered by PgBouncer +in that mode; only the `LISTEN` statement isn't (because it needs to keep the +server connection open). + +So for the unique "listening" connection per instance that requires this +statement (here_), we need odoo to connect directly to the pg server, bypassing +PgBouncer. + +That's what this module implements, by overriding the relevant method +of the Dispatcher_. + +.. _here: https://github.com/odoo/odoo/blob/12.0/addons/bus/models/bus.py#L166 +.. _Dispatcher: https://github.com/odoo/odoo/blob/12.0/addons/bus/models/bus.py#L105 + +**Table of contents** + +.. contents:: + :local: + +Installation +============ + +You don't need to install this module in the database(s) to enable it. + +But you need to load it server-wide: + +* By starting Odoo with ``--load=web,bus_alt_connection`` + +* Or by updating its configuration file: + +.. code-block:: ini + + [options] + (...) + server_wide_modules = web,bus_alt_connection + +Configuration +============= + +You need to define how to connect directly to the database: + +* Either by defining environment variables: + + - ``IMDISPATCHER_DB_HOST=db-01`` + - ``IMDISPATCHER_DB_PORT=5432`` + +* Or in Odoo's configuration file: + +.. code-block:: ini + + [options] + (...) + imdispatcher_db_host = db-01 + imdispatcher_db_port = 5432 + +Bug Tracker +=========== + +Bugs are tracked on `GitHub Issues `_. +In case of trouble, please check there if your issue has already been reported. +If you spotted it first, help us smashing it by providing a detailed and welcomed +`feedback `_. + +Do not contact contributors directly about support or help with technical issues. + +Credits +======= + +Authors +~~~~~~~ + +* Trobz + +Contributors +~~~~~~~~~~~~ + +* Nils Hamerlinck + +Maintainers +~~~~~~~~~~~ + +This module is maintained by the OCA. + +.. image:: https://odoo-community.org/logo.png + :alt: Odoo Community Association + :target: https://odoo-community.org + +OCA, or the Odoo Community Association, is a nonprofit organization whose +mission is to support the collaborative development of Odoo features and +promote its widespread use. + +This module is part of the `OCA/server-tools `_ project on GitHub. + +You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute. diff --git a/bus_alt_connection/__init__.py b/bus_alt_connection/__init__.py new file mode 100644 index 000000000..1bcfe0e2f --- /dev/null +++ b/bus_alt_connection/__init__.py @@ -0,0 +1,3 @@ +# Copyright 2019 Trobz +# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl). +from . import models diff --git a/bus_alt_connection/__manifest__.py b/bus_alt_connection/__manifest__.py new file mode 100644 index 000000000..9133093b9 --- /dev/null +++ b/bus_alt_connection/__manifest__.py @@ -0,0 +1,15 @@ +# Copyright 2019 Trobz +# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl). + +{ + 'name': 'Bus Alt Connection', + 'summary': 'Needed when using PgBouncer as a connection pooler', + 'version': '12.0.1.0.0', + 'author': 'Trobz,Odoo Community Association (OCA)', + 'website': 'https://github.com/OCA/server-tools', + 'category': 'Extra Tools', + 'license': 'AGPL-3', + 'depends': ['bus'], + 'installable': True, + 'auto_install': False, +} diff --git a/bus_alt_connection/models/__init__.py b/bus_alt_connection/models/__init__.py new file mode 100644 index 000000000..3dc9bd993 --- /dev/null +++ b/bus_alt_connection/models/__init__.py @@ -0,0 +1,3 @@ +# Copyright 2019 Trobz +# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl). +from . import bus diff --git a/bus_alt_connection/models/bus.py b/bus_alt_connection/models/bus.py new file mode 100644 index 000000000..d9c545e16 --- /dev/null +++ b/bus_alt_connection/models/bus.py @@ -0,0 +1,77 @@ +# Copyright 2019 Trobz +# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl). + +import os +import json +import logging +import select + +import psycopg2 +from psycopg2.extensions import ISOLATION_LEVEL_AUTOCOMMIT + +import odoo +from odoo.tools import config + +from odoo.addons.bus.models.bus import hashable, TIMEOUT +import odoo.addons.bus.models.bus + + +_logger = logging.getLogger(__name__) + + +def _connection_info_for(db_name): + db_or_uri, connection_info = odoo.sql_db.connection_info_for(db_name) + + for p in ('host', 'port'): + cfg = (os.environ.get('ODOO_IMDISPATCHER_DB_%s' % p.upper()) or + config.get('imdispatcher_db_' + p)) + + if cfg: + connection_info[p] = cfg + + return connection_info + + +class ImDispatch(odoo.addons.bus.models.bus.ImDispatch): + + def loop(self): + """ Dispatch postgres notifications to the relevant + polling threads/greenlets """ + connection_info = _connection_info_for('postgres') + _logger.info("Bus.loop listen imbus on db postgres " + "(via %(host)s:%(port)s)", + connection_info) + conn = psycopg2.connect(**connection_info) + conn.set_isolation_level(ISOLATION_LEVEL_AUTOCOMMIT) + with conn.cursor() as cr: + cr.execute("listen imbus") + conn.commit() + while True: + if select.select([conn], [], [], TIMEOUT) == ([], [], []): + pass + else: + conn.poll() + channels = [] + while conn.notifies: + channels.extend(json.loads(conn + .notifies.pop().payload)) + # dispatch to local threads/greenlets + events = set() + for channel in channels: + events.update(self.channels.pop(hashable(channel), + set())) + for event in events: + event.set() + + +odoo.addons.bus.models.bus.ImDispatch = ImDispatch + +# we can replace the existing dispatcher because its thread +# has not been started yet; indeed, since a2ed3d it only starts +# on first /poll request: +# https://github.com/odoo/odoo/commit/a2ed3d3d5bdb6025a1ba14ad557a115a86413e65 + +if not odoo.multi_process or odoo.evented: + dispatch = ImDispatch() + odoo.addons.bus.models.bus.dispatch = dispatch + odoo.addons.bus.controllers.main.dispatch = dispatch diff --git a/bus_alt_connection/readme/CONFIGURE.rst b/bus_alt_connection/readme/CONFIGURE.rst new file mode 100644 index 000000000..cd5021912 --- /dev/null +++ b/bus_alt_connection/readme/CONFIGURE.rst @@ -0,0 +1,15 @@ +You need to define how to connect directly to the database: + +* Either by defining environment variables: + + - ``IMDISPATCHER_DB_HOST=db-01`` + - ``IMDISPATCHER_DB_PORT=5432`` + +* Or in Odoo's configuration file: + +.. code-block:: ini + + [options] + (...) + imdispatcher_db_host = db-01 + imdispatcher_db_port = 5432 diff --git a/bus_alt_connection/readme/CONTRIBUTORS.rst b/bus_alt_connection/readme/CONTRIBUTORS.rst new file mode 100644 index 000000000..198a2f3bd --- /dev/null +++ b/bus_alt_connection/readme/CONTRIBUTORS.rst @@ -0,0 +1 @@ +* Nils Hamerlinck diff --git a/bus_alt_connection/readme/DESCRIPTION.rst b/bus_alt_connection/readme/DESCRIPTION.rst new file mode 100644 index 000000000..ea95393d2 --- /dev/null +++ b/bus_alt_connection/readme/DESCRIPTION.rst @@ -0,0 +1,79 @@ +This module makes it possible to use PgBouncer_ as a connection pooler +for odoo. + +.. _PgBouncer: https://pgbouncer.github.io/ + +Why isn't odoo's connection pooling good enough? +================================================ + +Odoo's builtin connection pooling works at process level: each Odoo process +has its own ConnectionPool_, limited to ``db_maxconn``. + +It does the job of re-using open connections available in the pool. +But it never closes these connections, `unless reaching db_maxconn`_. + +.. _ConnectionPool: https://github.com/odoo/odoo/blob/12.0/odoo/sql_db.py#L525 +.. _`unless reaching db_maxconn`: https://github.com/odoo/odoo/blob/12.0/odoo/sql_db.py#L593 + +In practice, we observe that each odoo worker will end up +with up to 3 open connection in its pool. +With 10 http workers, that's up to 30 connection continuously open just +for one single instance. + +Here comes PgBouncer +==================== + +PgBouncer will help to limit this number of open connections, +by sharing a pool of connections at the instance level, between +all workers. Odoo workers will still have up to 3 open connections, +but these will be connections to PgBouncer, that on its side will +close unnecessary connections to pg. + +This has proven to help performances on Odoo deployments with +multiple instances. + +It allows you to define how resources should be shared, +according to your priorities, e.g. : + +* key odoo instance on host A can open up to 30 connections +* while odoo instance on host B, dedicated to reports, + can open up to 10 connections only + +And most importantly, it helps you to ensure that +``max_connections`` will never be reached on pg server side. + + +Why is this module needed? +========================== + +When configuring PgBouncer, you can choose between 2 transaction pooling modes: + +* `pool_mode = session` +* `pool_mode = transaction` + +If we choose `pool_mode = session`, then one server connection will be tied +to a given odoo process until its death, which is exactly what we're trying +to change. Thus, to release the server connection once the transaction is +complete, we use `pool_mode = transaction`. + +This works fine, except for Odoo's longpolling features that relies +on the `LISTEN/NOTIFY`_ mechanism from pg, which is `not compatible`_ with that +mode. + +.. _`LISTEN/NOTIFY`: https://www.postgresql.org/docs/9.6/static/sql-notify.html +.. _`not compatible`: https://wiki.postgresql.org/wiki/PgBouncer + + +To be more precise, `NOTIFY` statements are properly transfered by PgBouncer +in that mode; only the `LISTEN` statement isn't (because it needs to keep the +server connection open). + +So for the unique "listening" connection per instance that requires this +statement (here_), we need odoo to connect directly to the pg server, bypassing +PgBouncer. + +That's what this module implements, by overriding the relevant method +of the Dispatcher_. + +.. _here: https://github.com/odoo/odoo/blob/12.0/addons/bus/models/bus.py#L166 +.. _Dispatcher: https://github.com/odoo/odoo/blob/12.0/addons/bus/models/bus.py#L105 diff --git a/bus_alt_connection/readme/INSTALL.rst b/bus_alt_connection/readme/INSTALL.rst new file mode 100644 index 000000000..c6a9fcbcf --- /dev/null +++ b/bus_alt_connection/readme/INSTALL.rst @@ -0,0 +1,13 @@ +You don't need to install this module in the database(s) to enable it. + +But you need to load it server-wide: + +* By starting Odoo with ``--load=web,bus_alt_connection`` + +* Or by updating its configuration file: + +.. code-block:: ini + + [options] + (...) + server_wide_modules = web,bus_alt_connection diff --git a/bus_alt_connection/static/description/icon.png b/bus_alt_connection/static/description/icon.png new file mode 100644 index 0000000000000000000000000000000000000000..3a0328b516c4980e8e44cdb63fd945757ddd132d GIT binary patch literal 9455 zcmW++2RxMjAAjx~&dlBk9S+%}OXg)AGE&Cb*&}d0jUxM@u(PQx^-s)697TX`ehR4?GS^qbkof1cslKgkU)h65qZ9Oc=ml_0temigYLJfnz{IDzUf>bGs4N!v3=Z3jMq&A#7%rM5eQ#dc?k~! zVpnB`o+K7|Al`Q_U;eD$B zfJtP*jH`siUq~{KE)`jP2|#TUEFGRryE2`i0**z#*^6~AI|YzIWy$Cu#CSLW3q=GA z6`?GZymC;dCPk~rBS%eCb`5OLr;RUZ;D`}um=H)BfVIq%7VhiMr)_#G0N#zrNH|__ zc+blN2UAB0=617@>_u;MPHN;P;N#YoE=)R#i$k_`UAA>WWCcEVMh~L_ zj--gtp&|K1#58Yz*AHCTMziU1Jzt_jG0I@qAOHsk$2}yTmVkBp_eHuY$A9)>P6o~I z%aQ?!(GqeQ-Y+b0I(m9pwgi(IIZZzsbMv+9w{PFtd_<_(LA~0H(xz{=FhLB@(1&qHA5EJw1>>=%q2f&^X>IQ{!GJ4e9U z&KlB)z(84HmNgm2hg2C0>WM{E(DdPr+EeU_N@57;PC2&DmGFW_9kP&%?X4}+xWi)( z;)z%wI5>D4a*5XwD)P--sPkoY(a~WBw;E~AW`Yue4kFa^LM3X`8x|}ZUeMnqr}>kH zG%WWW>3ml$Yez?i%)2pbKPI7?5o?hydokgQyZsNEr{a|mLdt;X2TX(#B1j35xPnPW z*bMSSOauW>o;*=kO8ojw91VX!qoOQb)zHJ!odWB}d+*K?#sY_jqPdg{Sm2HdYzdEx zOGVPhVRTGPtv0o}RfVP;Nd(|CB)I;*t&QO8h zFfekr30S!-LHmV_Su-W+rEwYXJ^;6&3|L$mMC8*bQptyOo9;>Qb9Q9`ySe3%V$A*9 zeKEe+b0{#KWGp$F+tga)0RtI)nhMa-K@JS}2krK~n8vJ=Ngm?R!9G<~RyuU0d?nz# z-5EK$o(!F?hmX*2Yt6+coY`6jGbb7tF#6nHA zuKk=GGJ;ZwON1iAfG$E#Y7MnZVmrY|j0eVI(DN_MNFJmyZ|;w4tf@=CCDZ#5N_0K= z$;R~bbk?}TpfDjfB&aiQ$VA}s?P}xPERJG{kxk5~R`iRS(SK5d+Xs9swCozZISbnS zk!)I0>t=A<-^z(cmSFz3=jZ23u13X><0b)P)^1T_))Kr`e!-pb#q&J*Q`p+B6la%C zuVl&0duN<;uOsB3%T9Fp8t{ED108<+W(nOZd?gDnfNBC3>M8WE61$So|P zVvqH0SNtDTcsUdzaMDpT=Ty0pDHHNL@Z0w$Y`XO z2M-_r1S+GaH%pz#Uy0*w$Vdl=X=rQXEzO}d6J^R6zjM1u&c9vYLvLp?W7w(?np9x1 zE_0JSAJCPB%i7p*Wvg)pn5T`8k3-uR?*NT|J`eS#_#54p>!p(mLDvmc-3o0mX*mp_ zN*AeS<>#^-{S%W<*mz^!X$w_2dHWpcJ6^j64qFBft-o}o_Vx80o0>}Du;>kLts;$8 zC`7q$QI(dKYG`Wa8#wl@V4jVWBRGQ@1dr-hstpQL)Tl+aqVpGpbSfN>5i&QMXfiZ> zaA?T1VGe?rpQ@;+pkrVdd{klI&jVS@I5_iz!=UMpTsa~mBga?1r}aRBm1WS;TT*s0f0lY=JBl66Upy)-k4J}lh=P^8(SXk~0xW=T9v*B|gzIhN z>qsO7dFd~mgxAy4V?&)=5ieYq?zi?ZEoj)&2o)RLy=@hbCRcfT5jigwtQGE{L*8<@Yd{zg;CsL5mvzfDY}P-wos_6PfprFVaeqNE%h zKZhLtcQld;ZD+>=nqN~>GvROfueSzJD&BE*}XfU|H&(FssBqY=hPCt`d zH?@s2>I(|;fcW&YM6#V#!kUIP8$Nkdh0A(bEVj``-AAyYgwY~jB zT|I7Bf@%;7aL7Wf4dZ%VqF$eiaC38OV6oy3Z#TER2G+fOCd9Iaoy6aLYbPTN{XRPz z;U!V|vBf%H!}52L2gH_+j;`bTcQRXB+y9onc^wLm5wi3-Be}U>k_u>2Eg$=k!(l@I zcCg+flakT2Nej3i0yn+g+}%NYb?ta;R?(g5SnwsQ49U8Wng8d|{B+lyRcEDvR3+`O{zfmrmvFrL6acVP%yG98X zo&+VBg@px@i)%o?dG(`T;n*$S5*rnyiR#=wW}}GsAcfyQpE|>a{=$Hjg=-*_K;UtD z#z-)AXwSRY?OPefw^iI+ z)AXz#PfEjlwTes|_{sB?4(O@fg0AJ^g8gP}ex9Ucf*@_^J(s_5jJV}c)s$`Myn|Kd z$6>}#q^n{4vN@+Os$m7KV+`}c%4)4pv@06af4-x5#wj!KKb%caK{A&Y#Rfs z-po?Dcb1({W=6FKIUirH&(yg=*6aLCekcKwyfK^JN5{wcA3nhO(o}SK#!CINhI`-I z1)6&n7O&ZmyFMuNwvEic#IiOAwNkR=u5it{B9n2sAJV5pNhar=j5`*N!Na;c7g!l$ z3aYBqUkqqTJ=Re-;)s!EOeij=7SQZ3Hq}ZRds%IM*PtM$wV z@;rlc*NRK7i3y5BETSKuumEN`Xu_8GP1Ri=OKQ$@I^ko8>H6)4rjiG5{VBM>B|%`&&s^)jS|-_95&yc=GqjNo{zFkw%%HHhS~e=s zD#sfS+-?*t|J!+ozP6KvtOl!R)@@-z24}`9{QaVLD^9VCSR2b`b!KC#o;Ki<+wXB6 zx3&O0LOWcg4&rv4QG0)4yb}7BFSEg~=IR5#ZRj8kg}dS7_V&^%#Do==#`u zpy6{ox?jWuR(;pg+f@mT>#HGWHAJRRDDDv~@(IDw&R>9643kK#HN`!1vBJHnC+RM&yIh8{gG2q zA%e*U3|N0XSRa~oX-3EAneep)@{h2vvd3Xvy$7og(sayr@95+e6~Xvi1tUqnIxoIH zVWo*OwYElb#uyW{Imam6f2rGbjR!Y3`#gPqkv57dB6K^wRGxc9B(t|aYDGS=m$&S!NmCtrMMaUg(c zc2qC=2Z`EEFMW-me5B)24AqF*bV5Dr-M5ig(l-WPS%CgaPzs6p_gnCIvTJ=Y<6!gT zVt@AfYCzjjsMEGi=rDQHo0yc;HqoRNnNFeWZgcm?f;cp(6CNylj36DoL(?TS7eU#+ z7&mfr#y))+CJOXQKUMZ7QIdS9@#-}7y2K1{8)cCt0~-X0O!O?Qx#E4Og+;A2SjalQ zs7r?qn0H044=sDN$SRG$arw~n=+T_DNdSrarmu)V6@|?1-ZB#hRn`uilTGPJ@fqEy zGt(f0B+^JDP&f=r{#Y_wi#AVDf-y!RIXU^0jXsFpf>=Ji*TeqSY!H~AMbJdCGLhC) zn7Rx+sXw6uYj;WRYrLd^5IZq@6JI1C^YkgnedZEYy<&4(z%Q$5yv#Boo{AH8n$a zhb4Y3PWdr269&?V%uI$xMcUrMzl=;w<_nm*qr=c3Rl@i5wWB;e-`t7D&c-mcQl7x! zZWB`UGcw=Y2=}~wzrfLx=uet<;m3~=8I~ZRuzvMQUQdr+yTV|ATf1Uuomr__nDf=X zZ3WYJtHp_ri(}SQAPjv+Y+0=fH4krOP@S&=zZ-t1jW1o@}z;xk8 z(Nz1co&El^HK^NrhVHa-_;&88vTU>_J33=%{if;BEY*J#1n59=07jrGQ#IP>@u#3A z;!q+E1Rj3ZJ+!4bq9F8PXJ@yMgZL;>&gYA0%_Kbi8?S=XGM~dnQZQ!yBSgcZhY96H zrWnU;k)qy`rX&&xlDyA%(a1Hhi5CWkmg(`Gb%m(HKi-7Z!LKGRP_B8@`7&hdDy5n= z`OIxqxiVfX@OX1p(mQu>0Ai*v_cTMiw4qRt3~NBvr9oBy0)r>w3p~V0SCm=An6@3n)>@z!|o-$HvDK z|3D2ZMJkLE5loMKl6R^ez@Zz%S$&mbeoqH5`Bb){Ei21q&VP)hWS2tjShfFtGE+$z zzCR$P#uktu+#!w)cX!lWN1XU%K-r=s{|j?)Akf@q#3b#{6cZCuJ~gCxuMXRmI$nGtnH+-h z+GEi!*X=AP<|fG`1>MBdTb?28JYc=fGvAi2I<$B(rs$;eoJCyR6_bc~p!XR@O-+sD z=eH`-ye})I5ic1eL~TDmtfJ|8`0VJ*Yr=hNCd)G1p2MMz4C3^Mj?7;!w|Ly%JqmuW zlIEW^Ft%z?*|fpXda>Jr^1noFZEwFgVV%|*XhH@acv8rdGxeEX{M$(vG{Zw+x(ei@ zmfXb22}8-?Fi`vo-YVrTH*C?a8%M=Hv9MqVH7H^J$KsD?>!SFZ;ZsvnHr_gn=7acz z#W?0eCdVhVMWN12VV^$>WlQ?f;P^{(&pYTops|btm6aj>_Uz+hqpGwB)vWp0Cf5y< zft8-je~nn?W11plq}N)4A{l8I7$!ks_x$PXW-2XaRFswX_BnF{R#6YIwMhAgd5F9X zGmwdadS6(a^fjHtXg8=l?Rc0Sm%hk6E9!5cLVloEy4eh(=FwgP`)~I^5~pBEWo+F6 zSf2ncyMurJN91#cJTy_u8Y}@%!bq1RkGC~-bV@SXRd4F{R-*V`bS+6;W5vZ(&+I<9$;-V|eNfLa5n-6% z2(}&uGRF;p92eS*sE*oR$@pexaqr*meB)VhmIg@h{uzkk$9~qh#cHhw#>O%)b@+(| z^IQgqzuj~Sk(J;swEM-3TrJAPCq9k^^^`q{IItKBRXYe}e0Tdr=Huf7da3$l4PdpwWDop%^}n;dD#K4s#DYA8SHZ z&1!riV4W4R7R#C))JH1~axJ)RYnM$$lIR%6fIVA@zV{XVyx}C+a-Dt8Y9M)^KU0+H zR4IUb2CJ{Hg>CuaXtD50jB(_Tcx=Z$^WYu2u5kubqmwp%drJ6 z?Fo40g!Qd<-l=TQxqHEOuPX0;^z7iX?Ke^a%XT<13TA^5`4Xcw6D@Ur&VT&CUe0d} z1GjOVF1^L@>O)l@?bD~$wzgf(nxX1OGD8fEV?TdJcZc2KoUe|oP1#=$$7ee|xbY)A zDZq+cuTpc(fFdj^=!;{k03C69lMQ(|>uhRfRu%+!k&YOi-3|1QKB z z?n?eq1XP>p-IM$Z^C;2L3itnbJZAip*Zo0aw2bs8@(s^~*8T9go!%dHcAz2lM;`yp zD=7&xjFV$S&5uDaiScyD?B-i1ze`+CoRtz`Wn+Zl&#s4&}MO{@N!ufrzjG$B79)Y2d3tBk&)TxUTw@QS0TEL_?njX|@vq?Uz(nBFK5Pq7*xj#u*R&i|?7+6# z+|r_n#SW&LXhtheZdah{ZVoqwyT{D>MC3nkFF#N)xLi{p7J1jXlmVeb;cP5?e(=f# zuT7fvjSbjS781v?7{)-X3*?>tq?)Yd)~|1{BDS(pqC zC}~H#WXlkUW*H5CDOo<)#x7%RY)A;ShGhI5s*#cRDA8YgqG(HeKDx+#(ZQ?386dv! zlXCO)w91~Vw4AmOcATuV653fa9R$fyK8ul%rG z-wfS zihugoZyr38Im?Zuh6@RcF~t1anQu7>#lPpb#}4cOA!EM11`%f*07RqOVkmX{p~KJ9 z^zP;K#|)$`^Rb{rnHGH{~>1(fawV0*Z#)}M`m8-?ZJV<+e}s9wE# z)l&az?w^5{)`S(%MRzxdNqrs1n*-=jS^_jqE*5XDrA0+VE`5^*p3CuM<&dZEeCjoz zR;uu_H9ZPZV|fQq`Cyw4nscrVwi!fE6ciMmX$!_hN7uF;jjKG)d2@aC4ropY)8etW=xJvni)8eHi`H$%#zn^WJ5NLc-rqk|u&&4Z6fD_m&JfSI1Bvb?b<*n&sfl0^t z=HnmRl`XrFvMKB%9}>PaA`m-fK6a0(8=qPkWS5bb4=v?XcWi&hRY?O5HdulRi4?fN zlsJ*N-0Qw+Yic@s0(2uy%F@ib;GjXt01Fmx5XbRo6+n|pP(&nodMoap^z{~q ziEeaUT@Mxe3vJSfI6?uLND(CNr=#^W<1b}jzW58bIfyWTDle$mmS(|x-0|2UlX+9k zQ^EX7Nw}?EzVoBfT(-LT|=9N@^hcn-_p&sqG z&*oVs2JSU+N4ZD`FhCAWaS;>|wH2G*Id|?pa#@>tyxX`+4HyIArWDvVrX)2WAOQff z0qyHu&-S@i^MS-+j--!pr4fPBj~_8({~e1bfcl0wI1kaoN>mJL6KUPQm5N7lB(ui1 zE-o%kq)&djzWJ}ob<-GfDlkB;F31j-VHKvQUGQ3sp`CwyGJk_i!y^sD0fqC@$9|jO zOqN!r!8-p==F@ZVP=U$qSpY(gQ0)59P1&t@y?5rvg<}E+GB}26NYPp4f2YFQrQtot5mn3wu_qprZ=>Ig-$ zbW26Ws~IgY>}^5w`vTB(G`PTZaDiGBo5o(tp)qli|NeV( z@H_=R8V39rt5J5YB2Ky?4eJJ#b`_iBe2ot~6%7mLt5t8Vwi^Jy7|jWXqa3amOIoRb zOr}WVFP--DsS`1WpN%~)t3R!arKF^Q$e12KEqU36AWwnCBICpH4XCsfnyrHr>$I$4 z!DpKX$OKLWarN7nv@!uIA+~RNO)l$$w}p(;b>mx8pwYvu;dD_unryX_NhT8*Tj>BTrTTL&!?O+%Rv;b?B??gSzdp?6Uug9{ zd@V08Z$BdI?fpoCS$)t4mg4rT8Q_I}h`0d-vYZ^|dOB*Q^S|xqTV*vIg?@fVFSmMpaw0qtTRbx} z({Pg?#{2`sc9)M5N$*N|4;^t$+QP?#mov zGVC@I*lBVrOU-%2y!7%)fAKjpEFsgQc4{amtiHb95KQEwvf<(3T<9-Zm$xIew#P22 zc2Ix|App^>v6(3L_MCU0d3W##AB0M~3D00EWoKZqsJYT(#@w$Y_H7G22M~ApVFTRHMI_3be)Lkn#0F*V8Pq zc}`Cjy$bE;FJ6H7p=0y#R>`}-m4(0F>%@P|?7fx{=R^uFdISRnZ2W_xQhD{YuR3t< z{6yxu=4~JkeA;|(J6_nv#>Nvs&FuLA&PW^he@t(UwFFE8)|a!R{`E`K`i^ZnyE4$k z;(749Ix|oi$c3QbEJ3b~D_kQsPz~fIUKym($a_7dJ?o+40*OLl^{=&oq$<#Q(yyrp z{J-FAniyAw9tPbe&IhQ|a`DqFTVQGQ&Gq3!C2==4x{6EJwiPZ8zub-iXoUtkJiG{} zPaR&}_fn8_z~(=;5lD-aPWD3z8PZS@AaUiomF!G8I}Mf>e~0g#BelA-5#`cj;O5>N Xviia!U7SGha1wx#SCgwmn*{w2TRX*I literal 0 HcmV?d00001 diff --git a/bus_alt_connection/static/description/index.html b/bus_alt_connection/static/description/index.html new file mode 100644 index 000000000..14b5312dc --- /dev/null +++ b/bus_alt_connection/static/description/index.html @@ -0,0 +1,498 @@ + + + + + + +Bus Alt Connection + + + +
+

Bus Alt Connection

+ + +

Beta License: AGPL-3 OCA/server-tools Translate me on Weblate Try me on Runbot

+

This module makes it possible to use PgBouncer as a connection pooler +for odoo.

+
+

Why isn’t odoo’s connection pooling good enough?

+

Odoo’s builtin connection pooling works at process level: each Odoo process +has its own ConnectionPool, limited to db_maxconn.

+

It does the job of re-using open connections available in the pool. +But it never closes these connections, unless reaching db_maxconn.

+

In practice, we observe that each odoo worker will end up +with up to 3 open connection in its pool. +With 10 http workers, that’s up to 30 connection continuously open just +for one single instance.

+
+
+

Here comes PgBouncer

+

PgBouncer will help to limit this number of open connections, +by sharing a pool of connections at the instance level, between +all workers. Odoo workers will still have up to 3 open connections, +but these will be connections to PgBouncer, that on its side will +close unnecessary connections to pg.

+

This has proven to help performances on Odoo deployments with +multiple instances.

+

It allows you to define how resources should be shared, +according to your priorities, e.g. :

+
    +
  • key odoo instance on host A can open up to 30 connections
    • +
  • while odoo instance on host B, dedicated to reports, +can open up to 10 connections only
    • +
+

And most importantly, it helps you to ensure that +max_connections will never be reached on pg server side.

+
+
+

Why is this module needed?

+

When configuring PgBouncer, you can choose between 2 transaction pooling modes:

+
    +
  • pool_mode = session
    • +
  • pool_mode = transaction
    • +
+

If we choose pool_mode = session, then one server connection will be tied +to a given odoo process until its death, which is exactly what we’re trying +to change. Thus, to release the server connection once the transaction is +complete, we use pool_mode = transaction.

+

This works fine, except for Odoo’s longpolling features that relies +on the LISTEN/NOTIFY mechanism from pg, which is not compatible with that +mode.

+

To be more precise, NOTIFY statements are properly transfered by PgBouncer +in that mode; only the LISTEN statement isn’t (because it needs to keep the +server connection open).

+

So for the unique “listening” connection per instance that requires this +statement (here), we need odoo to connect directly to the pg server, bypassing +PgBouncer.

+

That’s what this module implements, by overriding the relevant method +of the Dispatcher.

+

Table of contents

+
+
+

Installation

+

You don’t need to install this module in the database(s) to enable it.

+

But you need to load it server-wide:

+
    +
  • By starting Odoo with --load=web,bus_alt_connection
    • +
  • Or by updating its configuration file:
    • +
+
+[options]
+(...)
+server_wide_modules = web,bus_alt_connection
+
+
+
+

Configuration

+

You need to define how to connect directly to the database:

+
    +

  • Either by defining environment variables:

    +
    +
      +
    • IMDISPATCHER_DB_HOST=db-01
      • +
    • IMDISPATCHER_DB_PORT=5432
      • +
    +
    +
    • +

  • Or in Odoo’s configuration file:

    +
    • +
+
+[options]
+(...)
+imdispatcher_db_host = db-01
+imdispatcher_db_port = 5432
+
+
+
+

Bug Tracker

+

Bugs are tracked on GitHub Issues. +In case of trouble, please check there if your issue has already been reported. +If you spotted it first, help us smashing it by providing a detailed and welcomed +feedback.

+

Do not contact contributors directly about support or help with technical issues.

+
+
+

Credits

+
+

Authors

+
    +
  • Trobz
    • +
+
+
+

Contributors

+ +
+
+

Maintainers

+

This module is maintained by the OCA.

+Odoo Community Association +

OCA, or the Odoo Community Association, is a nonprofit organization whose +mission is to support the collaborative development of Odoo features and +promote its widespread use.

+

This module is part of the OCA/server-tools project on GitHub.

+

You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute.

+
+
+
+ +