Browse Source

[ADD] bus_alt_connection

12.0-mig-module_prototyper_last
Nils Hamerlinck 5 years ago
parent
commit
e6e75bd814
  1. 187
      bus_alt_connection/README.rst
  2. 3
      bus_alt_connection/__init__.py
  3. 15
      bus_alt_connection/__manifest__.py
  4. 3
      bus_alt_connection/models/__init__.py
  5. 77
      bus_alt_connection/models/bus.py
  6. 15
      bus_alt_connection/readme/CONFIGURE.rst
  7. 1
      bus_alt_connection/readme/CONTRIBUTORS.rst
  8. 79
      bus_alt_connection/readme/DESCRIPTION.rst
  9. 13
      bus_alt_connection/readme/INSTALL.rst
  10. BIN
      bus_alt_connection/static/description/icon.png
  11. 498
      bus_alt_connection/static/description/index.html

187
bus_alt_connection/README.rst

@ -0,0 +1,187 @@
==================
Bus Alt Connection
==================
.. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!! This file is generated by oca-gen-addon-readme !!
!! changes will be overwritten. !!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
.. |badge1| image:: https://img.shields.io/badge/maturity-Beta-yellow.png
:target: https://odoo-community.org/page/development-status
:alt: Beta
.. |badge2| image:: https://img.shields.io/badge/licence-AGPL--3-blue.png
:target: http://www.gnu.org/licenses/agpl-3.0-standalone.html
:alt: License: AGPL-3
.. |badge3| image:: https://img.shields.io/badge/github-OCA%2Fserver--tools-lightgray.png?logo=github
:target: https://github.com/OCA/server-tools/tree/12.0/bus_alt_connection
:alt: OCA/server-tools
.. |badge4| image:: https://img.shields.io/badge/weblate-Translate%20me-F47D42.png
:target: https://translation.odoo-community.org/projects/server-tools-12-0/server-tools-12-0-bus_alt_connection
:alt: Translate me on Weblate
.. |badge5| image:: https://img.shields.io/badge/runbot-Try%20me-875A7B.png
:target: https://runbot.odoo-community.org/runbot/149/12.0
:alt: Try me on Runbot
|badge1| |badge2| |badge3| |badge4| |badge5|
This module makes it possible to use PgBouncer_ as a connection pooler
for odoo.
.. _PgBouncer: https://pgbouncer.github.io/
Why isn't odoo's connection pooling good enough?
================================================
Odoo's builtin connection pooling works at process level: each Odoo process
has its own ConnectionPool_, limited to ``db_maxconn``.
It does the job of re-using open connections available in the pool.
But it never closes these connections, `unless reaching db_maxconn`_.
.. _ConnectionPool: https://github.com/odoo/odoo/blob/12.0/odoo/sql_db.py#L525
.. _`unless reaching db_maxconn`: https://github.com/odoo/odoo/blob/12.0/odoo/sql_db.py#L593
In practice, we observe that each odoo worker will end up
with up to 3 open connection in its pool.
With 10 http workers, that's up to 30 connection continuously open just
for one single instance.
Here comes PgBouncer
====================
PgBouncer will help to limit this number of open connections,
by sharing a pool of connections at the instance level, between
all workers. Odoo workers will still have up to 3 open connections,
but these will be connections to PgBouncer, that on its side will
close unnecessary connections to pg.
This has proven to help performances on Odoo deployments with
multiple instances.
It allows you to define how resources should be shared,
according to your priorities, e.g. :
* key odoo instance on host A can open up to 30 connections
* while odoo instance on host B, dedicated to reports,
can open up to 10 connections only
And most importantly, it helps you to ensure that
``max_connections`` will never be reached on pg server side.
Why is this module needed?
==========================
When configuring PgBouncer, you can choose between 2 transaction pooling modes:
* `pool_mode = session`
* `pool_mode = transaction`
If we choose `pool_mode = session`, then one server connection will be tied
to a given odoo process until its death, which is exactly what we're trying
to change. Thus, to release the server connection once the transaction is
complete, we use `pool_mode = transaction`.
This works fine, except for Odoo's longpolling features that relies
on the `LISTEN/NOTIFY`_ mechanism from pg, which is `not compatible`_ with that
mode.
.. _`LISTEN/NOTIFY`: https://www.postgresql.org/docs/9.6/static/sql-notify.html
.. _`not compatible`: https://wiki.postgresql.org/wiki/PgBouncer
To be more precise, `NOTIFY` statements are properly transfered by PgBouncer
in that mode; only the `LISTEN` statement isn't (because it needs to keep the
server connection open).
So for the unique "listening" connection per instance that requires this
statement (here_), we need odoo to connect directly to the pg server, bypassing
PgBouncer.
That's what this module implements, by overriding the relevant method
of the Dispatcher_.
.. _here: https://github.com/odoo/odoo/blob/12.0/addons/bus/models/bus.py#L166
.. _Dispatcher: https://github.com/odoo/odoo/blob/12.0/addons/bus/models/bus.py#L105
**Table of contents**
.. contents::
:local:
Installation
============
You don't need to install this module in the database(s) to enable it.
But you need to load it server-wide:
* By starting Odoo with ``--load=web,bus_alt_connection``
* Or by updating its configuration file:
.. code-block:: ini
[options]
(...)
server_wide_modules = web,bus_alt_connection
Configuration
=============
You need to define how to connect directly to the database:
* Either by defining environment variables:
- ``IMDISPATCHER_DB_HOST=db-01``
- ``IMDISPATCHER_DB_PORT=5432``
* Or in Odoo's configuration file:
.. code-block:: ini
[options]
(...)
imdispatcher_db_host = db-01
imdispatcher_db_port = 5432
Bug Tracker
===========
Bugs are tracked on `GitHub Issues <https://github.com/OCA/server-tools/issues>`_.
In case of trouble, please check there if your issue has already been reported.
If you spotted it first, help us smashing it by providing a detailed and welcomed
`feedback <https://github.com/OCA/server-tools/issues/new?body=module:%20bus_alt_connection%0Aversion:%2012.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**>`_.
Do not contact contributors directly about support or help with technical issues.
Credits
=======
Authors
~~~~~~~
* Trobz
Contributors
~~~~~~~~~~~~
* Nils Hamerlinck <nils@trobz.com>
Maintainers
~~~~~~~~~~~
This module is maintained by the OCA.
.. image:: https://odoo-community.org/logo.png
:alt: Odoo Community Association
:target: https://odoo-community.org
OCA, or the Odoo Community Association, is a nonprofit organization whose
mission is to support the collaborative development of Odoo features and
promote its widespread use.
This module is part of the `OCA/server-tools <https://github.com/OCA/server-tools/tree/12.0/bus_alt_connection>`_ project on GitHub.
You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute.

3
bus_alt_connection/__init__.py

@ -0,0 +1,3 @@
# Copyright 2019 Trobz <https://trobz.com>
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl).
from . import models

15
bus_alt_connection/__manifest__.py

@ -0,0 +1,15 @@
# Copyright 2019 Trobz <https://trobz.com>
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl).
{
'name': 'Bus Alt Connection',
'summary': 'Needed when using PgBouncer as a connection pooler',
'version': '12.0.1.0.0',
'author': 'Trobz,Odoo Community Association (OCA)',
'website': 'https://github.com/OCA/server-tools',
'category': 'Extra Tools',
'license': 'AGPL-3',
'depends': ['bus'],
'installable': True,
'auto_install': False,
}

3
bus_alt_connection/models/__init__.py

@ -0,0 +1,3 @@
# Copyright 2019 Trobz <https://trobz.com>
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl).
from . import bus

77
bus_alt_connection/models/bus.py

@ -0,0 +1,77 @@
# Copyright 2019 Trobz <https://trobz.com>
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl).
import os
import json
import logging
import select
import psycopg2
from psycopg2.extensions import ISOLATION_LEVEL_AUTOCOMMIT
import odoo
from odoo.tools import config
from odoo.addons.bus.models.bus import hashable, TIMEOUT
import odoo.addons.bus.models.bus
_logger = logging.getLogger(__name__)
def _connection_info_for(db_name):
db_or_uri, connection_info = odoo.sql_db.connection_info_for(db_name)
for p in ('host', 'port'):
cfg = (os.environ.get('ODOO_IMDISPATCHER_DB_%s' % p.upper()) or
config.get('imdispatcher_db_' + p))
if cfg:
connection_info[p] = cfg
return connection_info
class ImDispatch(odoo.addons.bus.models.bus.ImDispatch):
def loop(self):
""" Dispatch postgres notifications to the relevant
polling threads/greenlets """
connection_info = _connection_info_for('postgres')
_logger.info("Bus.loop listen imbus on db postgres "
"(via %(host)s:%(port)s)",
connection_info)
conn = psycopg2.connect(**connection_info)
conn.set_isolation_level(ISOLATION_LEVEL_AUTOCOMMIT)
with conn.cursor() as cr:
cr.execute("listen imbus")
conn.commit()
while True:
if select.select([conn], [], [], TIMEOUT) == ([], [], []):
pass
else:
conn.poll()
channels = []
while conn.notifies:
channels.extend(json.loads(conn
.notifies.pop().payload))
# dispatch to local threads/greenlets
events = set()
for channel in channels:
events.update(self.channels.pop(hashable(channel),
set()))
for event in events:
event.set()
odoo.addons.bus.models.bus.ImDispatch = ImDispatch
# we can replace the existing dispatcher because its thread
# has not been started yet; indeed, since a2ed3d it only starts
# on first /poll request:
# https://github.com/odoo/odoo/commit/a2ed3d3d5bdb6025a1ba14ad557a115a86413e65
if not odoo.multi_process or odoo.evented:
dispatch = ImDispatch()
odoo.addons.bus.models.bus.dispatch = dispatch
odoo.addons.bus.controllers.main.dispatch = dispatch

15
bus_alt_connection/readme/CONFIGURE.rst

@ -0,0 +1,15 @@
You need to define how to connect directly to the database:
* Either by defining environment variables:
- ``IMDISPATCHER_DB_HOST=db-01``
- ``IMDISPATCHER_DB_PORT=5432``
* Or in Odoo's configuration file:
.. code-block:: ini
[options]
(...)
imdispatcher_db_host = db-01
imdispatcher_db_port = 5432

1
bus_alt_connection/readme/CONTRIBUTORS.rst

@ -0,0 +1 @@
* Nils Hamerlinck <nils@trobz.com>

79
bus_alt_connection/readme/DESCRIPTION.rst

@ -0,0 +1,79 @@
This module makes it possible to use PgBouncer_ as a connection pooler
for odoo.
.. _PgBouncer: https://pgbouncer.github.io/
Why isn't odoo's connection pooling good enough?
================================================
Odoo's builtin connection pooling works at process level: each Odoo process
has its own ConnectionPool_, limited to ``db_maxconn``.
It does the job of re-using open connections available in the pool.
But it never closes these connections, `unless reaching db_maxconn`_.
.. _ConnectionPool: https://github.com/odoo/odoo/blob/12.0/odoo/sql_db.py#L525
.. _`unless reaching db_maxconn`: https://github.com/odoo/odoo/blob/12.0/odoo/sql_db.py#L593
In practice, we observe that each odoo worker will end up
with up to 3 open connection in its pool.
With 10 http workers, that's up to 30 connection continuously open just
for one single instance.
Here comes PgBouncer
====================
PgBouncer will help to limit this number of open connections,
by sharing a pool of connections at the instance level, between
all workers. Odoo workers will still have up to 3 open connections,
but these will be connections to PgBouncer, that on its side will
close unnecessary connections to pg.
This has proven to help performances on Odoo deployments with
multiple instances.
It allows you to define how resources should be shared,
according to your priorities, e.g. :
* key odoo instance on host A can open up to 30 connections
* while odoo instance on host B, dedicated to reports,
can open up to 10 connections only
And most importantly, it helps you to ensure that
``max_connections`` will never be reached on pg server side.
Why is this module needed?
==========================
When configuring PgBouncer, you can choose between 2 transaction pooling modes:
* `pool_mode = session`
* `pool_mode = transaction`
If we choose `pool_mode = session`, then one server connection will be tied
to a given odoo process until its death, which is exactly what we're trying
to change. Thus, to release the server connection once the transaction is
complete, we use `pool_mode = transaction`.
This works fine, except for Odoo's longpolling features that relies
on the `LISTEN/NOTIFY`_ mechanism from pg, which is `not compatible`_ with that
mode.
.. _`LISTEN/NOTIFY`: https://www.postgresql.org/docs/9.6/static/sql-notify.html
.. _`not compatible`: https://wiki.postgresql.org/wiki/PgBouncer
To be more precise, `NOTIFY` statements are properly transfered by PgBouncer
in that mode; only the `LISTEN` statement isn't (because it needs to keep the
server connection open).
So for the unique "listening" connection per instance that requires this
statement (here_), we need odoo to connect directly to the pg server, bypassing
PgBouncer.
That's what this module implements, by overriding the relevant method
of the Dispatcher_.
.. _here: https://github.com/odoo/odoo/blob/12.0/addons/bus/models/bus.py#L166
.. _Dispatcher: https://github.com/odoo/odoo/blob/12.0/addons/bus/models/bus.py#L105

13
bus_alt_connection/readme/INSTALL.rst

@ -0,0 +1,13 @@
You don't need to install this module in the database(s) to enable it.
But you need to load it server-wide:
* By starting Odoo with ``--load=web,bus_alt_connection``
* Or by updating its configuration file:
.. code-block:: ini
[options]
(...)
server_wide_modules = web,bus_alt_connection

BIN
bus_alt_connection/static/description/icon.png

After

Width: 128  |  Height: 128  |  Size: 9.2 KiB

498
bus_alt_connection/static/description/index.html

@ -0,0 +1,498 @@
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.15.1: http://docutils.sourceforge.net/" />
<title>Bus Alt Connection</title>
<style type="text/css">
/*
:Author: David Goodger (goodger@python.org)
:Id: $Id: html4css1.css 7952 2016-07-26 18:15:59Z milde $
:Copyright: This stylesheet has been placed in the public domain.
Default cascading style sheet for the HTML output of Docutils.
See http://docutils.sf.net/docs/howto/html-stylesheets.html for how to
customize this style sheet.
*/
/* used to remove borders from tables and images */
.borderless, table.borderless td, table.borderless th {
border: 0 }
table.borderless td, table.borderless th {
/* Override padding for "table.docutils td" with "! important".
The right padding separates the table cells. */
padding: 0 0.5em 0 0 ! important }
.first {
/* Override more specific margin styles with "! important". */
margin-top: 0 ! important }
.last, .with-subtitle {
margin-bottom: 0 ! important }
.hidden {
display: none }
.subscript {
vertical-align: sub;
font-size: smaller }
.superscript {
vertical-align: super;
font-size: smaller }
a.toc-backref {
text-decoration: none ;
color: black }
blockquote.epigraph {
margin: 2em 5em ; }
dl.docutils dd {
margin-bottom: 0.5em }
object[type="image/svg+xml"], object[type="application/x-shockwave-flash"] {
overflow: hidden;
}
/* Uncomment (and remove this text!) to get bold-faced definition list terms
dl.docutils dt {
font-weight: bold }
*/
div.abstract {
margin: 2em 5em }
div.abstract p.topic-title {
font-weight: bold ;
text-align: center }
div.admonition, div.attention, div.caution, div.danger, div.error,
div.hint, div.important, div.note, div.tip, div.warning {
margin: 2em ;
border: medium outset ;
padding: 1em }
div.admonition p.admonition-title, div.hint p.admonition-title,
div.important p.admonition-title, div.note p.admonition-title,
div.tip p.admonition-title {
font-weight: bold ;
font-family: sans-serif }
div.attention p.admonition-title, div.caution p.admonition-title,
div.danger p.admonition-title, div.error p.admonition-title,
div.warning p.admonition-title, .code .error {
color: red ;
font-weight: bold ;
font-family: sans-serif }
/* Uncomment (and remove this text!) to get reduced vertical space in
compound paragraphs.
div.compound .compound-first, div.compound .compound-middle {
margin-bottom: 0.5em }
div.compound .compound-last, div.compound .compound-middle {
margin-top: 0.5em }
*/
div.dedication {
margin: 2em 5em ;
text-align: center ;
font-style: italic }
div.dedication p.topic-title {
font-weight: bold ;
font-style: normal }
div.figure {
margin-left: 2em ;
margin-right: 2em }
div.footer, div.header {
clear: both;
font-size: smaller }
div.line-block {
display: block ;
margin-top: 1em ;
margin-bottom: 1em }
div.line-block div.line-block {
margin-top: 0 ;
margin-bottom: 0 ;
margin-left: 1.5em }
div.sidebar {
margin: 0 0 0.5em 1em ;
border: medium outset ;
padding: 1em ;
background-color: #ffffee ;
width: 40% ;
float: right ;
clear: right }
div.sidebar p.rubric {
font-family: sans-serif ;
font-size: medium }
div.system-messages {
margin: 5em }
div.system-messages h1 {
color: red }
div.system-message {
border: medium outset ;
padding: 1em }
div.system-message p.system-message-title {
color: red ;
font-weight: bold }
div.topic {
margin: 2em }
h1.section-subtitle, h2.section-subtitle, h3.section-subtitle,
h4.section-subtitle, h5.section-subtitle, h6.section-subtitle {
margin-top: 0.4em }
h1.title {
text-align: center }
h2.subtitle {
text-align: center }
hr.docutils {
width: 75% }
img.align-left, .figure.align-left, object.align-left, table.align-left {
clear: left ;
float: left ;
margin-right: 1em }
img.align-right, .figure.align-right, object.align-right, table.align-right {
clear: right ;
float: right ;
margin-left: 1em }
img.align-center, .figure.align-center, object.align-center {
display: block;
margin-left: auto;
margin-right: auto;
}
table.align-center {
margin-left: auto;
margin-right: auto;
}
.align-left {
text-align: left }
.align-center {
clear: both ;
text-align: center }
.align-right {
text-align: right }
/* reset inner alignment in figures */
div.align-right {
text-align: inherit }
/* div.align-center * { */
/* text-align: left } */
.align-top {
vertical-align: top }
.align-middle {
vertical-align: middle }
.align-bottom {
vertical-align: bottom }
ol.simple, ul.simple {
margin-bottom: 1em }
ol.arabic {
list-style: decimal }
ol.loweralpha {
list-style: lower-alpha }
ol.upperalpha {
list-style: upper-alpha }
ol.lowerroman {
list-style: lower-roman }
ol.upperroman {
list-style: upper-roman }
p.attribution {
text-align: right ;
margin-left: 50% }
p.caption {
font-style: italic }
p.credits {
font-style: italic ;
font-size: smaller }
p.label {
white-space: nowrap }
p.rubric {
font-weight: bold ;
font-size: larger ;
color: maroon ;
text-align: center }
p.sidebar-title {
font-family: sans-serif ;
font-weight: bold ;
font-size: larger }
p.sidebar-subtitle {
font-family: sans-serif ;
font-weight: bold }
p.topic-title {
font-weight: bold }
pre.address {
margin-bottom: 0 ;
margin-top: 0 ;
font: inherit }
pre.literal-block, pre.doctest-block, pre.math, pre.code {
margin-left: 2em ;
margin-right: 2em }
pre.code .ln { color: grey; } /* line numbers */
pre.code, code { background-color: #eeeeee }
pre.code .comment, code .comment { color: #5C6576 }
pre.code .keyword, code .keyword { color: #3B0D06; font-weight: bold }
pre.code .literal.string, code .literal.string { color: #0C5404 }
pre.code .name.builtin, code .name.builtin { color: #352B84 }
pre.code .deleted, code .deleted { background-color: #DEB0A1}
pre.code .inserted, code .inserted { background-color: #A3D289}
span.classifier {
font-family: sans-serif ;
font-style: oblique }
span.classifier-delimiter {
font-family: sans-serif ;
font-weight: bold }
span.interpreted {
font-family: sans-serif }
span.option {
white-space: nowrap }
span.pre {
white-space: pre }
span.problematic {
color: red }
span.section-subtitle {
/* font-size relative to parent (h1..h6 element) */
font-size: 80% }
table.citation {
border-left: solid 1px gray;
margin-left: 1px }
table.docinfo {
margin: 2em 4em }
table.docutils {
margin-top: 0.5em ;
margin-bottom: 0.5em }
table.footnote {
border-left: solid 1px black;
margin-left: 1px }
table.docutils td, table.docutils th,
table.docinfo td, table.docinfo th {
padding-left: 0.5em ;
padding-right: 0.5em ;
vertical-align: top }
table.docutils th.field-name, table.docinfo th.docinfo-name {
font-weight: bold ;
text-align: left ;
white-space: nowrap ;
padding-left: 0 }
/* "booktabs" style (no vertical lines) */
table.docutils.booktabs {
border: 0px;
border-top: 2px solid;
border-bottom: 2px solid;
border-collapse: collapse;
}
table.docutils.booktabs * {
border: 0px;
}
table.docutils.booktabs th {
border-bottom: thin solid;
text-align: left;
}
h1 tt.docutils, h2 tt.docutils, h3 tt.docutils,
h4 tt.docutils, h5 tt.docutils, h6 tt.docutils {
font-size: 100% }
ul.auto-toc {
list-style-type: none }
</style>
</head>
<body>
<div class="document" id="bus-alt-connection">
<h1 class="title">Bus Alt Connection</h1>
<!-- !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!! This file is generated by oca-gen-addon-readme !!
!! changes will be overwritten. !!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! -->
<p><a class="reference external" href="https://odoo-community.org/page/development-status"><img alt="Beta" src="https://img.shields.io/badge/maturity-Beta-yellow.png" /></a> <a class="reference external" href="http://www.gnu.org/licenses/agpl-3.0-standalone.html"><img alt="License: AGPL-3" src="https://img.shields.io/badge/licence-AGPL--3-blue.png" /></a> <a class="reference external" href="https://github.com/OCA/server-tools/tree/12.0/bus_alt_connection"><img alt="OCA/server-tools" src="https://img.shields.io/badge/github-OCA%2Fserver--tools-lightgray.png?logo=github" /></a> <a class="reference external" href="https://translation.odoo-community.org/projects/server-tools-12-0/server-tools-12-0-bus_alt_connection"><img alt="Translate me on Weblate" src="https://img.shields.io/badge/weblate-Translate%20me-F47D42.png" /></a> <a class="reference external" href="https://runbot.odoo-community.org/runbot/149/12.0"><img alt="Try me on Runbot" src="https://img.shields.io/badge/runbot-Try%20me-875A7B.png" /></a></p>
<p>This module makes it possible to use <a class="reference external" href="https://pgbouncer.github.io/">PgBouncer</a> as a connection pooler
for odoo.</p>
<div class="section" id="why-isn-t-odoo-s-connection-pooling-good-enough">
<h1>Why isn’t odoo’s connection pooling good enough?</h1>
<p>Odoo’s builtin connection pooling works at process level: each Odoo process
has its own <a class="reference external" href="https://github.com/odoo/odoo/blob/12.0/odoo/sql_db.py#L525">ConnectionPool</a>, limited to <tt class="docutils literal">db_maxconn</tt>.</p>
<p>It does the job of re-using open connections available in the pool.
But it never closes these connections, <a class="reference external" href="https://github.com/odoo/odoo/blob/12.0/odoo/sql_db.py#L593">unless reaching db_maxconn</a>.</p>
<p>In practice, we observe that each odoo worker will end up
with up to 3 open connection in its pool.
With 10 http workers, that’s up to 30 connection continuously open just
for one single instance.</p>
</div>
<div class="section" id="here-comes-pgbouncer">
<h1>Here comes PgBouncer</h1>
<p>PgBouncer will help to limit this number of open connections,
by sharing a pool of connections at the instance level, between
all workers. Odoo workers will still have up to 3 open connections,
but these will be connections to PgBouncer, that on its side will
close unnecessary connections to pg.</p>
<p>This has proven to help performances on Odoo deployments with
multiple instances.</p>
<p>It allows you to define how resources should be shared,
according to your priorities, e.g. :</p>
<ul class="simple">
<li>key odoo instance on host A can open up to 30 connections</li>
<li>while odoo instance on host B, dedicated to reports,
can open up to 10 connections only</li>
</ul>
<p>And most importantly, it helps you to ensure that
<tt class="docutils literal">max_connections</tt> will never be reached on pg server side.</p>
</div>
<div class="section" id="why-is-this-module-needed">
<h1>Why is this module needed?</h1>
<p>When configuring PgBouncer, you can choose between 2 transaction pooling modes:</p>
<ul class="simple">
<li><cite>pool_mode = session</cite></li>
<li><cite>pool_mode = transaction</cite></li>
</ul>
<p>If we choose <cite>pool_mode = session</cite>, then one server connection will be tied
to a given odoo process until its death, which is exactly what we’re trying
to change. Thus, to release the server connection once the transaction is
complete, we use <cite>pool_mode = transaction</cite>.</p>
<p>This works fine, except for Odoo’s longpolling features that relies
on the <a class="reference external" href="https://www.postgresql.org/docs/9.6/static/sql-notify.html">LISTEN/NOTIFY</a> mechanism from pg, which is <a class="reference external" href="https://wiki.postgresql.org/wiki/PgBouncer">not compatible</a> with that
mode.</p>
<p>To be more precise, <cite>NOTIFY</cite> statements are properly transfered by PgBouncer
in that mode; only the <cite>LISTEN</cite> statement isn’t (because it needs to keep the
server connection open).</p>
<p>So for the unique “listening” connection per instance that requires this
statement (<a class="reference external" href="https://github.com/odoo/odoo/blob/12.0/addons/bus/models/bus.py#L166">here</a>), we need odoo to connect directly to the pg server, bypassing
PgBouncer.</p>
<p>That’s what this module implements, by overriding the relevant method
of the <a class="reference external" href="https://github.com/odoo/odoo/blob/12.0/addons/bus/models/bus.py#L105">Dispatcher</a>.</p>
<p><strong>Table of contents</strong></p>
</div>
<div class="section" id="installation">
<h1>Installation</h1>
<p>You don’t need to install this module in the database(s) to enable it.</p>
<p>But you need to load it server-wide:</p>
<ul class="simple">
<li>By starting Odoo with <tt class="docutils literal"><span class="pre">--load=web,bus_alt_connection</span></tt></li>
<li>Or by updating its configuration file:</li>
</ul>
<pre class="code ini literal-block">
<span class="k">[options]</span>
<span class="na">(...)</span>
<span class="na">server_wide_modules</span> <span class="o">=</span> <span class="s">web,bus_alt_connection</span>
</pre>
</div>
<div class="section" id="configuration">
<h1>Configuration</h1>
<p>You need to define how to connect directly to the database:</p>
<ul>
<li><p class="first">Either by defining environment variables:</p>
<blockquote>
<ul class="simple">
<li><tt class="docutils literal"><span class="pre">IMDISPATCHER_DB_HOST=db-01</span></tt></li>
<li><tt class="docutils literal">IMDISPATCHER_DB_PORT=5432</tt></li>
</ul>
</blockquote>
</li>
<li><p class="first">Or in Odoo’s configuration file:</p>
</li>
</ul>
<pre class="code ini literal-block">
<span class="k">[options]</span>
<span class="na">(...)</span>
<span class="na">imdispatcher_db_host</span> <span class="o">=</span> <span class="s">db-01</span>
<span class="na">imdispatcher_db_port</span> <span class="o">=</span> <span class="s">5432</span>
</pre>
</div>
<div class="section" id="bug-tracker">
<h1>Bug Tracker</h1>
<p>Bugs are tracked on <a class="reference external" href="https://github.com/OCA/server-tools/issues">GitHub Issues</a>.
In case of trouble, please check there if your issue has already been reported.
If you spotted it first, help us smashing it by providing a detailed and welcomed
<a class="reference external" href="https://github.com/OCA/server-tools/issues/new?body=module:%20bus_alt_connection%0Aversion:%2012.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**">feedback</a>.</p>
<p>Do not contact contributors directly about support or help with technical issues.</p>
</div>
<div class="section" id="credits">
<h1>Credits</h1>
<div class="section" id="authors">
<h2>Authors</h2>
<ul class="simple">
<li>Trobz</li>
</ul>
</div>
<div class="section" id="contributors">
<h2>Contributors</h2>
<ul class="simple">
<li>Nils Hamerlinck &lt;<a class="reference external" href="mailto:nils&#64;trobz.com">nils&#64;trobz.com</a>&gt;</li>
</ul>
</div>
<div class="section" id="maintainers">
<h2>Maintainers</h2>
<p>This module is maintained by the OCA.</p>
<a class="reference external image-reference" href="https://odoo-community.org"><img alt="Odoo Community Association" src="https://odoo-community.org/logo.png" /></a>
<p>OCA, or the Odoo Community Association, is a nonprofit organization whose
mission is to support the collaborative development of Odoo features and
promote its widespread use.</p>
<p>This module is part of the <a class="reference external" href="https://github.com/OCA/server-tools/tree/12.0/bus_alt_connection">OCA/server-tools</a> project on GitHub.</p>
<p>You are welcome to contribute. To learn how please visit <a class="reference external" href="https://odoo-community.org/page/Contribute">https://odoo-community.org/page/Contribute</a>.</p>
</div>
</div>
</div>
</body>
</html>
Loading…
Cancel
Save