server-tools/bus_alt_connection/README.rst

==================
Bus Alt Connection
==================

.. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
   !! This file is generated by oca-gen-addon-readme !!
   !! changes will be overwritten.                   !!
   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

.. |badge1| image:: https://img.shields.io/badge/maturity-Beta-yellow.png
    :target: https://odoo-community.org/page/development-status
    :alt: Beta
.. |badge2| image:: https://img.shields.io/badge/licence-AGPL--3-blue.png
    :target: http://www.gnu.org/licenses/agpl-3.0-standalone.html
    :alt: License: AGPL-3
.. |badge3| image:: https://img.shields.io/badge/github-OCA%2Fserver--tools-lightgray.png?logo=github
    :target: https://github.com/OCA/server-tools/tree/12.0/bus_alt_connection
    :alt: OCA/server-tools
.. |badge4| image:: https://img.shields.io/badge/weblate-Translate%20me-F47D42.png
    :target: https://translation.odoo-community.org/projects/server-tools-12-0/server-tools-12-0-bus_alt_connection
    :alt: Translate me on Weblate
.. |badge5| image:: https://img.shields.io/badge/runbot-Try%20me-875A7B.png
    :target: https://runbot.odoo-community.org/runbot/149/12.0
    :alt: Try me on Runbot

|badge1| |badge2| |badge3| |badge4| |badge5| 

This module makes it possible to use PgBouncer_ as a connection pooler
for odoo.

.. _PgBouncer: https://pgbouncer.github.io/

Why isn't odoo's connection pooling good enough?
================================================

Odoo's builtin connection pooling works at process level: each Odoo process
has its own ConnectionPool_, limited to ``db_maxconn``.

It does the job of re-using open connections available in the pool.
But it never closes these connections, `unless reaching db_maxconn`_.

.. _ConnectionPool: https://github.com/odoo/odoo/blob/12.0/odoo/sql_db.py#L525
.. _`unless reaching db_maxconn`: https://github.com/odoo/odoo/blob/12.0/odoo/sql_db.py#L593

In practice, we observe that each odoo worker will end up
with up to 3 open connection in its pool.
With 10 http workers, that's up to 30 connection continuously open just
for one single instance.

Here comes PgBouncer
====================

PgBouncer will help to limit this number of open connections,
by sharing a pool of connections at the instance level, between
all workers. Odoo workers will still have up to 3 open connections,
but these will be connections to PgBouncer, that on its side will
close unnecessary connections to pg.

This has proven to help performances on Odoo deployments with
multiple instances.

It allows you to define how resources should be shared,
according to your priorities, e.g. :

* key odoo instance on host A can open up to 30 connections
* while odoo instance on host B, dedicated to reports,
  can open up to 10 connections only

And most importantly, it helps you to ensure that
``max_connections`` will never be reached on pg server side.


Why is this module needed?
==========================

When configuring PgBouncer, you can choose between 2 transaction pooling modes:

* `pool_mode = session`
* `pool_mode = transaction`

If we choose `pool_mode = session`, then one server connection will be tied
to a given odoo process until its death, which is exactly what we're trying
to change. Thus, to release the server connection once the transaction is
complete, we use `pool_mode = transaction`.

This works fine, except for Odoo's longpolling features that relies
on the `LISTEN/NOTIFY`_ mechanism from pg, which is `not compatible`_ with that
mode.

.. _`LISTEN/NOTIFY`: https://www.postgresql.org/docs/9.6/static/sql-notify.html
.. _`not compatible`: https://wiki.postgresql.org/wiki/PgBouncer


To be more precise, `NOTIFY` statements are properly transfered by PgBouncer
in that mode; only the `LISTEN` statement isn't (because it needs to keep the
server connection open).

So for the unique "listening" connection per instance that requires this
statement (here_), we need odoo to connect directly to the pg server, bypassing
PgBouncer.

That's what this module implements, by overriding the relevant method
of the Dispatcher_.

.. _here: https://github.com/odoo/odoo/blob/12.0/addons/bus/models/bus.py#L166
.. _Dispatcher: https://github.com/odoo/odoo/blob/12.0/addons/bus/models/bus.py#L105

**Table of contents**

.. contents::
   :local:

Installation
============

You don't need to install this module in the database(s) to enable it.

But you need to load it server-wide:

* By starting Odoo with ``--load=web,bus_alt_connection``

* Or by updating its configuration file:

.. code-block:: ini

  [options]
  (...)
  server_wide_modules = web,bus_alt_connection

Configuration
=============

You need to define how to connect directly to the database:

* Either by defining environment variables:

    - ``IMDISPATCHER_DB_HOST=db-01``
    - ``IMDISPATCHER_DB_PORT=5432``

* Or in Odoo's configuration file:

.. code-block:: ini

  [options]
  (...)
  imdispatcher_db_host = db-01
  imdispatcher_db_port = 5432

Bug Tracker
===========

Bugs are tracked on `GitHub Issues <https://github.com/OCA/server-tools/issues>`_.
In case of trouble, please check there if your issue has already been reported.
If you spotted it first, help us smashing it by providing a detailed and welcomed
`feedback <https://github.com/OCA/server-tools/issues/new?body=module:%20bus_alt_connection%0Aversion:%2012.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**>`_.

Do not contact contributors directly about support or help with technical issues.

Credits
=======

Authors
~~~~~~~

* Trobz

Contributors
~~~~~~~~~~~~

* Nils Hamerlinck <nils@trobz.com>

Maintainers
~~~~~~~~~~~

This module is maintained by the OCA.

.. image:: https://odoo-community.org/logo.png
   :alt: Odoo Community Association
   :target: https://odoo-community.org

OCA, or the Odoo Community Association, is a nonprofit organization whose
mission is to support the collaborative development of Odoo features and
promote its widespread use.

This module is part of the `OCA/server-tools <https://github.com/OCA/server-tools/tree/12.0/bus_alt_connection>`_ project on GitHub.

You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute.