Merge pull request #180 from hbrunn/10.0-report_py3o_fusion_server

[RFR] split off the fusion server to its own module; use libreoffice for conversions in the base version

.travis.yml

@@ -6,6 +6,7 @@ addons:
     packages:
       - expect-dev  # provides unbuffer utility
       - python-lxml # because pip installation is slow
+      - libreoffice
 
 language: python
 

report_py3o/README.rst

@@ -12,99 +12,30 @@ The py3o reporting engine is a reporting engine for Odoo based on `Libreoffice <
 * the report is stored on the server in OpenDocument format (.odt or .ods file)
 * the report is sent to the user in OpenDocument format or in any output format supported by Libreoffice (PDF, HTML, DOC, DOCX, Docbook, XLS, etc.)
 
-The key advantages of a Libreoffice-based reporting engine are:
-
-* no need to be a developper to create or modify a report: the report is created and modified with Libreoffice. So this reporting engine has a fully WYSIWYG report developpment tool!
-
-* For a PDF report in A4/Letter format, it's easier to develop it with a tool such as Libreoffice that is designed to create A4/Letter documents than to develop it in HTML/CSS.
+The key advantages of a Libreoffice based reporting engine are:
 
+* no need to be a developer to create or modify a report: the report is created and modified with Libreoffice. So this reporting engine has a full WYSIWYG report development tool!
+* For a PDF report in A4/Letter format, it's easier to develop it with a tool such as Libreoffice that is designed to create A4/Letter documents than to develop it in HTML/CSS, also some print peculiarities (backgrounds, margin boxes) are not very well supported by the HTML/CSS based solutions.
 * If you want your users to be able to modify the document after its generation by Odoo, just configure the document with ODT output (or DOC or DOCX) and the user will be able to modify the document with Libreoffice (or Word) after its generation by Odoo.
-
 * Easy development of spreadsheet reports in ODS format (XLS output possible).
 
-This reporting engine is an alternative to `Aeroo <https://github.com/aeroo/aeroo_reports>`_: these 2 reporting engines have similar features but their codes are completely different.
+This reporting engine is an alternative to `Aeroo <https://github.com/aeroo-community/aeroo_reports>`_: these two reporting engines have similar features but their implementation is entirely different. You cannot use aeroo templates as drop in replacement though, you'll have to change a few details.
 
 Installation
 ============
 
-You must install 2 additionnal python libs:
+Install the required python libs:
 
 .. code::
 
   pip install py3o.template
   pip install py3o.formats
 
-To allow the conversion of ODT or ODS reports to other formats (PDF, DOC, DOCX, etc.), you must install several additionnal components and Python libs:
-
-* `Py3o Fusion server <https://bitbucket.org/faide/py3o.fusion>`_,
-* `Py3o render server <https://bitbucket.org/faide/py3o.renderserver>`_,
-* a Java Runtime Environment (JRE), which can be OpenJDK,
-* Libreoffice started in the background in headless mode,
-* the Java driver for Libreoffice (Juno).
-
-It is also possible to use the Python driver for Libreoffice (PyUNO), but it is recommended to use the Java driver because it is more stable.
-
-The installation procedure below uses the Java driver. It has been successfully tested on Ubuntu 16.04 LTS ; if you use another OS, you may have to change a few details.
-
-Installation of py3o.fusion:
-
-.. code::
-
-  pip install py3o.fusion
-  pip install service-identity
-
-Installation of py3o.renderserver:
-
-.. code::
-
-  pip install py3o.renderserver
-
-Installation of Libreoffice and JRE on Debian/Ubuntu:
-
-.. code::
-
-  sudo apt-get install default-jre ure libreoffice-java-common libreoffice-writer
-
-You may have to install additionnal fonts. For example, to have the special unicode symbols for phone/fax/email in the PDF reports generated by Py3o, you should install the following package:
-
-.. code::
-
-  sudo apt-get install fonts-symbola
-
-At the end, with the dependencies, you should have the following py3o python libs:
-
-.. code::
-
-  % pip freeze | grep py3o
-  py3o.formats==0.3
-  py3o.fusion==0.8.6
-  py3o.renderclient==0.2
-  py3o.renderers.juno==0.8
-  py3o.renderserver==0.5.1
-  py3o.template==0.9.11
-  py3o.types==0.1.1
-
-Start the Py3o Fusion server:
-
-.. code::
-
-  start-py3o-fusion --debug -s localhost
-
-Start the Py3o render server:
-
-.. code::
-
-  start-py3o-renderserver --java=/usr/lib/jvm/default-java/jre/lib/amd64/server/libjvm.so --ure=/usr/share --office=/usr/lib/libreoffice --driver=juno --sofficeport=8997
-
-On the output of the Py3o render server, the first line looks like:
+To allow the conversion of ODT or ODS reports to other formats (PDF, DOC, DOCX, etc.), install libreoffice:
 
 .. code::
 
-  DEBUG:root:Starting JVM: /usr/lib/jvm/default-java/jre/lib/amd64/server/libjvm.so with options: -Djava.class.path=/usr/local/lib/python2.7/dist-packages/py3o/renderers/juno/py3oconverter.jar:/usr/share/java/juh.jar:/usr/share/java/jurt.jar:/usr/share/java/ridl.jar:/usr/share/java/unoloader.jar:/usr/share/java/java_uno.jar:/usr/lib/libreoffice/program/classes/unoil.jar -Xmx150M
-
-After **-Djava.class.path**, there is a list of Java libs with *.jar* extension ; check that each JAR file is really present on your filesystem. If one of the jar files is present in another directory, create a symlink that points to the real location of the file. If all the jar files are present on another directory, adapt the *--ure=* argument on the command line of Py3o render server.
-
-To check that the Py3o Fusion server is running fine, visit the URL http://<IP_address>:8765/form. On this web page, under the section *Target format*, make sure that you have a line *This server currently supports these formats: ods, odt, docx, doc, html, docbook, pdf, xls.*.
+  apt-get --no-install-recommends install libreoffice
 
 Configuration
 =============
@@ -139,13 +70,12 @@ the path to the template as *py3o_template_fallback*.
   <record id="account.account_invoices" model="ir.actions.report.xml">
       <field name="report_type">py3o</field>
       <field name="py3o_filetype">odt</field>
-      <field name="module">/field>
       <field name="py3o_template_fallback">/odoo/templates/py3o/report/account_invoice.odt</field>
   </record>
 
   </odoo>
 
-Moreover you must also modify the odoo server configuration file to declare
+Moreover, you must also modify the Odoo server configuration file to declare
 the allowed root directory for your py3o templates. Only templates located
 into this directory can be loaded by py3o report.
 
@@ -164,14 +94,9 @@ If you want an invoice in PDF format instead of ODT format, the XML file should
   <?xml version="1.0" encoding="utf-8"?>
   <odoo>
 
-  <record id="local_py3o_server" model="py3o.server">
-      <field name="url">http://localhost:8765/form</field>
-  </record>
-
   <record id="account.account_invoices" model="ir.actions.report.xml">
       <field name="report_type">py3o</field>
       <field name="py3o_filetype">pdf</field>
-      <field name="py3o_server_id" ref="local_py3o_server"/>
       <field name="module">my_custom_module_base</field>
       <field name="py3o_template_fallback">report/account_invoice.odt</field>
   </record>
@@ -185,17 +110,12 @@ If you want to add a new py3o PDF report (and not replace a native report), the
   <?xml version="1.0" encoding="utf-8"?>
   <odoo>
 
-  <record id="local_py3o_server" model="py3o.server">
-      <field name="url">http://localhost:8765/form</field>
-  </record>
-
   <record id="partner_summary_report" model="ir.actions.report.xml">
       <field name="name">Partner Summary</field>
       <field name="model">res.partner</field>
       <field name="report_name">res.partner.summary</field>
       <field name="report_type">py3o</field>
       <field name="py3o_filetype">pdf</field>
-      <field name="py3o_server_id" ref="local_py3o_server"/>
       <field name="module">my_custom_module_base</field>
       <field name="py3o_template_fallback">report/partner_summary.odt</field>
   </record>
@@ -210,6 +130,12 @@ If you want to add a new py3o PDF report (and not replace a native report), the
 
   </odoo>
 
+Configuration parameters
+------------------------
+
+py3o.conversion_command
+    The command to be used to run the conversion, ``libreoffice`` by default. If you change this, whatever you set here must accept the parameters ``--headless --convert-to $ext $file`` and put the resulting file into ``$file``'s directory with extension ``$ext``. The command will be started in ``$file``'s directory.
+
 Usage
 =====
 
@@ -217,10 +143,37 @@ Usage
    :alt: Try me on Runbot
    :target: https://runbot.odoo-community.org/runbot/143/10.0
 
+The templating language is `extensively documented <http://py3otemplate.readthedocs.io/en/latest/templating.html>`_, the records are exposed in libreoffice as ``objects``, on which you can also call functions.
+
+Available functions and objects
+-------------------------------
+
+user
+    Browse record of current user
+lang
+    The user's company's language as string (ISO code)
+b64decode
+    ``base64.b64decode``
+format_multiline_value(string)
+    Generate the ODF equivalent of ``<br/>`` and ``&nbsp;`` for multiline fields (ODF is XML internally, so those would be skipped otherwise)
+html_sanitize(string)
+    Sanitize HTML string
+time
+    Python's ``time`` module
+display_address(partner)
+    Return a formatted string of the partner's address
+
+Sample report templates
+-----------------------
+
+Sample py3o report templates for the main Odoo native reports (invoice, sale order, purchase order, picking, ...) are available on the Github project `odoo-py3o-report-templates <https://github.com/akretion/odoo-py3o-report-templates>`_.
+
 Known issues / Roadmap
 ======================
 
 * generate barcode ?
+* add more detailed example in demo file to showcase features
+* add migration guide aeroo -> py3o
 
 Bug Tracker
 ===========
@@ -241,6 +194,7 @@ Contributors
 * Alexis de Lattre <alexis.delattre@akretion.com>,
 * Guewen Baconnier <guewen.baconnier@camptocamp.com>
 * Omar Castiñeira <omar@comunitea.com>
+* Holger Brunn <hbrunn@therp.nl>
 
 
 Maintainer

report_py3o/__manifest__.py

@@ -5,7 +5,7 @@
     'name': 'Py3o Report Engine',
     'summary': 'Reporting engine based on Libreoffice (ODT -> ODT, '
                'ODT -> PDF, ODT -> DOC, ODT -> DOCX, ODS -> ODS, etc.)',
-    'version': '10.0.1.2.0',
+    'version': '10.0.2.0.0',
     'category': 'Reporting',
     'license': 'AGPL-3',
     'author': 'XCG Consulting,'
@@ -21,7 +21,6 @@
         'security/ir.model.access.csv',
         'views/menu.xml',
         'views/py3o_template.xml',
-        'views/py3o_server.xml',
         'views/ir_report.xml',
         'views/report_py3o.xml',
         'demo/report_py3o.xml',

report_py3o/demo/report_py3o.xml

@@ -11,8 +11,6 @@
       <field name="report_name">py3o_user_info</field>
       <field name="report_type">py3o</field>
       <field name="py3o_filetype">odt</field>
-      <field name="py3o_is_local_fusion" eval="1"/>
-      <field name="py3o_filetype">odt</field>
       <field name="module">report_py3o</field>
       <field name="py3o_template_fallback">demo/res_user.odt</field>
     </record>

report_py3o/migrations/10.0.2.0.0/pre-migration.py

@@ -0,0 +1,12 @@
+# -*- coding: utf-8 -*-
+# © 2018 Therp BV <http://therp.nl>
+# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
+
+
+def migrate(cr, version=None):
+    # when migrating from a pre-split version of the module, pull the fusion
+    # server module too to have no loss of features
+    cr.execute(
+        "update ir_module_module set state='to install' "
+        "where name='report_py3o_fusion_server' and state='uninstalled'"
+    )

report_py3o/models/__init__.py

@@ -1,5 +1,4 @@
 from . import ir_actions_report_xml
 from . import py3o_template
-from . import py3o_server
 from . import report
 from . import py3o_report

report_py3o/models/ir_actions_report_xml.py

@@ -31,19 +31,6 @@ class IrActionsReportXml(models.Model):
                 raise ValidationError(_(
                     "Field 'Output Format' is required for Py3O report"))
 
-    @api.multi
-    @api.constrains("py3o_is_local_fusion", "py3o_server_id",
-                    "py3o_filetype")
-    def _check_py3o_server_id(self):
-        for report in self:
-            if report.report_type == "py3o":
-                is_native = Formats().get_format(report.py3o_filetype).native
-                if ((not is_native or not report.py3o_is_local_fusion) and
-                        not report.py3o_server_id):
-                    raise ValidationError(_(
-                        "Can not use not native format in local fusion. "
-                        "Please specify a Fusion Server"))
-
     @api.model
     def _get_py3o_filetypes(self):
         formats = Formats()
@@ -62,15 +49,6 @@ class IrActionsReportXml(models.Model):
     py3o_template_id = fields.Many2one(
         'py3o.template',
         "Template")
-    py3o_is_local_fusion = fields.Boolean(
-        "Local Fusion",
-        help="Native formats will be processed without a server. "
-             "You must use this mode if you call methods on your model into "
-             "the template.",
-        default=True)
-    py3o_server_id = fields.Many2one(
-        "py3o.server",
-        "Fusion Server")
     module = fields.Char(
         "Module",
         help="The implementer module that provides this report")

report_py3o/models/py3o_report.py

@@ -5,26 +5,23 @@
 import base64
 from base64 import b64decode
 from cStringIO import StringIO
-import json
 import logging
 import os
 from contextlib import closing
+import subprocess
 
 import pkg_resources
-import requests
 import sys
 import tempfile
 from zipfile import ZipFile, ZIP_DEFLATED
 
 from odoo.exceptions import AccessError
-from odoo.exceptions import UserError
 from odoo.report.report_sxw import rml_parse
 from odoo import api, fields, models, tools, _
 
 logger = logging.getLogger(__name__)
 
 try:
-    from py3o.template.helpers import Py3oConvertor
     from py3o.template import Template
     from py3o import formats
     from genshi.core import Markup
@@ -244,60 +241,63 @@ class Py3oReport(models.TransientModel):
         """ This function to generate our py3o report
         """
         self.ensure_one()
-        report_xml = self.ir_actions_report_xml_id
-        filetype = report_xml.py3o_filetype
         result_fd, result_path = tempfile.mkstemp(
-            suffix='.' + filetype, prefix='p3o.report.tmp.')
+            suffix='.ods', prefix='p3o.report.tmp.')
         tmpl_data = self.get_template(model_instance)
 
         in_stream = StringIO(tmpl_data)
         with closing(os.fdopen(result_fd, 'w+')) as out_stream:
             template = Template(in_stream, out_stream, escape_false=True)
             localcontext = self._get_parser_context(model_instance, data)
-            is_native = Formats().get_format(filetype).native
-            if report_xml.py3o_is_local_fusion:
             template.render(localcontext)
             out_stream.seek(0)
             tmpl_data = out_stream.read()
-                datadict = {}
-            else:
-                expressions = template.get_all_user_python_expression()
-                py_expression = template.convert_py3o_to_python_ast(
-                    expressions)
-                convertor = Py3oConvertor()
-                data_struct = convertor(py_expression)
-                datadict = data_struct.render(localcontext)
-
-        if not is_native or not report_xml.py3o_is_local_fusion:
-            # Call py3o.server to render the template in the desired format
-            files = {
-                'tmpl_file': tmpl_data,
-            }
-            fields = {
-                "targetformat": filetype,
-                "datadict": json.dumps(datadict),
-                "image_mapping": "{}",
-                "escape_false": "on",
-            }
-            if report_xml.py3o_is_local_fusion:
-                fields['skipfusion'] = '1'
-            r = requests.post(
-                report_xml.py3o_server_id.url, data=fields, files=files)
-            if r.status_code != 200:
-                # server says we have an issue... let's tell that to enduser
-                raise UserError(
-                    _('Fusion server error %s') % r.text,
+
+        result_path = self._convert_single_report(
+            result_path, model_instance, data
         )
 
-            chunk_size = 1024
-            with open(result_path, 'w+') as fd:
-                for chunk in r.iter_content(chunk_size):
-                    fd.write(chunk)
         if len(model_instance) == 1:
             self._postprocess_report(
                 result_path, model_instance.id, save_in_attachment)
+
+        return result_path
+
+    @api.multi
+    def _convert_single_report(self, result_path, model_instance, data):
+        """Run a command to convert to our target format"""
+        filetype = self.ir_actions_report_xml_id.py3o_filetype
+        if not Formats().get_format(filetype).native:
+            command = self._convert_single_report_cmd(
+                result_path, model_instance, data,
+            )
+            logger.debug('Running command %s', command)
+            output = subprocess.check_output(
+                command, cwd=os.path.dirname(result_path),
+            )
+            logger.debug('Output was %s', output)
+            self._cleanup_tempfiles([result_path])
+            result_path, result_filename = os.path.split(result_path)
+            result_path = os.path.join(
+                result_path, '%s.%s' % (
+                    os.path.splitext(result_filename)[0], filetype
+                )
+            )
         return result_path
 
+    @api.multi
+    def _convert_single_report_cmd(self, result_path, model_instance, data):
+        """Return a command list suitable for use in subprocess.call"""
+        return [
+            self.env['ir.config_parameter'].get_param(
+                'py3o.conversion_command', 'libreoffice',
+            ),
+            '--headless',
+            '--convert-to',
+            self.ir_actions_report_xml_id.py3o_filetype,
+            result_path,
+        ]
+
     @api.multi
     def _get_or_create_single_report(self, model_instance, data,
                                      save_in_attachment):

report_py3o/models/py3o_template.py

@@ -13,6 +13,10 @@ class Py3oTemplate(models.Model):
         selection=[
             ('odt', "ODF Text Document"),
             ('ods', "ODF Spreadsheet"),
+            ('odp', "ODF Presentation"),
+            ('fodt', "ODF Text Document (Flat)"),
+            ('fods', "ODF Spreadsheet (Flat)"),
+            ('fodp', "ODF Presentation (Flat)"),
         ],
         string="LibreOffice Template File Type",
         required=True,

report_py3o/security/ir.model.access.csv

@@ -1,5 +1,3 @@
 id,name,model_id:id,group_id:id,perm_read,perm_write,perm_create,perm_unlink
 access_py3o_template_admin,access_py3o_template_admin,model_py3o_template,base.group_no_one,1,1,1,1
 access_py3o_template_user,access_py3o_template_user,model_py3o_template,base.group_user,1,0,0,0
-access_py3o_server_admin,access_py3o_server_admin,model_py3o_server,base.group_no_one,1,1,1,1
-access_py3o_server_user,access_py3o_server_user,model_py3o_server,base.group_user,1,0,0,0

report_py3o/tests/test_report_py3o.py

@@ -11,8 +11,6 @@ import shutil
 import tempfile
 from contextlib import contextmanager
 
-from py3o.formats import Formats
-
 from odoo import tools
 from odoo.tests.common import TransactionCase
 from odoo.exceptions import ValidationError
@@ -40,34 +38,6 @@ class TestReportPy3o(TransactionCase):
         self.py3o_report = self.env['py3o.report'].create({
             'ir_actions_report_xml_id': self.report.id})
 
-    def test_no_local_fusion_without_fusion_server(self):
-        self.assertTrue(self.report.py3o_is_local_fusion)
-        with self.assertRaises(ValidationError) as e:
-            self.report.py3o_is_local_fusion = False
-        self.assertEqual(
-            e.exception.name,
-            "Can not use not native format in local fusion. "
-            "Please specify a Fusion Server")
-
-    def test_no_native_format_without_fusion_server(self):
-        report = self.env.ref("report_py3o.res_users_report_py3o")
-        formats = Formats()
-        is_native = formats.get_format(report.py3o_filetype).native
-        self.assertTrue(is_native)
-        new_format = None
-        for name in formats.get_known_format_names():
-            format = formats.get_format(name)
-            if not format.native:
-                new_format = name
-                break
-        self.assertTrue(new_format)
-        with self.assertRaises(ValidationError) as e:
-            report.py3o_filetype = new_format
-        self.assertEqual(
-            e.exception.name,
-            "Can not use not native format in local fusion. "
-            "Please specify a Fusion Server")
-
     def test_required_py3_filetype(self):
         self.assertEqual(self.report.report_type, "py3o")
         with self.assertRaises(ValidationError) as e:
@@ -76,70 +46,40 @@ class TestReportPy3o(TransactionCase):
             e.exception.name,
             "Field 'Output Format' is required for Py3O report")
 
-    def test_reports(self):
+    def _render_patched(self, result_text='test result', call_count=1):
         py3o_report = self.env['py3o.report']
         with mock.patch.object(
                 py3o_report.__class__, '_create_single_report') as patched_pdf:
             result = tempfile.mktemp('.txt')
             with open(result, 'w') as fp:
-                fp.write('dummy')
+                fp.write(result_text)
             patched_pdf.return_value = result
+            patched_pdf.side_effect = lambda record, data, save_attachments:\
+                py3o_report._postprocess_report(
+                    result, record.id, save_attachments,
+                ) or result
             # test the call the the create method inside our custom parser
             self.report.render_report(self.env.user.ids,
                                       self.report.report_name,
                                       {})
-            self.assertEqual(1, patched_pdf.call_count)
+            self.assertEqual(call_count, patched_pdf.call_count)
             # generated files no more exists
             self.assertFalse(os.path.exists(result))
+
+    def test_reports(self):
         res = self.report.render_report(
             self.env.user.ids, self.report.report_name, {})
         self.assertTrue(res)
-        py3o_server = self.env['py3o.server'].create({"url": "http://dummy"})
-        # check the call to the fusion server
-        self.report.write({"py3o_filetype": "pdf",
-                           "py3o_server_id": py3o_server.id})
-        with mock.patch('requests.post') as patched_post:
-            magick_response = mock.MagicMock()
-            magick_response.status_code = 200
-            patched_post.return_value = magick_response
-            magick_response.iter_content.return_value = "test result"
+        self.report.py3o_filetype = 'pdf'
         res = self.report.render_report(
             self.env.user.ids, self.report.report_name, {})
-            self.assertEqual(('test result', 'pdf'), res)
+        self.assertTrue(res)
 
     def test_report_load_from_attachment(self):
-        py3o_report = self.env['py3o.report']
-        with mock.patch.object(
-                py3o_report.__class__, '_create_single_report') as patched_pdf:
-            result = tempfile.mktemp('.txt')
-            with open(result, 'w') as fp:
-                fp.write('dummy')
-            patched_pdf.return_value = result
-            # test the call the the create method inside our custom parser
-            self.report.render_report(self.env.user.ids,
-                                      self.report.report_name,
-                                      {})
-            self.assertEqual(1, patched_pdf.call_count)
-            # generated files no more exists
-            self.assertFalse(os.path.exists(result))
-        res = self.report.render_report(
-            self.env.user.ids, self.report.report_name, {})
-        self.assertTrue(res)
-        py3o_server = self.env['py3o.server'].create({"url": "http://dummy"})
-        # check the call to the fusion server
-        self.report.write({"py3o_filetype": "pdf",
-                           "py3o_server_id": py3o_server.id,
-                           "attachment_use": True,
+        self.report.write({"attachment_use": True,
                            "attachment": "'my_saved_report'"})
         attachments = self.env['ir.attachment'].search([])
-        with mock.patch('requests.post') as patched_post:
-            magick_response = mock.MagicMock()
-            magick_response.status_code = 200
-            patched_post.return_value = magick_response
-            magick_response.iter_content.return_value = "test result"
-            res = self.report.render_report(
-                self.env.user.ids, self.report.report_name, {})
-            self.assertEqual(('test result', 'pdf'), res)
+        self._render_patched()
         new_attachments = self.env['ir.attachment'].search([])
         created_attachement = new_attachments - attachments
         self.assertEqual(1, len(created_attachement))
@@ -151,29 +91,17 @@ class TestReportPy3o(TransactionCase):
         created_attachement.datas = base64.encodestring("new content")
         res = self.report.render_report(
             self.env.user.ids, self.report.report_name, {})
-        self.assertEqual(('new content', 'pdf'), res)
+        self.assertEqual(('new content', self.report.py3o_filetype), res)
 
     def test_report_post_process(self):
         """
         By default the post_process method is in charge to save the
         generated report into an ir.attachment if requested.
         """
-        report = self.env.ref("report_py3o.res_users_report_py3o")
-        report.attachment = "object.name + '.txt'"
-        py3o_server = self.env['py3o.server'].create({"url": "http://dummy"})
-        # check the call to the fusion server
-        report.write({"py3o_filetype": "pdf",
-                      "py3o_server_id": py3o_server.id})
+        self.report.attachment = "object.name + '.txt'"
         ir_attachment = self.env['ir.attachment']
         attachements = ir_attachment.search([(1, '=', 1)])
-        with mock.patch('requests.post') as patched_post:
-            magick_response = mock.MagicMock()
-            magick_response.status_code = 200
-            patched_post.return_value = magick_response
-            magick_response.iter_content.return_value = "test result"
-            res = report.render_report(
-                self.env.user.ids, report.report_name, {})
-            self.assertEqual(('test result', 'pdf'), res)
+        self._render_patched()
         attachements = ir_attachment.search([(1, '=', 1)]) - attachements
         self.assertEqual(1, len(attachements.ids))
         self.assertEqual(self.env.user.name + '.txt', attachements.name)
@@ -181,6 +109,7 @@ class TestReportPy3o(TransactionCase):
         self.assertEqual(self.env.user.id, attachements.res_id)
         self.assertEqual('test result', b64decode(attachements.datas))
 
+    @tools.misc.mute_logger('odoo.addons.report_py3o.models.py3o_report')
     def test_report_template_configs(self):
         # the demo template is specified with a relative path in in the module
         # path
@@ -192,7 +121,7 @@ class TestReportPy3o(TransactionCase):
         res = self.report.render_report(
             self.env.user.ids, self.report.report_name, {})
         self.assertTrue(res)
-        # The generation fails if the tempalte is not found
+        # The generation fails if the template is not found
         self.report.module = False
         with self.assertRaises(TemplateNotFound), self.env.cr.savepoint():
             self.report.render_report(
@@ -212,7 +141,7 @@ class TestReportPy3o(TransactionCase):
                 self.env.user.ids, self.report.report_name, {})
             self.assertTrue(res)
 
-        # the tempalte can also be provided as a binay field
+        # the tempalte can also be provided as a binary field
         self.report.py3o_template_fallback = False
 
         with open(flbk_filename) as tmpl_file:
@@ -227,6 +156,7 @@ class TestReportPy3o(TransactionCase):
             self.env.user.ids, self.report.report_name, {})
         self.assertTrue(res)
 
+    @tools.misc.mute_logger('odoo.addons.report_py3o.models.py3o_report')
     def test_report_template_fallback_validity(self):
         tmpl_name = self.report.py3o_template_fallback
         flbk_filename = pkg_resources.resource_filename(

report_py3o/views/ir_report.xml

@@ -16,8 +16,6 @@
                     <group name="py3o_params">
                         <field name="py3o_filetype" />
                         <field name="py3o_multi_in_one"/>
-                        <field name="py3o_is_local_fusion"/>
-                        <field name="py3o_server_id" />
                         <field name="py3o_template_id" />
                         <field name="module" />
                         <field name="py3o_template_fallback" />

report_py3o/views/py3o_template.xml

@@ -24,7 +24,7 @@
                 <group name="main">
                     <field name="name" />
                     <field name="filetype" />
-                    <field name="py3o_template_data" />
+                    <field name="py3o_template_data" filename="name" />
                 </group>
             </form>
         </field>

report_py3o_fusion_server/README.rst

@@ -0,0 +1,127 @@
+.. image:: https://img.shields.io/badge/licence-AGPL--3-blue.svg
+    :target: http://www.gnu.org/licenses/agpl-3.0-standalone.html
+    :alt: License: AGPL-3
+
+==========================================
+Py3o Report Engine - Fusion server support
+==========================================
+
+This module was written to let a py3o fusion server handle format conversion instead of local libreoffice.
+
+Installation
+============
+
+Install several additional components and Python libs:
+
+* `Py3o Fusion server <https://bitbucket.org/faide/py3o.fusion>`_,
+* `Py3o render server <https://bitbucket.org/faide/py3o.renderserver>`_,
+* a Java Runtime Environment (JRE), which can be OpenJDK,
+* Libreoffice started in the background in headless mode,
+* the Java driver for Libreoffice (Juno).
+
+It is also possible to use the Python driver for Libreoffice (PyUNO), but it is recommended to use the Java driver because it is more stable.
+
+The installation procedure below uses the Java driver. It has been successfully tested on Ubuntu 16.04 LTS ; if you use another OS, you may have to change a few details.
+
+Installation of py3o.fusion:
+
+.. code::
+
+  pip install py3o.fusion
+  pip install service-identity
+
+Installation of py3o.renderserver:
+
+.. code::
+
+  pip install py3o.renderserver
+
+Installation of Libreoffice and JRE on Debian/Ubuntu:
+
+.. code::
+
+  sudo apt-get install default-jre ure libreoffice-java-common libreoffice-writer
+
+You may have to install additionnal fonts. For example, to have the special unicode symbols for phone/fax/email in the PDF reports generated by Py3o, you should install the following package:
+
+.. code::
+
+  sudo apt-get install fonts-symbola
+
+At the end, with the dependencies, you should have the following py3o python libs:
+
+.. code::
+
+  % pip freeze | grep py3o
+  py3o.formats==0.3
+  py3o.fusion==0.8.7
+  py3o.renderclient==0.2
+  py3o.renderers.juno==0.8
+  py3o.renderserver==0.5.1
+  py3o.template==0.9.11
+  py3o.types==0.1.1
+
+Start the Py3o Fusion server:
+
+.. code::
+
+  start-py3o-fusion --debug -s localhost
+
+Start the Py3o render server:
+
+.. code::
+
+  start-py3o-renderserver --java=/usr/lib/jvm/default-java/jre/lib/amd64/server/libjvm.so --ure=/usr/share --office=/usr/lib/libreoffice --driver=juno --sofficeport=8997
+
+On the output of the Py3o render server, the first line looks like:
+
+.. code::
+
+  DEBUG:root:Starting JVM: /usr/lib/jvm/default-java/jre/lib/amd64/server/libjvm.so with options: -Djava.class.path=/usr/local/lib/python2.7/dist-packages/py3o/renderers/juno/py3oconverter.jar:/usr/share/java/juh.jar:/usr/share/java/jurt.jar:/usr/share/java/ridl.jar:/usr/share/java/unoloader.jar:/usr/share/java/java_uno.jar:/usr/lib/libreoffice/program/classes/unoil.jar -Xmx150M
+
+After **-Djava.class.path**, there is a list of Java libs with *.jar* extension ; check that each JAR file is really present on your filesystem. If one of the jar files is present in another directory, create a symlink that points to the real location of the file. If all the jar files are present on another directory, adapt the *--ure=* argument on the command line of Py3o render server.
+
+To check that the Py3o Fusion server is running fine, visit the URL http://<IP_address>:8765/form. On this web page, under the section *Target format*, make sure that you have a line *This server currently supports these formats: ods, odt, docx, doc, html, docbook, pdf, xls.*.
+
+Known issues / Roadmap
+======================
+
+* none yet
+
+Bug Tracker
+===========
+
+Bugs are tracked on `GitHub Issues
+<https://github.com/OCA/reporting-engine/issues>`_. In case of trouble, please
+check there if your issue has already been reported. If you spotted it first,
+help us smashing it by providing a detailed and welcomed feedback.
+
+Credits
+=======
+
+Contributors
+------------
+
+* Florent Aide (`XCG Consulting <http://odoo.consulting/>`_)
+* Laurent Mignon <laurent.mignon@acsone.eu>,
+* Alexis de Lattre <alexis.delattre@akretion.com>,
+* Guewen Baconnier <guewen.baconnier@camptocamp.com>
+* Omar Castiñeira <omar@comunitea.com>
+* Holger Brunn <hbrunn@therp.nl>
+
+Do not contact contributors directly about help with questions or problems concerning this addon, but use the `community mailing list <mailto:community@mail.odoo.com>`_ or the `appropriate specialized mailinglist <https://odoo-community.org/groups>`_ for help, and the bug tracker linked in `Bug Tracker`_ above for technical issues.
+
+Maintainer
+----------
+
+.. image:: https://odoo-community.org/logo.png
+   :alt: Odoo Community Association
+   :target: https://odoo-community.org
+
+This module is maintained by the OCA.
+
+OCA, or the Odoo Community Association, is a nonprofit organization whose
+mission is to support the collaborative development of Odoo features and
+promote its widespread use.
+
+To contribute to this module, please visit https://odoo-community.org.

report_py3o_fusion_server/__init__.py

@@ -0,0 +1,4 @@
+# -*- coding: utf-8 -*-
+# Copyright 2017 Therp BV <http://therp.nl>
+# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
+from . import models

report_py3o_fusion_server/__manifest__.py

@@ -0,0 +1,30 @@
+# -*- coding: utf-8 -*-
+# Copyright 2017 Therp BV <http://therp.nl>
+# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
+{
+    'name': 'Py3o Report Engine - Fusion server support',
+    'summary': 'Let the fusion server handle format conversion.',
+    'version': '10.0.1.0.0',
+    'category': 'Reporting',
+    'license': 'AGPL-3',
+    'author': 'XCG Consulting,'
+              'ACSONE SA/NV,'
+              'Odoo Community Association (OCA)',
+    'website': 'https://github.com/OCA/reporting-engine',
+    'depends': ['report_py3o'],
+    'external_dependencies': {
+        'python': [
+            'py3o.template',
+            'py3o.formats',
+        ],
+    },
+    'demo': [
+        "demo/report_py3o.xml",
+    ],
+    'data': [
+        "views/ir_report.xml",
+        'security/ir.model.access.csv',
+        'views/py3o_server.xml',
+    ],
+    'installable': True,
+}

report_py3o_fusion_server/demo/report_py3o.xml

@@ -0,0 +1,6 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<odoo>
+    <record id="report_py3o.res_users_report_py3o" model="ir.actions.report.xml">
+      <field name="py3o_is_local_fusion" eval="1"/>
+    </record>
+</odoo>

report_py3o_fusion_server/models/__init__.py

@@ -0,0 +1,6 @@
+# -*- coding: utf-8 -*-
+# Copyright 2017 Therp BV <http://therp.nl>
+# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
+from . import ir_actions_report_xml
+from . import py3o_report
+from . import py3o_server

report_py3o_fusion_server/models/ir_actions_report_xml.py

@@ -0,0 +1,41 @@
+# -*- coding: utf-8 -*-
+# © 2013 XCG Consulting <http://odoo.consulting>
+# © 2017 Therp BV <http://therp.nl>
+# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
+import logging
+from openerp import _, api, fields, models
+from odoo.exceptions import ValidationError
+
+logger = logging.getLogger(__name__)
+
+try:
+    from py3o.formats import Formats
+except ImportError:
+    logger.debug('Cannot import py3o.formats')
+
+
+class IrActionsReportXml(models.Model):
+    _inherit = 'ir.actions.report.xml'
+
+    @api.multi
+    @api.constrains("py3o_is_local_fusion", "py3o_server_id", "py3o_filetype")
+    def _check_py3o_server_id(self):
+        for report in self:
+            if report.report_type != "py3o":
+                continue
+            is_native = Formats().get_format(report.py3o_filetype).native
+            if ((not is_native or not report.py3o_is_local_fusion) and
+                    not report.py3o_server_id):
+                raise ValidationError(_(
+                    "Can not use not native format in local fusion. "
+                    "Please specify a Fusion Server"))
+
+    py3o_is_local_fusion = fields.Boolean(
+        "Local Fusion",
+        help="Native formats will be processed without a server. "
+             "You must use this mode if you call methods on your model into "
+             "the template.",
+        default=True)
+    py3o_server_id = fields.Many2one(
+        "py3o.server",
+        "Fusion Server")

report_py3o_fusion_server/models/py3o_report.py

@@ -0,0 +1,85 @@
+# -*- coding: utf-8 -*-
+# © 2013 XCG Consulting <http://odoo.consulting>
+# © 2016 ACSONE SA/NV
+# © 2017 Therp BV <http://therp.nl>
+# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
+import json
+import logging
+import os
+import requests
+import tempfile
+from contextlib import closing
+from openerp import _, api, models
+from openerp.exceptions import UserError
+from StringIO import StringIO
+
+logger = logging.getLogger(__name__)
+
+try:
+    from py3o.template import Template
+    from py3o.template.helpers import Py3oConvertor
+except ImportError:
+    logger.debug('Cannot import py3o.template')
+
+
+class Py3oReport(models.TransientModel):
+    _inherit = 'py3o.report'
+
+    @api.multi
+    def _create_single_report(self, model_instance, data, save_in_attachment):
+        """ This function to generate our py3o report
+        """
+        self.ensure_one()
+        report_xml = self.ir_actions_report_xml_id
+        filetype = report_xml.py3o_filetype
+        if report_xml.py3o_is_local_fusion:
+            result_path = super(Py3oReport, self)._create_single_report(
+                model_instance, data, save_in_attachment,
+            )
+            with closing(open(result_path, 'r')) as out_stream:
+                tmpl_data = out_stream.read()
+            datadict = {}
+        else:
+            result_fd, result_path = tempfile.mkstemp(
+                suffix='.' + filetype, prefix='p3o.report.tmp.')
+            tmpl_data = self.get_template(model_instance)
+
+            in_stream = StringIO(tmpl_data)
+            with closing(os.fdopen(result_fd, 'w+')) as out_stream:
+                template = Template(in_stream, out_stream, escape_false=True)
+                localcontext = self._get_parser_context(model_instance, data)
+                expressions = template.get_all_user_python_expression()
+                py_expression = template.convert_py3o_to_python_ast(
+                    expressions)
+                convertor = Py3oConvertor()
+                data_struct = convertor(py_expression)
+                datadict = data_struct.render(localcontext)
+
+        # Call py3o.server to render the template in the desired format
+        files = {
+            'tmpl_file': tmpl_data,
+        }
+        fields = {
+            "targetformat": filetype,
+            "datadict": json.dumps(datadict),
+            "image_mapping": "{}",
+            "escape_false": "on",
+        }
+        if report_xml.py3o_is_local_fusion:
+            fields['skipfusion'] = '1'
+        r = requests.post(
+            report_xml.py3o_server_id.url, data=fields, files=files)
+        if r.status_code != 200:
+            # server says we have an issue... let's tell that to enduser
+            raise UserError(
+                _('Fusion server error %s') % r.text,
+            )
+
+        chunk_size = 1024
+        with open(result_path, 'w+') as fd:
+            for chunk in r.iter_content(chunk_size):
+                fd.write(chunk)
+        if len(model_instance) == 1:
+            self._postprocess_report(
+                result_path, model_instance.id, save_in_attachment)
+        return result_path

report_py3o_fusion_server/security/ir.model.access.csv

@@ -0,0 +1,3 @@
+"id","name","model_id:id","group_id:id","perm_read","perm_write","perm_create","perm_unlink"
+access_py3o_server_admin,access_py3o_server_admin,model_py3o_server,base.group_no_one,1,1,1,1
+access_py3o_server_user,access_py3o_server_user,model_py3o_server,base.group_user,1,0,0,0

report_py3o_fusion_server/tests/__init__.py

@@ -0,0 +1,4 @@
+# -*- coding: utf-8 -*-
+# Copyright 2017 Therp BV <http://therp.nl>
+# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
+from . import test_report_py3o_fusion_server

report_py3o_fusion_server/tests/test_report_py3o_fusion_server.py

@@ -0,0 +1,38 @@
+# -*- coding: utf-8 -*-
+# Copyright 2017 Therp BV <http://therp.nl>
+# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
+import mock
+from odoo.exceptions import ValidationError
+from odoo.addons.report_py3o.tests import test_report_py3o
+
+
+@mock.patch(
+    'requests.post', mock.Mock(
+        return_value=mock.Mock(
+            status_code=200,
+            iter_content=mock.Mock(return_value=['test_result']),
+        )
+    )
+)
+class TestReportPy3oFusionServer(test_report_py3o.TestReportPy3o):
+    def setUp(self):
+        super(TestReportPy3oFusionServer, self).setUp()
+        py3o_server = self.env['py3o.server'].create({"url": "http://dummy"})
+        # check the call to the fusion server
+        self.report.write({
+            "py3o_server_id": py3o_server.id,
+            "py3o_filetype": 'pdf',
+        })
+
+    def test_no_local_fusion_without_fusion_server(self):
+        self.assertTrue(self.report.py3o_is_local_fusion)
+        with self.assertRaises(ValidationError) as e:
+            self.report.write({"py3o_server_id": None})
+        self.assertEqual(
+            e.exception.name,
+            "Can not use not native format in local fusion. "
+            "Please specify a Fusion Server")
+
+    def test_reports_no_local_fusion(self):
+        self.report.py3o_is_local_fusion = False
+        self.test_reports()

report_py3o_fusion_server/views/ir_report.xml

@@ -0,0 +1,13 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<odoo>
+    <record id="view_ir_actions_report_base" model="ir.ui.view">
+        <field name="model">ir.actions.report.xml</field>
+        <field name="inherit_id" ref="report_py3o.py3o_report_view" />
+        <field name="arch" type="xml">
+            <field name="py3o_multi_in_one" position="after">
+                <field name="py3o_is_local_fusion"/>
+                <field name="py3o_server_id" />
+            </field>
+        </field>
+    </record>
+</odoo>

report_py3o/views/py3o_server.xml → report_py3o_fusion_server/views/py3o_server.xml

@@ -32,7 +32,7 @@
     </record>
 
     <menuitem id="py3o_server_configuration_menu"
-        parent="py3o_config_menu"
+        parent="report_py3o.py3o_config_menu"
         action="py3o_server_configuration_action" />
 
 </odoo>