Browse Source

[RFR] split off the fusion server to its own module

use libreoffice for conversions in the base version. Fixes #179
pull/258/head
Holger Brunn 7 years ago
committed by Laurent Mignon (ACSONE)
parent
commit
ae65d78882
  1. 123
      report_py3o/README.rst
  2. 3
      report_py3o/__manifest__.py
  3. 2
      report_py3o/demo/report_py3o.xml
  4. 1
      report_py3o/models/__init__.py
  5. 22
      report_py3o/models/ir_actions_report_xml.py
  6. 82
      report_py3o/models/py3o_report.py
  7. 4
      report_py3o/models/py3o_template.py
  8. 2
      report_py3o/security/ir.model.access.csv
  9. BIN
      report_py3o/static/description/icon.png
  10. 110
      report_py3o/tests/test_report_py3o.py
  11. 2
      report_py3o/views/ir_report.xml
  12. 2
      report_py3o/views/py3o_template.xml
  13. 127
      report_py3o_fusion_server/README.rst
  14. 4
      report_py3o_fusion_server/__init__.py
  15. 30
      report_py3o_fusion_server/__manifest__.py
  16. 6
      report_py3o_fusion_server/demo/report_py3o.xml
  17. 6
      report_py3o_fusion_server/models/__init__.py
  18. 41
      report_py3o_fusion_server/models/ir_actions_report_xml.py
  19. 85
      report_py3o_fusion_server/models/py3o_report.py
  20. 0
      report_py3o_fusion_server/models/py3o_server.py
  21. 3
      report_py3o_fusion_server/security/ir.model.access.csv
  22. BIN
      report_py3o_fusion_server/static/description/icon.png
  23. 4
      report_py3o_fusion_server/tests/__init__.py
  24. 38
      report_py3o_fusion_server/tests/test_report_py3o_fusion_server.py
  25. 13
      report_py3o_fusion_server/views/ir_report.xml
  26. 2
      report_py3o_fusion_server/views/py3o_server.xml

123
report_py3o/README.rst

@ -12,99 +12,30 @@ The py3o reporting engine is a reporting engine for Odoo based on `Libreoffice <
* the report is stored on the server in OpenDocument format (.odt or .ods file) * the report is stored on the server in OpenDocument format (.odt or .ods file)
* the report is sent to the user in OpenDocument format or in any output format supported by Libreoffice (PDF, HTML, DOC, DOCX, Docbook, XLS, etc.) * the report is sent to the user in OpenDocument format or in any output format supported by Libreoffice (PDF, HTML, DOC, DOCX, Docbook, XLS, etc.)
The key advantages of a Libreoffice-based reporting engine are:
* no need to be a developper to create or modify a report: the report is created and modified with Libreoffice. So this reporting engine has a fully WYSIWYG report developpment tool!
* For a PDF report in A4/Letter format, it's easier to develop it with a tool such as Libreoffice that is designed to create A4/Letter documents than to develop it in HTML/CSS.
The key advantages of a Libreoffice based reporting engine are:
* no need to be a developer to create or modify a report: the report is created and modified with Libreoffice. So this reporting engine has a full WYSIWYG report development tool!
* For a PDF report in A4/Letter format, it's easier to develop it with a tool such as Libreoffice that is designed to create A4/Letter documents than to develop it in HTML/CSS, also some print peculiarities (backgrounds, margin boxes) are not very well supported by the HTML/CSS based solutions.
* If you want your users to be able to modify the document after its generation by Odoo, just configure the document with ODT output (or DOC or DOCX) and the user will be able to modify the document with Libreoffice (or Word) after its generation by Odoo. * If you want your users to be able to modify the document after its generation by Odoo, just configure the document with ODT output (or DOC or DOCX) and the user will be able to modify the document with Libreoffice (or Word) after its generation by Odoo.
* Easy development of spreadsheet reports in ODS format (XLS output possible). * Easy development of spreadsheet reports in ODS format (XLS output possible).
This reporting engine is an alternative to `Aeroo <https://github.com/aeroo/aeroo_reports>`_: these 2 reporting engines have similar features but their codes are completely different.
This reporting engine is an alternative to `Aeroo <https://github.com/aeroo/aeroo_reports>`_: these two reporting engines have similar features but their implementation is entirely different. You cannot use aeroo templates as drop in replacement though, you'll have to change a few details.
Installation Installation
============ ============
You must install 2 additionnal python libs:
Install the required python libs:
.. code:: .. code::
pip install py3o.template pip install py3o.template
pip install py3o.formats pip install py3o.formats
To allow the conversion of ODT or ODS reports to other formats (PDF, DOC, DOCX, etc.), you must install several additionnal components and Python libs:
* `Py3o Fusion server <https://bitbucket.org/faide/py3o.fusion>`_,
* `Py3o render server <https://bitbucket.org/faide/py3o.renderserver>`_,
* a Java Runtime Environment (JRE), which can be OpenJDK,
* Libreoffice started in the background in headless mode,
* the Java driver for Libreoffice (Juno).
It is also possible to use the Python driver for Libreoffice (PyUNO), but it is recommended to use the Java driver because it is more stable.
The installation procedure below uses the Java driver. It has been successfully tested on Ubuntu 16.04 LTS ; if you use another OS, you may have to change a few details.
Installation of py3o.fusion:
.. code::
pip install py3o.fusion
pip install service-identity
Installation of py3o.renderserver:
.. code::
pip install py3o.renderserver
Installation of Libreoffice and JRE on Debian/Ubuntu:
.. code::
sudo apt-get install default-jre ure libreoffice-java-common libreoffice-writer
You may have to install additionnal fonts. For example, to have the special unicode symbols for phone/fax/email in the PDF reports generated by Py3o, you should install the following package:
.. code::
sudo apt-get install fonts-symbola
At the end, with the dependencies, you should have the following py3o python libs:
.. code::
% pip freeze | grep py3o
py3o.formats==0.3
py3o.fusion==0.8.6
py3o.renderclient==0.2
py3o.renderers.juno==0.8
py3o.renderserver==0.5.1
py3o.template==0.9.11
py3o.types==0.1.1
Start the Py3o Fusion server:
.. code::
start-py3o-fusion --debug -s localhost
Start the Py3o render server:
.. code::
start-py3o-renderserver --java=/usr/lib/jvm/default-java/jre/lib/amd64/server/libjvm.so --ure=/usr/share --office=/usr/lib/libreoffice --driver=juno --sofficeport=8997
On the output of the Py3o render server, the first line looks like:
To allow the conversion of ODT or ODS reports to other formats (PDF, DOC, DOCX, etc.), install libreoffice:
.. code:: .. code::
DEBUG:root:Starting JVM: /usr/lib/jvm/default-java/jre/lib/amd64/server/libjvm.so with options: -Djava.class.path=/usr/local/lib/python2.7/dist-packages/py3o/renderers/juno/py3oconverter.jar:/usr/share/java/juh.jar:/usr/share/java/jurt.jar:/usr/share/java/ridl.jar:/usr/share/java/unoloader.jar:/usr/share/java/java_uno.jar:/usr/lib/libreoffice/program/classes/unoil.jar -Xmx150M
After **-Djava.class.path**, there is a list of Java libs with *.jar* extension ; check that each JAR file is really present on your filesystem. If one of the jar files is present in another directory, create a symlink that points to the real location of the file. If all the jar files are present on another directory, adapt the *--ure=* argument on the command line of Py3o render server.
To check that the Py3o Fusion server is running fine, visit the URL http://<IP_address>:8765/form. On this web page, under the section *Target format*, make sure that you have a line *This server currently supports these formats: ods, odt, docx, doc, html, docbook, pdf, xls.*.
apt-get --no-install-recommends install libreoffice
Configuration Configuration
============= =============
@ -139,7 +70,6 @@ the path to the template as *py3o_template_fallback*.
<record id="account.account_invoices" model="ir.actions.report.xml"> <record id="account.account_invoices" model="ir.actions.report.xml">
<field name="report_type">py3o</field> <field name="report_type">py3o</field>
<field name="py3o_filetype">odt</field> <field name="py3o_filetype">odt</field>
<field name="module">/field>
<field name="py3o_template_fallback">/odoo/templates/py3o/report/account_invoice.odt</field> <field name="py3o_template_fallback">/odoo/templates/py3o/report/account_invoice.odt</field>
</record> </record>
@ -164,14 +94,9 @@ If you want an invoice in PDF format instead of ODT format, the XML file should
<?xml version="1.0" encoding="utf-8"?> <?xml version="1.0" encoding="utf-8"?>
<odoo> <odoo>
<record id="local_py3o_server" model="py3o.server">
<field name="url">http://localhost:8765/form</field>
</record>
<record id="account.account_invoices" model="ir.actions.report.xml"> <record id="account.account_invoices" model="ir.actions.report.xml">
<field name="report_type">py3o</field> <field name="report_type">py3o</field>
<field name="py3o_filetype">pdf</field> <field name="py3o_filetype">pdf</field>
<field name="py3o_server_id" ref="local_py3o_server"/>
<field name="module">my_custom_module_base</field> <field name="module">my_custom_module_base</field>
<field name="py3o_template_fallback">report/account_invoice.odt</field> <field name="py3o_template_fallback">report/account_invoice.odt</field>
</record> </record>
@ -185,17 +110,12 @@ If you want to add a new py3o PDF report (and not replace a native report), the
<?xml version="1.0" encoding="utf-8"?> <?xml version="1.0" encoding="utf-8"?>
<odoo> <odoo>
<record id="local_py3o_server" model="py3o.server">
<field name="url">http://localhost:8765/form</field>
</record>
<record id="partner_summary_report" model="ir.actions.report.xml"> <record id="partner_summary_report" model="ir.actions.report.xml">
<field name="name">Partner Summary</field> <field name="name">Partner Summary</field>
<field name="model">res.partner</field> <field name="model">res.partner</field>
<field name="report_name">res.partner.summary</field> <field name="report_name">res.partner.summary</field>
<field name="report_type">py3o</field> <field name="report_type">py3o</field>
<field name="py3o_filetype">pdf</field> <field name="py3o_filetype">pdf</field>
<field name="py3o_server_id" ref="local_py3o_server"/>
<field name="module">my_custom_module_base</field> <field name="module">my_custom_module_base</field>
<field name="py3o_template_fallback">report/partner_summary.odt</field> <field name="py3o_template_fallback">report/partner_summary.odt</field>
</record> </record>
@ -210,6 +130,12 @@ If you want to add a new py3o PDF report (and not replace a native report), the
</odoo> </odoo>
Configuration parameters
------------------------
py3o.conversion_command
The command to be used to run the conversion, ``libreoffice`` by default. If you change this, whatever you set here must accept the parameters ``--headless --convert-to $ext $file`` and put the resulting file into ``$file``'s directory with extension ``$ext``. The command will be started in ``$file``'s directory.
Usage Usage
===== =====
@ -217,10 +143,32 @@ Usage
:alt: Try me on Runbot :alt: Try me on Runbot
:target: https://runbot.odoo-community.org/runbot/143/10.0 :target: https://runbot.odoo-community.org/runbot/143/10.0
The templating language is `extensively documented <http://py3otemplate.readthedocs.io/en/latest/templating.html>`_, the records are exposed in libreoffice as ``objects``, on which you can also call functions.
Available functions and objects
-------------------------------
user
Browse record of current user
lang
The user's company's language as string (ISO code)
b64decode
``base64.b64decode``
format_multiline_value(string)
Generate the ODF equivalent of ``<br/>`` and ``&nbsp;`` for multiline fields (ODF is XML internally, so those would be skipped otherwise)
html_sanitize(string)
Sanitize HTML string
time
Python's ``time`` module
display_address(partner)
Return a formatted string of the partner's address
Known issues / Roadmap Known issues / Roadmap
====================== ======================
* generate barcode ? * generate barcode ?
* add more detailed example in demo file to showcase features
* add migration guide aeroo -> py3o
Bug Tracker Bug Tracker
=========== ===========
@ -241,6 +189,7 @@ Contributors
* Alexis de Lattre <alexis.delattre@akretion.com>, * Alexis de Lattre <alexis.delattre@akretion.com>,
* Guewen Baconnier <guewen.baconnier@camptocamp.com> * Guewen Baconnier <guewen.baconnier@camptocamp.com>
* Omar Castiñeira <omar@comunitea.com> * Omar Castiñeira <omar@comunitea.com>
* Holger Brunn <hbrunn@therp.nl>
Maintainer Maintainer

3
report_py3o/__manifest__.py

@ -5,7 +5,7 @@
'name': 'Py3o Report Engine', 'name': 'Py3o Report Engine',
'summary': 'Reporting engine based on Libreoffice (ODT -> ODT, ' 'summary': 'Reporting engine based on Libreoffice (ODT -> ODT, '
'ODT -> PDF, ODT -> DOC, ODT -> DOCX, ODS -> ODS, etc.)', 'ODT -> PDF, ODT -> DOC, ODT -> DOCX, ODS -> ODS, etc.)',
'version': '10.0.1.2.0',
'version': '10.0.2.0.0',
'category': 'Reporting', 'category': 'Reporting',
'license': 'AGPL-3', 'license': 'AGPL-3',
'author': 'XCG Consulting,' 'author': 'XCG Consulting,'
@ -21,7 +21,6 @@
'security/ir.model.access.csv', 'security/ir.model.access.csv',
'views/menu.xml', 'views/menu.xml',
'views/py3o_template.xml', 'views/py3o_template.xml',
'views/py3o_server.xml',
'views/ir_report.xml', 'views/ir_report.xml',
'views/report_py3o.xml', 'views/report_py3o.xml',
'demo/report_py3o.xml', 'demo/report_py3o.xml',

2
report_py3o/demo/report_py3o.xml

@ -11,8 +11,6 @@
<field name="report_name">py3o_user_info</field> <field name="report_name">py3o_user_info</field>
<field name="report_type">py3o</field> <field name="report_type">py3o</field>
<field name="py3o_filetype">odt</field> <field name="py3o_filetype">odt</field>
<field name="py3o_is_local_fusion" eval="1"/>
<field name="py3o_filetype">odt</field>
<field name="module">report_py3o</field> <field name="module">report_py3o</field>
<field name="py3o_template_fallback">demo/res_user.odt</field> <field name="py3o_template_fallback">demo/res_user.odt</field>
</record> </record>

1
report_py3o/models/__init__.py

@ -1,5 +1,4 @@
from . import ir_actions_report_xml from . import ir_actions_report_xml
from . import py3o_template from . import py3o_template
from . import py3o_server
from . import report from . import report
from . import py3o_report from . import py3o_report

22
report_py3o/models/ir_actions_report_xml.py

@ -31,19 +31,6 @@ class IrActionsReportXml(models.Model):
raise ValidationError(_( raise ValidationError(_(
"Field 'Output Format' is required for Py3O report")) "Field 'Output Format' is required for Py3O report"))
@api.multi
@api.constrains("py3o_is_local_fusion", "py3o_server_id",
"py3o_filetype")
def _check_py3o_server_id(self):
for report in self:
if report.report_type == "py3o":
is_native = Formats().get_format(report.py3o_filetype).native
if ((not is_native or not report.py3o_is_local_fusion) and
not report.py3o_server_id):
raise ValidationError(_(
"Can not use not native format in local fusion. "
"Please specify a Fusion Server"))
@api.model @api.model
def _get_py3o_filetypes(self): def _get_py3o_filetypes(self):
formats = Formats() formats = Formats()
@ -62,15 +49,6 @@ class IrActionsReportXml(models.Model):
py3o_template_id = fields.Many2one( py3o_template_id = fields.Many2one(
'py3o.template', 'py3o.template',
"Template") "Template")
py3o_is_local_fusion = fields.Boolean(
"Local Fusion",
help="Native formats will be processed without a server. "
"You must use this mode if you call methods on your model into "
"the template.",
default=True)
py3o_server_id = fields.Many2one(
"py3o.server",
"Fusion Server")
module = fields.Char( module = fields.Char(
"Module", "Module",
help="The implementer module that provides this report") help="The implementer module that provides this report")

82
report_py3o/models/py3o_report.py

@ -5,26 +5,23 @@
import base64 import base64
from base64 import b64decode from base64 import b64decode
from cStringIO import StringIO from cStringIO import StringIO
import json
import logging import logging
import os import os
from contextlib import closing from contextlib import closing
import subprocess
import pkg_resources import pkg_resources
import requests
import sys import sys
import tempfile import tempfile
from zipfile import ZipFile, ZIP_DEFLATED from zipfile import ZipFile, ZIP_DEFLATED
from odoo.exceptions import AccessError from odoo.exceptions import AccessError
from odoo.exceptions import UserError
from odoo.report.report_sxw import rml_parse from odoo.report.report_sxw import rml_parse
from odoo import api, fields, models, tools, _ from odoo import api, fields, models, tools, _
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
try: try:
from py3o.template.helpers import Py3oConvertor
from py3o.template import Template from py3o.template import Template
from py3o import formats from py3o import formats
from genshi.core import Markup from genshi.core import Markup
@ -244,60 +241,63 @@ class Py3oReport(models.TransientModel):
""" This function to generate our py3o report """ This function to generate our py3o report
""" """
self.ensure_one() self.ensure_one()
report_xml = self.ir_actions_report_xml_id
filetype = report_xml.py3o_filetype
result_fd, result_path = tempfile.mkstemp( result_fd, result_path = tempfile.mkstemp(
suffix='.' + filetype, prefix='p3o.report.tmp.')
suffix='.ods', prefix='p3o.report.tmp.')
tmpl_data = self.get_template(model_instance) tmpl_data = self.get_template(model_instance)
in_stream = StringIO(tmpl_data) in_stream = StringIO(tmpl_data)
with closing(os.fdopen(result_fd, 'w+')) as out_stream: with closing(os.fdopen(result_fd, 'w+')) as out_stream:
template = Template(in_stream, out_stream, escape_false=True) template = Template(in_stream, out_stream, escape_false=True)
localcontext = self._get_parser_context(model_instance, data) localcontext = self._get_parser_context(model_instance, data)
is_native = Formats().get_format(filetype).native
if report_xml.py3o_is_local_fusion:
template.render(localcontext) template.render(localcontext)
out_stream.seek(0) out_stream.seek(0)
tmpl_data = out_stream.read() tmpl_data = out_stream.read()
datadict = {}
else:
expressions = template.get_all_user_python_expression()
py_expression = template.convert_py3o_to_python_ast(
expressions)
convertor = Py3oConvertor()
data_struct = convertor(py_expression)
datadict = data_struct.render(localcontext)
if not is_native or not report_xml.py3o_is_local_fusion:
# Call py3o.server to render the template in the desired format
files = {
'tmpl_file': tmpl_data,
}
fields = {
"targetformat": filetype,
"datadict": json.dumps(datadict),
"image_mapping": "{}",
"escape_false": "on",
}
if report_xml.py3o_is_local_fusion:
fields['skipfusion'] = '1'
r = requests.post(
report_xml.py3o_server_id.url, data=fields, files=files)
if r.status_code != 200:
# server says we have an issue... let's tell that to enduser
raise UserError(
_('Fusion server error %s') % r.text,
result_path = self._convert_single_report(
result_path, model_instance, data
) )
chunk_size = 1024
with open(result_path, 'w+') as fd:
for chunk in r.iter_content(chunk_size):
fd.write(chunk)
if len(model_instance) == 1: if len(model_instance) == 1:
self._postprocess_report( self._postprocess_report(
result_path, model_instance.id, save_in_attachment) result_path, model_instance.id, save_in_attachment)
return result_path
@api.multi
def _convert_single_report(self, result_path, model_instance, data):
"""Run a command to convert to our target format"""
filetype = self.ir_actions_report_xml_id.py3o_filetype
if not Formats().get_format(filetype).native:
command = self._convert_single_report_cmd(
result_path, model_instance, data,
)
logger.debug('Running command %s', command)
output = subprocess.check_output(
command, cwd=os.path.dirname(result_path),
)
logger.debug('Output was %s', output)
self._cleanup_tempfiles([result_path])
result_path, result_filename = os.path.split(result_path)
result_path = os.path.join(
result_path, '%s.%s' % (
os.path.splitext(result_filename)[0], filetype
)
)
return result_path return result_path
@api.multi
def _convert_single_report_cmd(self, result_path, model_instance, data):
"""Return a command list suitable for use in subprocess.call"""
return [
self.env['ir.config_parameter'].get_param(
'py3o.conversion_command', 'libreoffice',
),
'--headless',
'--convert-to',
self.ir_actions_report_xml_id.py3o_filetype,
result_path,
]
@api.multi @api.multi
def _get_or_create_single_report(self, model_instance, data, def _get_or_create_single_report(self, model_instance, data,
save_in_attachment): save_in_attachment):

4
report_py3o/models/py3o_template.py

@ -13,6 +13,10 @@ class Py3oTemplate(models.Model):
selection=[ selection=[
('odt', "ODF Text Document"), ('odt', "ODF Text Document"),
('ods', "ODF Spreadsheet"), ('ods', "ODF Spreadsheet"),
('odp', "ODF Presentation"),
('fodt', "ODF Text Document (Flat)"),
('fods', "ODF Spreadsheet (Flat)"),
('fodp', "ODF Presentation (Flat)"),
], ],
string="LibreOffice Template File Type", string="LibreOffice Template File Type",
required=True, required=True,

2
report_py3o/security/ir.model.access.csv

@ -1,5 +1,3 @@
id,name,model_id:id,group_id:id,perm_read,perm_write,perm_create,perm_unlink id,name,model_id:id,group_id:id,perm_read,perm_write,perm_create,perm_unlink
access_py3o_template_admin,access_py3o_template_admin,model_py3o_template,base.group_no_one,1,1,1,1 access_py3o_template_admin,access_py3o_template_admin,model_py3o_template,base.group_no_one,1,1,1,1
access_py3o_template_user,access_py3o_template_user,model_py3o_template,base.group_user,1,0,0,0 access_py3o_template_user,access_py3o_template_user,model_py3o_template,base.group_user,1,0,0,0
access_py3o_server_admin,access_py3o_server_admin,model_py3o_server,base.group_no_one,1,1,1,1
access_py3o_server_user,access_py3o_server_user,model_py3o_server,base.group_user,1,0,0,0

BIN
report_py3o/static/description/icon.png

After

Width: 128  |  Height: 128  |  Size: 9.2 KiB

110
report_py3o/tests/test_report_py3o.py

@ -11,8 +11,6 @@ import shutil
import tempfile import tempfile
from contextlib import contextmanager from contextlib import contextmanager
from py3o.formats import Formats
from odoo import tools from odoo import tools
from odoo.tests.common import TransactionCase from odoo.tests.common import TransactionCase
from odoo.exceptions import ValidationError from odoo.exceptions import ValidationError
@ -40,34 +38,6 @@ class TestReportPy3o(TransactionCase):
self.py3o_report = self.env['py3o.report'].create({ self.py3o_report = self.env['py3o.report'].create({
'ir_actions_report_xml_id': self.report.id}) 'ir_actions_report_xml_id': self.report.id})
def test_no_local_fusion_without_fusion_server(self):
self.assertTrue(self.report.py3o_is_local_fusion)
with self.assertRaises(ValidationError) as e:
self.report.py3o_is_local_fusion = False
self.assertEqual(
e.exception.name,
"Can not use not native format in local fusion. "
"Please specify a Fusion Server")
def test_no_native_format_without_fusion_server(self):
report = self.env.ref("report_py3o.res_users_report_py3o")
formats = Formats()
is_native = formats.get_format(report.py3o_filetype).native
self.assertTrue(is_native)
new_format = None
for name in formats.get_known_format_names():
format = formats.get_format(name)
if not format.native:
new_format = name
break
self.assertTrue(new_format)
with self.assertRaises(ValidationError) as e:
report.py3o_filetype = new_format
self.assertEqual(
e.exception.name,
"Can not use not native format in local fusion. "
"Please specify a Fusion Server")
def test_required_py3_filetype(self): def test_required_py3_filetype(self):
self.assertEqual(self.report.report_type, "py3o") self.assertEqual(self.report.report_type, "py3o")
with self.assertRaises(ValidationError) as e: with self.assertRaises(ValidationError) as e:
@ -76,70 +46,40 @@ class TestReportPy3o(TransactionCase):
e.exception.name, e.exception.name,
"Field 'Output Format' is required for Py3O report") "Field 'Output Format' is required for Py3O report")
def test_reports(self):
def _render_patched(self, result_text='test result', call_count=1):
py3o_report = self.env['py3o.report'] py3o_report = self.env['py3o.report']
with mock.patch.object( with mock.patch.object(
py3o_report.__class__, '_create_single_report') as patched_pdf: py3o_report.__class__, '_create_single_report') as patched_pdf:
result = tempfile.mktemp('.txt') result = tempfile.mktemp('.txt')
with open(result, 'w') as fp: with open(result, 'w') as fp:
fp.write('dummy')
fp.write(result_text)
patched_pdf.return_value = result patched_pdf.return_value = result
patched_pdf.side_effect = lambda record, data, save_attachments:\
py3o_report._postprocess_report(
result, record.id, save_attachments,
) or result
# test the call the the create method inside our custom parser # test the call the the create method inside our custom parser
self.report.render_report(self.env.user.ids, self.report.render_report(self.env.user.ids,
self.report.report_name, self.report.report_name,
{}) {})
self.assertEqual(1, patched_pdf.call_count)
self.assertEqual(call_count, patched_pdf.call_count)
# generated files no more exists # generated files no more exists
self.assertFalse(os.path.exists(result)) self.assertFalse(os.path.exists(result))
def test_reports(self):
res = self.report.render_report( res = self.report.render_report(
self.env.user.ids, self.report.report_name, {}) self.env.user.ids, self.report.report_name, {})
self.assertTrue(res) self.assertTrue(res)
py3o_server = self.env['py3o.server'].create({"url": "http://dummy"})
# check the call to the fusion server
self.report.write({"py3o_filetype": "pdf",
"py3o_server_id": py3o_server.id})
with mock.patch('requests.post') as patched_post:
magick_response = mock.MagicMock()
magick_response.status_code = 200
patched_post.return_value = magick_response
magick_response.iter_content.return_value = "test result"
self.report.py3o_filetype = 'pdf'
res = self.report.render_report( res = self.report.render_report(
self.env.user.ids, self.report.report_name, {}) self.env.user.ids, self.report.report_name, {})
self.assertEqual(('test result', 'pdf'), res)
self.assertTrue(res)
def test_report_load_from_attachment(self): def test_report_load_from_attachment(self):
py3o_report = self.env['py3o.report']
with mock.patch.object(
py3o_report.__class__, '_create_single_report') as patched_pdf:
result = tempfile.mktemp('.txt')
with open(result, 'w') as fp:
fp.write('dummy')
patched_pdf.return_value = result
# test the call the the create method inside our custom parser
self.report.render_report(self.env.user.ids,
self.report.report_name,
{})
self.assertEqual(1, patched_pdf.call_count)
# generated files no more exists
self.assertFalse(os.path.exists(result))
res = self.report.render_report(
self.env.user.ids, self.report.report_name, {})
self.assertTrue(res)
py3o_server = self.env['py3o.server'].create({"url": "http://dummy"})
# check the call to the fusion server
self.report.write({"py3o_filetype": "pdf",
"py3o_server_id": py3o_server.id,
"attachment_use": True,
self.report.write({"attachment_use": True,
"attachment": "'my_saved_report'"}) "attachment": "'my_saved_report'"})
attachments = self.env['ir.attachment'].search([]) attachments = self.env['ir.attachment'].search([])
with mock.patch('requests.post') as patched_post:
magick_response = mock.MagicMock()
magick_response.status_code = 200
patched_post.return_value = magick_response
magick_response.iter_content.return_value = "test result"
res = self.report.render_report(
self.env.user.ids, self.report.report_name, {})
self.assertEqual(('test result', 'pdf'), res)
self._render_patched()
new_attachments = self.env['ir.attachment'].search([]) new_attachments = self.env['ir.attachment'].search([])
created_attachement = new_attachments - attachments created_attachement = new_attachments - attachments
self.assertEqual(1, len(created_attachement)) self.assertEqual(1, len(created_attachement))
@ -151,29 +91,17 @@ class TestReportPy3o(TransactionCase):
created_attachement.datas = base64.encodestring("new content") created_attachement.datas = base64.encodestring("new content")
res = self.report.render_report( res = self.report.render_report(
self.env.user.ids, self.report.report_name, {}) self.env.user.ids, self.report.report_name, {})
self.assertEqual(('new content', 'pdf'), res)
self.assertEqual(('new content', self.report.py3o_filetype), res)
def test_report_post_process(self): def test_report_post_process(self):
""" """
By default the post_process method is in charge to save the By default the post_process method is in charge to save the
generated report into an ir.attachment if requested. generated report into an ir.attachment if requested.
""" """
report = self.env.ref("report_py3o.res_users_report_py3o")
report.attachment = "object.name + '.txt'"
py3o_server = self.env['py3o.server'].create({"url": "http://dummy"})
# check the call to the fusion server
report.write({"py3o_filetype": "pdf",
"py3o_server_id": py3o_server.id})
self.report.attachment = "object.name + '.txt'"
ir_attachment = self.env['ir.attachment'] ir_attachment = self.env['ir.attachment']
attachements = ir_attachment.search([(1, '=', 1)]) attachements = ir_attachment.search([(1, '=', 1)])
with mock.patch('requests.post') as patched_post:
magick_response = mock.MagicMock()
magick_response.status_code = 200
patched_post.return_value = magick_response
magick_response.iter_content.return_value = "test result"
res = report.render_report(
self.env.user.ids, report.report_name, {})
self.assertEqual(('test result', 'pdf'), res)
self._render_patched()
attachements = ir_attachment.search([(1, '=', 1)]) - attachements attachements = ir_attachment.search([(1, '=', 1)]) - attachements
self.assertEqual(1, len(attachements.ids)) self.assertEqual(1, len(attachements.ids))
self.assertEqual(self.env.user.name + '.txt', attachements.name) self.assertEqual(self.env.user.name + '.txt', attachements.name)
@ -181,6 +109,7 @@ class TestReportPy3o(TransactionCase):
self.assertEqual(self.env.user.id, attachements.res_id) self.assertEqual(self.env.user.id, attachements.res_id)
self.assertEqual('test result', b64decode(attachements.datas)) self.assertEqual('test result', b64decode(attachements.datas))
@tools.misc.mute_logger('odoo.addons.report_py3o.models.py3o_report')
def test_report_template_configs(self): def test_report_template_configs(self):
# the demo template is specified with a relative path in in the module # the demo template is specified with a relative path in in the module
# path # path
@ -192,7 +121,7 @@ class TestReportPy3o(TransactionCase):
res = self.report.render_report( res = self.report.render_report(
self.env.user.ids, self.report.report_name, {}) self.env.user.ids, self.report.report_name, {})
self.assertTrue(res) self.assertTrue(res)
# The generation fails if the tempalte is not found
# The generation fails if the template is not found
self.report.module = False self.report.module = False
with self.assertRaises(TemplateNotFound), self.env.cr.savepoint(): with self.assertRaises(TemplateNotFound), self.env.cr.savepoint():
self.report.render_report( self.report.render_report(
@ -212,7 +141,7 @@ class TestReportPy3o(TransactionCase):
self.env.user.ids, self.report.report_name, {}) self.env.user.ids, self.report.report_name, {})
self.assertTrue(res) self.assertTrue(res)
# the tempalte can also be provided as a binay field
# the tempalte can also be provided as a binary field
self.report.py3o_template_fallback = False self.report.py3o_template_fallback = False
with open(flbk_filename) as tmpl_file: with open(flbk_filename) as tmpl_file:
@ -227,6 +156,7 @@ class TestReportPy3o(TransactionCase):
self.env.user.ids, self.report.report_name, {}) self.env.user.ids, self.report.report_name, {})
self.assertTrue(res) self.assertTrue(res)
@tools.misc.mute_logger('odoo.addons.report_py3o.models.py3o_report')
def test_report_template_fallback_validity(self): def test_report_template_fallback_validity(self):
tmpl_name = self.report.py3o_template_fallback tmpl_name = self.report.py3o_template_fallback
flbk_filename = pkg_resources.resource_filename( flbk_filename = pkg_resources.resource_filename(

2
report_py3o/views/ir_report.xml

@ -16,8 +16,6 @@
<group name="py3o_params"> <group name="py3o_params">
<field name="py3o_filetype" /> <field name="py3o_filetype" />
<field name="py3o_multi_in_one"/> <field name="py3o_multi_in_one"/>
<field name="py3o_is_local_fusion"/>
<field name="py3o_server_id" />
<field name="py3o_template_id" /> <field name="py3o_template_id" />
<field name="module" /> <field name="module" />
<field name="py3o_template_fallback" /> <field name="py3o_template_fallback" />

2
report_py3o/views/py3o_template.xml

@ -24,7 +24,7 @@
<group name="main"> <group name="main">
<field name="name" /> <field name="name" />
<field name="filetype" /> <field name="filetype" />
<field name="py3o_template_data" />
<field name="py3o_template_data" filename="name" />
</group> </group>
</form> </form>
</field> </field>

127
report_py3o_fusion_server/README.rst

@ -0,0 +1,127 @@
.. image:: https://img.shields.io/badge/licence-AGPL--3-blue.svg
:target: http://www.gnu.org/licenses/agpl-3.0-standalone.html
:alt: License: AGPL-3
==========================================
Py3o Report Engine - Fusion server support
==========================================
This addons was written to let a fusion server handle format conversion instead of local libreoffice.
Installation
============
Install several additional components and Python libs:
* `Py3o Fusion server <https://bitbucket.org/faide/py3o.fusion>`_,
* `Py3o render server <https://bitbucket.org/faide/py3o.renderserver>`_,
* a Java Runtime Environment (JRE), which can be OpenJDK,
* Libreoffice started in the background in headless mode,
* the Java driver for Libreoffice (Juno).
It is also possible to use the Python driver for Libreoffice (PyUNO), but it is recommended to use the Java driver because it is more stable.
The installation procedure below uses the Java driver. It has been successfully tested on Ubuntu 16.04 LTS ; if you use another OS, you may have to change a few details.
Installation of py3o.fusion:
.. code::
pip install py3o.fusion
pip install service-identity
Installation of py3o.renderserver:
.. code::
pip install py3o.renderserver
Installation of Libreoffice and JRE on Debian/Ubuntu:
.. code::
sudo apt-get install default-jre ure libreoffice-java-common libreoffice-writer
You may have to install additionnal fonts. For example, to have the special unicode symbols for phone/fax/email in the PDF reports generated by Py3o, you should install the following package:
.. code::
sudo apt-get install fonts-symbola
At the end, with the dependencies, you should have the following py3o python libs:
.. code::
% pip freeze | grep py3o
py3o.formats==0.3
py3o.fusion==0.8.6
py3o.renderclient==0.2
py3o.renderers.juno==0.8
py3o.renderserver==0.5.1
py3o.template==0.9.11
py3o.types==0.1.1
Start the Py3o Fusion server:
.. code::
start-py3o-fusion --debug -s localhost
Start the Py3o render server:
.. code::
start-py3o-renderserver --java=/usr/lib/jvm/default-java/jre/lib/amd64/server/libjvm.so --ure=/usr/share --office=/usr/lib/libreoffice --driver=juno --sofficeport=8997
On the output of the Py3o render server, the first line looks like:
.. code::
DEBUG:root:Starting JVM: /usr/lib/jvm/default-java/jre/lib/amd64/server/libjvm.so with options: -Djava.class.path=/usr/local/lib/python2.7/dist-packages/py3o/renderers/juno/py3oconverter.jar:/usr/share/java/juh.jar:/usr/share/java/jurt.jar:/usr/share/java/ridl.jar:/usr/share/java/unoloader.jar:/usr/share/java/java_uno.jar:/usr/lib/libreoffice/program/classes/unoil.jar -Xmx150M
After **-Djava.class.path**, there is a list of Java libs with *.jar* extension ; check that each JAR file is really present on your filesystem. If one of the jar files is present in another directory, create a symlink that points to the real location of the file. If all the jar files are present on another directory, adapt the *--ure=* argument on the command line of Py3o render server.
To check that the Py3o Fusion server is running fine, visit the URL http://<IP_address>:8765/form. On this web page, under the section *Target format*, make sure that you have a line *This server currently supports these formats: ods, odt, docx, doc, html, docbook, pdf, xls.*.
Known issues / Roadmap
======================
* none yet
Bug Tracker
===========
Bugs are tracked on `GitHub Issues
<https://github.com/OCA/reporting-engine/issues>`_. In case of trouble, please
check there if your issue has already been reported. If you spotted it first,
help us smashing it by providing a detailed and welcomed feedback.
Credits
=======
Contributors
------------
* Florent Aide (`XCG Consulting <http://odoo.consulting/>`_)
* Laurent Mignon <laurent.mignon@acsone.eu>,
* Alexis de Lattre <alexis.delattre@akretion.com>,
* Guewen Baconnier <guewen.baconnier@camptocamp.com>
* Omar Castiñeira <omar@comunitea.com>
* Holger Brunn <hbrunn@therp.nl>
Do not contact contributors directly about help with questions or problems concerning this addon, but use the `community mailing list <mailto:community@mail.odoo.com>`_ or the `appropriate specialized mailinglist <https://odoo-community.org/groups>`_ for help, and the bug tracker linked in `Bug Tracker`_ above for technical issues.
Maintainer
----------
.. image:: https://odoo-community.org/logo.png
:alt: Odoo Community Association
:target: https://odoo-community.org
This module is maintained by the OCA.
OCA, or the Odoo Community Association, is a nonprofit organization whose
mission is to support the collaborative development of Odoo features and
promote its widespread use.
To contribute to this module, please visit https://odoo-community.org.

4
report_py3o_fusion_server/__init__.py

@ -0,0 +1,4 @@
# -*- coding: utf-8 -*-
# Copyright 2017 Therp BV <http://therp.nl>
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
from . import models

30
report_py3o_fusion_server/__manifest__.py

@ -0,0 +1,30 @@
# -*- coding: utf-8 -*-
# Copyright 2017 Therp BV <http://therp.nl>
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
{
'name': 'Py3o Report Engine - Fusion server support',
'summary': 'Let the fusion server handle format conversion.',
'version': '10.0.1.0.0',
'category': 'Reporting',
'license': 'AGPL-3',
'author': 'XCG Consulting,'
'ACSONE SA/NV,'
'Odoo Community Association (OCA)',
'website': 'https://github.com/OCA/reporting-engine',
'depends': ['report_py3o'],
'external_dependencies': {
'python': [
'py3o.template',
'py3o.formats',
],
},
'demo': [
"demo/report_py3o.xml",
],
'data': [
"views/ir_report.xml",
'security/ir.model.access.csv',
'views/py3o_server.xml',
],
'installable': True,
}

6
report_py3o_fusion_server/demo/report_py3o.xml

@ -0,0 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<odoo>
<record id="report_py3o.res_users_report_py3o" model="ir.actions.report.xml">
<field name="py3o_is_local_fusion" eval="1"/>
</record>
</odoo>

6
report_py3o_fusion_server/models/__init__.py

@ -0,0 +1,6 @@
# -*- coding: utf-8 -*-
# Copyright 2017 Therp BV <http://therp.nl>
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
from . import ir_actions_report_xml
from . import py3o_report
from . import py3o_server

41
report_py3o_fusion_server/models/ir_actions_report_xml.py

@ -0,0 +1,41 @@
# -*- coding: utf-8 -*-
# © 2013 XCG Consulting <http://odoo.consulting>
# © 2017 Therp BV <http://therp.nl>
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
import logging
from openerp import _, api, fields, models
from odoo.exceptions import ValidationError
logger = logging.getLogger(__name__)
try:
from py3o.formats import Formats
except ImportError:
logger.debug('Cannot import py3o.formats')
class IrActionsReportXml(models.Model):
_inherit = 'ir.actions.report.xml'
@api.multi
@api.constrains("py3o_is_local_fusion", "py3o_server_id", "py3o_filetype")
def _check_py3o_server_id(self):
for report in self:
if report.report_type != "py3o":
continue
is_native = Formats().get_format(report.py3o_filetype).native
if ((not is_native or not report.py3o_is_local_fusion) and
not report.py3o_server_id):
raise ValidationError(_(
"Can not use not native format in local fusion. "
"Please specify a Fusion Server"))
py3o_is_local_fusion = fields.Boolean(
"Local Fusion",
help="Native formats will be processed without a server. "
"You must use this mode if you call methods on your model into "
"the template.",
default=True)
py3o_server_id = fields.Many2one(
"py3o.server",
"Fusion Server")

85
report_py3o_fusion_server/models/py3o_report.py

@ -0,0 +1,85 @@
# -*- coding: utf-8 -*-
# © 2013 XCG Consulting <http://odoo.consulting>
# © 2016 ACSONE SA/NV
# © 2017 Therp BV <http://therp.nl>
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
import json
import logging
import os
import requests
import tempfile
from contextlib import closing
from openerp import _, api, models
from openerp.exceptions import UserError
from StringIO import StringIO
logger = logging.getLogger(__name__)
try:
from py3o.template import Template
from py3o.template.helpers import Py3oConvertor
except ImportError:
logger.debug('Cannot import py3o.template')
class Py3oReport(models.TransientModel):
_inherit = 'py3o.report'
@api.multi
def _create_single_report(self, model_instance, data, save_in_attachment):
""" This function to generate our py3o report
"""
self.ensure_one()
report_xml = self.ir_actions_report_xml_id
filetype = report_xml.py3o_filetype
if report_xml.py3o_is_local_fusion:
result_path = super(Py3oReport, self)._create_single_report(
model_instance, data, save_in_attachment,
)
with closing(open(result_path, 'r')) as out_stream:
tmpl_data = out_stream.read()
datadict = {}
else:
result_fd, result_path = tempfile.mkstemp(
suffix='.' + filetype, prefix='p3o.report.tmp.')
tmpl_data = self.get_template(model_instance)
in_stream = StringIO(tmpl_data)
with closing(os.fdopen(result_fd, 'w+')) as out_stream:
template = Template(in_stream, out_stream, escape_false=True)
localcontext = self._get_parser_context(model_instance, data)
expressions = template.get_all_user_python_expression()
py_expression = template.convert_py3o_to_python_ast(
expressions)
convertor = Py3oConvertor()
data_struct = convertor(py_expression)
datadict = data_struct.render(localcontext)
# Call py3o.server to render the template in the desired format
files = {
'tmpl_file': tmpl_data,
}
fields = {
"targetformat": filetype,
"datadict": json.dumps(datadict),
"image_mapping": "{}",
"escape_false": "on",
}
if report_xml.py3o_is_local_fusion:
fields['skipfusion'] = '1'
r = requests.post(
report_xml.py3o_server_id.url, data=fields, files=files)
if r.status_code != 200:
# server says we have an issue... let's tell that to enduser
raise UserError(
_('Fusion server error %s') % r.text,
)
chunk_size = 1024
with open(result_path, 'w+') as fd:
for chunk in r.iter_content(chunk_size):
fd.write(chunk)
if len(model_instance) == 1:
self._postprocess_report(
result_path, model_instance.id, save_in_attachment)
return result_path

0
report_py3o/models/py3o_server.py → report_py3o_fusion_server/models/py3o_server.py

3
report_py3o_fusion_server/security/ir.model.access.csv

@ -0,0 +1,3 @@
"id","name","model_id:id","group_id:id","perm_read","perm_write","perm_create","perm_unlink"
access_py3o_server_admin,access_py3o_server_admin,model_py3o_server,base.group_no_one,1,1,1,1
access_py3o_server_user,access_py3o_server_user,model_py3o_server,base.group_user,1,0,0,0

BIN
report_py3o_fusion_server/static/description/icon.png

After

Width: 128  |  Height: 128  |  Size: 9.2 KiB

4
report_py3o_fusion_server/tests/__init__.py

@ -0,0 +1,4 @@
# -*- coding: utf-8 -*-
# Copyright 2017 Therp BV <http://therp.nl>
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
from . import test_report_py3o_fusion_server

38
report_py3o_fusion_server/tests/test_report_py3o_fusion_server.py

@ -0,0 +1,38 @@
# -*- coding: utf-8 -*-
# Copyright 2017 Therp BV <http://therp.nl>
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
import mock
from odoo.exceptions import ValidationError
from odoo.addons.report_py3o.tests import test_report_py3o
@mock.patch(
'requests.post', mock.Mock(
return_value=mock.Mock(
status_code=200,
iter_content=mock.Mock(return_value=['test_result']),
)
)
)
class TestReportPy3oFusionServer(test_report_py3o.TestReportPy3o):
def setUp(self):
super(TestReportPy3oFusionServer, self).setUp()
py3o_server = self.env['py3o.server'].create({"url": "http://dummy"})
# check the call to the fusion server
self.report.write({
"py3o_server_id": py3o_server.id,
"py3o_filetype": 'pdf',
})
def test_no_local_fusion_without_fusion_server(self):
self.assertTrue(self.report.py3o_is_local_fusion)
with self.assertRaises(ValidationError) as e:
self.report.write({"py3o_server_id": None})
self.assertEqual(
e.exception.name,
"Can not use not native format in local fusion. "
"Please specify a Fusion Server")
def test_reports_no_local_fusion(self):
self.report.py3o_is_local_fusion = False
self.test_reports()

13
report_py3o_fusion_server/views/ir_report.xml

@ -0,0 +1,13 @@
<?xml version="1.0" encoding="UTF-8"?>
<odoo>
<record id="view_ir_actions_report_base" model="ir.ui.view">
<field name="model">ir.actions.report.xml</field>
<field name="inherit_id" ref="report_py3o.py3o_report_view" />
<field name="arch" type="xml">
<field name="py3o_multi_in_one" position="after">
<field name="py3o_is_local_fusion"/>
<field name="py3o_server_id" />
</field>
</field>
</record>
</odoo>

2
report_py3o/views/py3o_server.xml → report_py3o_fusion_server/views/py3o_server.xml

@ -32,7 +32,7 @@
</record> </record>
<menuitem id="py3o_server_configuration_menu" <menuitem id="py3o_server_configuration_menu"
parent="py3o_config_menu"
parent="report_py3o.py3o_config_menu"
action="py3o_server_configuration_action" /> action="py3o_server_configuration_action" />
</odoo> </odoo>
Loading…
Cancel
Save