OCA
/
server-tools
5
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
Browse Source

Port external_file_location to v9 

Add company_id on external location and external task
pull/516/head
Florian da Costa 8 years ago
parent
adbb878f83
commit
c434e06ac5
26 changed files with 1364 additions and 747 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 47
      external_file_location/README.rst
  2. 12
      external_file_location/__openerp__.py
  3. 4
      external_file_location/data/cron.xml
  4. 99
      external_file_location/demo/task_demo.xml
  5. 347
      external_file_location/i18n/external_file_location.pot
  6. 347
      external_file_location/i18n/fr.po
  7. 30
      external_file_location/models/attachment.py
  8. 84
      external_file_location/models/helper.py
  9. 71
      external_file_location/models/location.py
  10. 235
      external_file_location/models/task.py
  11. 1
      external_file_location/tasks/__init__.py
  12. 196
      external_file_location/tasks/abstract_fs.py
  13. 28
      external_file_location/tasks/abstract_task.py
  14. 50
      external_file_location/tasks/filestore.py
  15. 55
      external_file_location/tasks/ftp.py
  16. 55
      external_file_location/tasks/sftp.py
  17. 2
      external_file_location/tests/__init__.py
  18. 32
      external_file_location/tests/common.py
  19. 19
      external_file_location/tests/mock_server.py
  20. 50
      external_file_location/tests/test_filestore.py
  21. 86
      external_file_location/tests/test_ftp.py
  22. 213
      external_file_location/tests/test_sftp.py
  23. 28
      external_file_location/views/attachment_view.xml
  24. 7
      external_file_location/views/location_view.xml
  25. 12
      external_file_location/views/task_view.xml
  26. 1
      requirements.txt

47
external_file_location/README.rst
View File

@ -1,17 +1,20 @@
.. image:: https://img.shields.io/badge/licence-AGPL--3-blue.svg
:alt: License
:target: http://www.gnu.org/licenses/agpl-3.0-standalone.html
:alt: License: AGPL-3
======================
External File Location
======================
This module was written to extend the functionality of ir.attachment to support remote communication and allow you to import/export file to a remote server
This module was written to extend the functionality of ir.attachment to support remote communication and allow you to import/export file to a remote server.
For now, FTP, SFTP and local filestore are handled by the module.
Installation
============
To install this module, you need to:
* fs python module
* fs python module at version 0.5.4 or under
* Paramiko python module
Usage
@ -23,38 +26,44 @@ To use this module, you need to:
* Create a task with your file info and remote communication method
* A cron task will trigger each task
For further information, please visit:
* https://www.odoo.com/forum/help-1
.. image:: https://odoo-community.org/website/image/ir.attachment/5784_f2813bd/datas
:alt: Try me on Runbot
:target: https://runbot.odoo-community.org/runbot/149/9.0
Known issues / Roadmap
======================
Bug Tracker
===========
Bugs are tracked on `GitHub Issues
<https://github.com/OCA/server-tools/issues>`_. In case of trouble, please
check there if your issue has already been reported. If you spotted it first,
help us smashing it by providing a detailed and welcomed feedback.
Credits
=======
* Joel Grand-Guillaume Camptocamp
* initOS <http://initos.com>
* Valentin CHEMIERE <valentin.chemiere@akretion.com>
* Mourad EL HADJ MIMOUNE <mourad.elhadj.mimoune@akretion.com>
Images
------
* Odoo Community Association: `Icon <https://github.com/OCA/maintainer-tools/blob/master/template/module/static/description/icon.svg>`_.
Contributors
------------
* Sebastien BEAU <sebastian.beau@akretion.com>
* David BEAL <david.beal@akretion.com>
* Valentin CHEMIERE <valentin.chemiere@akretion.com>
* Mourad EL HADJ MIMOUNE <mourad.elhadj.mimoune@akretion.com>
* Florian DA COSTA <florian.dacosta@akretion.com>
Maintainer
----------
.. image:: http://odoo-community.org/logo.png
:alt: Odoo Community Association
:target: http://odoo-community.org
.. image:: https://odoo-community.org/logo.png
:alt: Odoo Community Association
:target: https://odoo-community.org
This module is maintained by the OCA.
OCA, or the Odoo Community Association, is a nonprofit organization whose mission is to support the collaborative development of Odoo features and promote its widespread use.
OCA, or the Odoo Community Association, is a nonprofit organization whose
mission is to support the collaborative development of Odoo features and
promote its widespread use.
To contribute to this module, please visit http://odoo-community.org.
To contribute to this module, please visit https://odoo-community.org.

12
external_file_location/__openerp__.py
View File

@ -1,17 +1,17 @@
# coding: utf-8
# @ 2015 Valentin CHEMIERE @ Akretion
# @ 2016 florian DA COSTA @ Akretion
# © 2016 @author Mourad EL HADJ MIMOUNE <mourad.elhadj.mimoune@akretion.com>
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
{
'name': 'external_file_location',
'version': '8.0.1.0.0',
'name': 'External File Location',
'version': '9.0.1.0.0',
'author': 'Akretion,Odoo Community Association (OCA)',
'website': 'www.akretion.com',
'license': 'AGPL-3',
'category': 'Generic Modules',
'depends': [
'attachment_metadata',
'attachment_base_synchronize',
],
'external_dependencies': {
'python': [
@ -27,7 +27,9 @@
'data/cron.xml',
'security/ir.model.access.csv',
],
'demo': [
'demo/task_demo.xml',
],
'installable': True,
'application': False,
'images': [],
}

4
external_file_location/data/cron.xml
View File

@ -10,8 +10,8 @@
<field name="active">True</field>
<field name="doall" eval="False" />
<field name="model">external.file.task</field>
<field name="function">_run</field>
<field name="args">([])</field>
<field name="function">run_task_scheduler</field>
<field name="args">([[('method_type', '=', 'import')]])</field>
</record>
</data>

99
external_file_location/demo/task_demo.xml
View File

@ -0,0 +1,99 @@
<?xml version="1.0" encoding="UTF-8"?>
<openerp>
<data>
<record id="location_ftp" model="external.file.location">
<field name="name">TEST FTP</field>
<field name="protocol">ftp</field>
<field name="address">my-ftp-address</field>
<field name="login">my-ftp-user</field>
<field name="password">my-ftp-password</field>
<field name="port">21</field>
</record>
<record id="location_sftp" model="external.file.location">
<field name="name">TEST SFTP</field>
<field name="protocol">sftp</field>
<field name="address">my-sftp-address</field>
<field name="login">my-sftp-user</field>
<field name="password">my-sftp-password</field>
<field name="port">22</field>
</record>
<record id="location_filestore" model="external.file.location">
<field name="name">TEST File Store</field>
<field name="protocol">file_store</field>
<field name="filestore_rootpath">/</field>
</record>
<record id="ftp_import_task" model="external.file.task">
<field name="method_type">import</field>
<field name="location_id" eval="ref('external_file_location.location_ftp')"/>
<field name="filename">test-import-ftp.txt</field>
<field name="filepath">/home/user/test</field>
<field name="name">Import FTP Task</field>
</record>
<record id="ftp_export_task" model="external.file.task">
<field name="method_type">export</field>
<field name="location_id" eval="ref('external_file_location.location_ftp')"/>
<field name="filepath">/home/user/test</field>
<field name="name">Export FTP Task</field>
</record>
<record id="sftp_import_task" model="external.file.task">
<field name="method_type">import</field>
<field name="location_id" eval="ref('external_file_location.location_sftp')"/>
<field name="filename">test-import-sftp.txt</field>
<field name="filepath">/home/user/test</field>
<field name="name">Import SFTP Task</field>
</record>
<record id="sftp_export_task" model="external.file.task">
<field name="method_type">export</field>
<field name="location_id" eval="ref('external_file_location.location_sftp')"/>
<field name="filepath">/home/user/test</field>
<field name="name">Export SFTP Task</field>
</record>
<record id="filestore_import_task" model="external.file.task">
<field name="method_type">import</field>
<field name="location_id" eval="ref('external_file_location.location_filestore')"/>
<field name="filename">test-import-filestore.txt</field>
<field name="filepath">/home/user/test</field>
<field name="name">Import filestore Task</field>
</record>
<record id="filestore_export_task" model="external.file.task">
<field name="method_type">export</field>
<field name="location_id" eval="ref('external_file_location.location_filestore')"/>
<field name="filepath">/home/user/test</field>
<field name="name">Export filestore Task</field>
</record>
<record id="ir_attachment_export_file_sftp" model="ir.attachment.metadata">
<field name="name">Sftp text export file</field>
<field name="datas">dGVzdCBzZnRwIGZpbGUgZXhwb3J0</field>
<field name="datas_fname">sftp_test_export.txt</field>
<field name="task_id" eval="ref('external_file_location.sftp_export_task')"/>
<field name="file_type">export_external_location</field>
</record>
<record id="ir_attachment_export_file_ftp" model="ir.attachment.metadata">
<field name="name">ftp text export file</field>
<field name="datas">dGVzdCBmdHAgZmlsZSBleHBvcnQ=</field>
<field name="datas_fname">ftp_test_export.txt</field>
<field name="task_id" eval="ref('external_file_location.ftp_export_task')"/>
<field name="file_type">export_external_location</field>
</record>
<record id="ir_attachment_export_file_filestore" model="ir.attachment.metadata">
<field name="name">filestore text export file</field>
<field name="datas">dGVzdCBmaWxlc3RvcmUgZmlsZSBleHBvcnQ=</field>
<field name="datas_fname">filestore_test_export.txt</field>
<field name="task_id" eval="ref('external_file_location.filestore_export_task')"/>
<field name="file_type">export_external_location</field>
</record>
</data>
</openerp>

347
external_file_location/i18n/external_file_location.pot
View File

@ -0,0 +1,347 @@
# Translation of Odoo Server.
# This file contains the translation of the following modules:
# * external_file_location
#
msgid ""
msgstr ""
"Project-Id-Version: Odoo Server 9.0c\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2016-08-10 16:51+0000\n"
"PO-Revision-Date: 2016-08-10 16:51+0000\n"
"Last-Translator: <>\n"
"Language-Team: \n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: \n"
"Plural-Forms: \n"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_message_needaction
msgid "Action Needed"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_external_file_task_after_import
msgid "Action after import a file"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_address
msgid "Address"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_after_import
msgid "After import"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_attachment_ids
msgid "Attachment"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_external_file_task_md5_check
msgid "Control file integrity after import with a md5 file"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_create_uid
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_create_uid
msgid "Created by"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_create_date
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_create_date
msgid "Created on"
msgstr ""
#. module: external_file_location
#: model:ir.ui.view,arch_db:external_file_location.view_task_form
msgid "Data importation setting"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_ir_attachment_metadata_message_last_post
msgid "Date of the last message posted on the record."
msgstr ""
#. module: external_file_location
#: model:ir.model,name:external_file_location.model_external_file_location
msgid "Description"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_display_name
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_display_name
msgid "Display Name"
msgstr ""
#. module: external_file_location
#: selection:external.file.task,method_type:0
msgid "Export"
msgstr ""
#. module: external_file_location
#: model:ir.model,name:external_file_location.model_external_file_task
msgid "External file task"
msgstr ""
#. module: external_file_location
#: model:ir.ui.view,arch_db:external_file_location.view_location_tree
msgid "File Location"
msgstr ""
#. module: external_file_location
#: model:ir.actions.act_window,name:external_file_location.action_location
#: model:ir.ui.menu,name:external_file_location.menu_ir_location
msgid "File Locations"
msgstr ""
#. module: external_file_location
#: model:ir.ui.menu,name:external_file_location.menu_file_exchange
msgid "File exchange"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_external_file_task_filename
msgid "File name which is imported.You can use file pattern like *.txtto import all txt files"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_file_type
msgid "File type"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_filestore_rootpath
msgid "FileStore Root Path"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_filename
msgid "Filename"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_filepath
msgid "Filepath"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_message_follower_ids
msgid "Followers"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_message_channel_ids
msgid "Followers (Channels)"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_message_partner_ids
msgid "Followers (Partners)"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_hide_login
msgid "Hide login"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_hide_password
msgid "Hide password"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_hide_port
msgid "Hide port"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_id
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_id
msgid "ID"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_ir_attachment_metadata_message_unread
msgid "If checked new messages require your attention."
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_ir_attachment_metadata_message_needaction
msgid "If checked, new messages require your attention."
msgstr ""
#. module: external_file_location
#: selection:external.file.task,method_type:0
msgid "Import"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_external_file_task_move_path
msgid "Imported File will be moved to this path"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_external_file_task_new_name
msgid "Imported File will be renamed to this nameName can use mako template where obj is an ir_attachement. template exemple : ${obj.name}-${obj.create_date}.csv"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_message_is_follower
msgid "Is Follower"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_message_last_post
msgid "Last Message Date"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location___last_update
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task___last_update
msgid "Last Modified on"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_write_uid
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_write_uid
msgid "Last Updated by"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_write_date
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_write_date
msgid "Last Updated on"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_location_id
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_location_id
#: model:ir.ui.view,arch_db:external_file_location.view_location_form
msgid "Location"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_login
msgid "Login"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_md5_check
msgid "Md5 check"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_message_ids
msgid "Messages"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_method_type
msgid "Method type"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_move_path
msgid "Move path"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_name
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_name
#: model:ir.ui.view,arch_db:external_file_location.view_location_form
#: model:ir.ui.view,arch_db:external_file_location.view_task_form
msgid "Name"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_new_name
msgid "New name"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_message_needaction_counter
msgid "Number of Actions"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_ir_attachment_metadata_message_needaction_counter
msgid "Number of messages which requires an action"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_ir_attachment_metadata_message_unread_counter
msgid "Number of unread messages"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_password
msgid "Password"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_external_file_task_filepath
msgid "Path to imported/exported file"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_port
msgid "Port"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_protocol
msgid "Protocol"
msgstr ""
#. module: external_file_location
#: model:ir.ui.view,arch_db:external_file_location.view_task_form
msgid "Run"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_external_file_location_filestore_rootpath
msgid "Server's root path"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_task_id
msgid "Task"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_task_ids
msgid "Task ids"
msgstr ""
#. module: external_file_location
#: model:ir.ui.view,arch_db:external_file_location.view_location_form
#: model:ir.ui.view,arch_db:external_file_location.view_task_form
#: model:ir.ui.view,arch_db:external_file_location.view_task_tree
msgid "Tasks"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_external_file_task_file_type
msgid "The file type determines an import method to be used to parse and transform data before their import in ERP"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_message_unread
msgid "Unread Messages"
msgstr ""
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_message_unread_counter
msgid "Unread Messages Counter"
msgstr ""
#. module: external_file_location
#: model:ir.model,name:external_file_location.model_ir_attachment_metadata
msgid "ir.attachment.metadata"
msgstr ""

347
external_file_location/i18n/fr.po
View File

@ -0,0 +1,347 @@
# Translation of Odoo Server.
# This file contains the translation of the following modules:
# * external_file_location
#
msgid ""
msgstr ""
"Project-Id-Version: Odoo Server 9.0c\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2016-08-10 16:51+0000\n"
"PO-Revision-Date: 2016-08-10 16:51+0000\n"
"Last-Translator: <>\n"
"Language-Team: \n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: \n"
"Plural-Forms: \n"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_message_needaction
msgid "Action Needed"
msgstr "A besoin d'une action"
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_external_file_task_after_import
msgid "Action after import a file"
msgstr "Action après l'import du fichier"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_address
msgid "Address"
msgstr "Addresse"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_after_import
msgid "After import"
msgstr "Après import"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_attachment_ids
msgid "Attachment"
msgstr "Pièce jointe"
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_external_file_task_md5_check
msgid "Control file integrity after import with a md5 file"
msgstr "Contrôle l'intégrité du fichier après l'import avec un fichier md5"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_create_uid
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_create_uid
msgid "Created by"
msgstr "Créé par"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_create_date
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_create_date
msgid "Created on"
msgstr "Créé le"
#. module: external_file_location
#: model:ir.ui.view,arch_db:external_file_location.view_task_form
msgid "Data importation setting"
msgstr "Data importation setting"
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_ir_attachment_metadata_message_last_post
msgid "Date of the last message posted on the record."
msgstr "Date du dernier message publié sur cet enregistrement"
#. module: external_file_location
#: model:ir.model,name:external_file_location.model_external_file_location
msgid "Description"
msgstr "Description"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_display_name
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_display_name
msgid "Display Name"
msgstr "Afficher le nom"
#. module: external_file_location
#: selection:external.file.task,method_type:0
msgid "Export"
msgstr "Export"
#. module: external_file_location
#: model:ir.model,name:external_file_location.model_external_file_task
msgid "External file task"
msgstr "Tache"
#. module: external_file_location
#: model:ir.ui.view,arch_db:external_file_location.view_location_tree
msgid "File Location"
msgstr "Emplacement fichier"
#. module: external_file_location
#: model:ir.actions.act_window,name:external_file_location.action_location
#: model:ir.ui.menu,name:external_file_location.menu_ir_location
msgid "File Locations"
msgstr "Emplacements fichiers"
#. module: external_file_location
#: model:ir.ui.menu,name:external_file_location.menu_file_exchange
msgid "File exchange"
msgstr "Echange de fichier"
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_external_file_task_filename
msgid "File name which is imported.You can use file pattern like *.txtto import all txt files"
msgstr "Nom du fichier importé. Vous pouvez utiliser une expression comme *.txt pour importer tous les fichiers txt"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_file_type
msgid "File type"
msgstr "Type de fichier"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_filestore_rootpath
msgid "FileStore Root Path"
msgstr "Emplacement racine"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_filename
msgid "Filename"
msgstr "Nom du fichier"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_filepath
msgid "Filepath"
msgstr "Chemin"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_message_follower_ids
msgid "Followers"
msgstr "Abonnés"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_message_channel_ids
msgid "Followers (Channels)"
msgstr "Abonnés (Canaux)"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_message_partner_ids
msgid "Followers (Partners)"
msgstr "Abonnés (Partenaires)"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_hide_login
msgid "Hide login"
msgstr "Cacher le login"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_hide_password
msgid "Hide password"
msgstr "Cacher le mot de passe"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_hide_port
msgid "Hide port"
msgstr "Cacher le port"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_id
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_id
msgid "ID"
msgstr "Identifiant"
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_ir_attachment_metadata_message_unread
msgid "If checked new messages require your attention."
msgstr "Si coché, de nouveaux messages demandent votre attention."
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_ir_attachment_metadata_message_needaction
msgid "If checked, new messages require your attention."
msgstr "si elle est cochée, de nouveaux messages requièrent votre attention."
#. module: external_file_location
#: selection:external.file.task,method_type:0
msgid "Import"
msgstr "Import"
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_external_file_task_move_path
msgid "Imported File will be moved to this path"
msgstr "Le fichier importé sera déplacé dans cet emplacement"
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_external_file_task_new_name
msgid "Imported File will be renamed to this nameName can use mako template where obj is an ir_attachement. template exemple : ${obj.name}-${obj.create_date}.csv"
msgstr "Imported File will be renamed to this nameName can use mako template where obj is an ir_attachement. template exemple : ${obj.name}-${obj.create_date}.csv"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_message_is_follower
msgid "Is Follower"
msgstr "Est un abonné"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_message_last_post
msgid "Last Message Date"
msgstr "Date du dernier message"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location___last_update
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task___last_update
msgid "Last Modified on"
msgstr "Dernière modification le"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_write_uid
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_write_uid
msgid "Last Updated by"
msgstr "Dernière mise à jour par"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_write_date
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_write_date
msgid "Last Updated on"
msgstr "Dernière mise à jour le"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_location_id
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_location_id
#: model:ir.ui.view,arch_db:external_file_location.view_location_form
msgid "Location"
msgstr "Emplacement"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_login
msgid "Login"
msgstr "Identifiant"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_md5_check
msgid "Md5 check"
msgstr "Md5 check"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_message_ids
msgid "Messages"
msgstr "Messages"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_method_type
msgid "Method type"
msgstr "Type de méthode"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_move_path
msgid "Move path"
msgstr "chemin des archives"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_name
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_name
#: model:ir.ui.view,arch_db:external_file_location.view_location_form
#: model:ir.ui.view,arch_db:external_file_location.view_task_form
msgid "Name"
msgstr "Nom"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_task_new_name
msgid "New name"
msgstr "Nouveau nom"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_message_needaction_counter
msgid "Number of Actions"
msgstr "Nombre d'Actions"
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_ir_attachment_metadata_message_needaction_counter
msgid "Number of messages which requires an action"
msgstr "Nombre de messages demandant une action"
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_ir_attachment_metadata_message_unread_counter
msgid "Number of unread messages"
msgstr "Nombre de messages non lus"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_password
msgid "Password"
msgstr "Mot de passe"
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_external_file_task_filepath
msgid "Path to imported/exported file"
msgstr "Path to imported/exported file"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_port
msgid "Port"
msgstr "Port"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_protocol
msgid "Protocol"
msgstr "Protocole"
#. module: external_file_location
#: model:ir.ui.view,arch_db:external_file_location.view_task_form
msgid "Run"
msgstr "Run"
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_external_file_location_filestore_rootpath
msgid "Server's root path"
msgstr "Chemin racine du serveur"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_task_id
msgid "Task"
msgstr "Tache"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_external_file_location_task_ids
msgid "Task ids"
msgstr "Task ids"
#. module: external_file_location
#: model:ir.ui.view,arch_db:external_file_location.view_location_form
#: model:ir.ui.view,arch_db:external_file_location.view_task_form
#: model:ir.ui.view,arch_db:external_file_location.view_task_tree
msgid "Tasks"
msgstr "Taches"
#. module: external_file_location
#: model:ir.model.fields,help:external_file_location.field_external_file_task_file_type
msgid "The file type determines an import method to be used to parse and transform data before their import in ERP"
msgstr "Le type de fichier détermine la méthode d'import utilisée pour parser le fichier et transformer les données avant l'import dans l'ERP"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_message_unread
msgid "Unread Messages"
msgstr "Messages non lus"
#. module: external_file_location
#: model:ir.model.fields,field_description:external_file_location.field_ir_attachment_metadata_message_unread_counter
msgid "Unread Messages Counter"
msgstr "Compteur de messages non lus"
#. module: external_file_location
#: model:ir.model,name:external_file_location.model_ir_attachment_metadata
msgid "ir.attachment.metadata"
msgstr "ir.attachment.metadata"

30
external_file_location/models/attachment.py
View File

@ -1,21 +1,33 @@
# coding: utf-8
# @ 2015 Valentin CHEMIERE @ Akretion
# @ 2016 Florian DA COSTA @ Akretion
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
from openerp import models, fields
from openerp import models, fields, api
import base64
import os
class IrAttachmentMetadata(models.Model):
_inherit = 'ir.attachment.metadata'
sync_date = fields.Datetime()
state = fields.Selection([
('pending', 'Pending'),
('failed', 'Failed'),
('done', 'Done'),
], readonly=False, required=True, default='pending')
state_message = fields.Text()
task_id = fields.Many2one('external.file.task', string='Task')
location_id = fields.Many2one(
'external.file.location', string='Location',
related='task_id.location_id', store=True)
file_type = fields.Selection(
selection_add=[
('export_external_location',
'Export File (External location)')
])
@api.multi
def _run(self):
super(IrAttachmentMetadata, self)._run()
if self.file_type == 'export_external_location':
protocols = self.env['external.file.location']._get_classes()
location = self.location_id
cls = protocols.get(location.protocol)[1]
path = os.path.join(self.task_id.filepath, self.datas_fname)
with cls.connect(location) as conn:
datas = base64.decodestring(self.datas)
conn.setcontents(path, data=datas)

84
external_file_location/models/helper.py
View File

@ -1,84 +0,0 @@
# coding: utf-8
# Author: Joel Grand-Guillaume
# Copyright 2011-2012 Camptocamp SA
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
def itersubclasses(cls, _seen=None):
"""
itersubclasses(cls)
Generator over all subclasses of a given class, in depth first order.
>>> list(itersubclasses(int)) == [bool]
True
>>> class A(object): pass
>>> class B(A): pass
>>> class C(A): pass
>>> class D(B,C): pass
>>> class E(D): pass
>>>
>>> for cls in itersubclasses(A):
... print(cls.__name__)
B
D
E
C
>>> # get ALL (new-style) classes currently defined
>>> [cls.__name__ for cls in itersubclasses(object)] #doctest: +ELLIPSIS
['type', ...'tuple', ...]
"""
if not isinstance(cls, type):
raise TypeError('itersubclasses must be called with '
'new-style classes, not %.100r' % cls
)
if _seen is None:
_seen = set()
try:
subs = cls.__subclasses__()
except TypeError: # fails only when cls is type
subs = cls.__subclasses__(cls)
for sub in subs:
if sub not in _seen:
_seen.add(sub)
yield sub
for sub in itersubclasses(sub, _seen):
yield sub
def _get_erp_module_name(module_path):
# see this PR for v9 https://github.com/odoo/odoo/pull/11084
""" Extract the name of the Odoo module from the path of the
Python module.
Taken from Odoo server: ``openerp.models.MetaModel``
The (Odoo) module name can be in the ``openerp.addons`` namespace
or not. For instance module ``sale`` can be imported as
``openerp.addons.sale`` (the good way) or ``sale`` (for backward
compatibility).
"""
module_parts = module_path.split('.')
if len(module_parts) > 2 and module_parts[:2] == ['openerp', 'addons']:
module_name = module_parts[2]
else:
module_name = module_parts[0]
return module_name
def is_module_installed(env, module_name):
""" Check if an Odoo addon is installed.
:param module_name: name of the addon
"""
# the registry maintains a set of fully loaded modules so we can
# lookup for our module there
return module_name in env.registry._init_modules
def get_erp_module(cls_or_func):
""" For a top level function or class, returns the
name of the Odoo module where it lives.
So we will be able to filter them according to the modules
installation state.
"""
return _get_erp_module_name(cls_or_func.__module__)

71
external_file_location/models/location.py
View File

@ -3,17 +3,22 @@
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
from openerp import models, fields, api
from ..tasks.abstract_task import AbstractTask
from .helper import itersubclasses
from ..tasks.filestore import FileStoreTask
from ..tasks.ftp import FtpTask
from ..tasks.sftp import SftpTask
class Location(models.Model):
_name = 'external.file.location'
_description = 'Description'
_description = 'Location'
name = fields.Char(string='Name', required=True)
protocol = fields.Selection(selection='_get_protocol', required=True)
address = fields.Char(string='Address', required=True)
address = fields.Char(
string='Address')
filestore_rootpath = fields.Char(
string='FileStore Root Path',
help="Server's root path")
port = fields.Integer()
login = fields.Char()
password = fields.Char()
@ -21,31 +26,43 @@ class Location(models.Model):
hide_login = fields.Boolean()
hide_password = fields.Boolean()
hide_port = fields.Boolean()
company_id = fields.Many2one(
'res.company', 'Company',
default=lambda self: self.env['res.company']._company_default_get(
'external.file.location'))
@api.model
def _get_classes(self):
"surcharge this method to add new protocols"
return {
'ftp': ('FTP', FtpTask),
'sftp': ('SFTP', SftpTask),
'file_store': ('File Store', FileStoreTask),
}
@api.model
def _get_protocol(self):
res = []
for cls in itersubclasses(AbstractTask):
if not cls._synchronize_type:
cls_info = (cls._key, cls._name)
res.append(cls_info)
elif not cls._synchronize_type and cls._key and cls._name:
pass
return res
protocols = self._get_classes()
selection = []
for key, val in protocols.iteritems():
selection.append((key, val[0]))
return selection
@api.onchange('protocol')
def onchange_protocol(self):
for cls in itersubclasses(AbstractTask):
if cls._key == self.protocol:
self.port = cls._default_port
if cls._hide_login:
self.hide_login = True
else:
self.hide_login = False
if cls._hide_password:
self.hide_password = True
else:
self.hide_password = False
if cls._hide_port:
self.hide_port = True
else:
self.hide_port = False
protocols = self._get_classes()
if self.protocol:
cls = protocols.get(self.protocol)[1]
self.port = cls._default_port
if cls._hide_login:
self.hide_login = True
else:
self.hide_login = False
if cls._hide_password:
self.hide_password = True
else:
self.hide_password = False
if cls._hide_port:
self.hide_port = True
else:
self.hide_port = False

235
external_file_location/models/task.py
View File

@ -4,8 +4,46 @@
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
from openerp import models, fields, api
from .helper import itersubclasses, get_erp_module, is_module_installed
from ..tasks.abstract_task import AbstractTask
import openerp
from openerp import tools
from base64 import b64encode
import os
import datetime
import logging
_logger = logging.getLogger(__name__)
try:
# We use a jinja2 sandboxed environment to render mako templates.
# Note that the rendering does not cover all the mako syntax, in particular
# arbitrary Python statements are not accepted, and not all expressions are
# allowed: only "public" attributes (not starting with '_') of objects may
# be accessed.
# This is done on purpose: it prevents incidental or malicious execution of
# Python code that may break the security of the server.
from jinja2.sandbox import SandboxedEnvironment
mako_template_env = SandboxedEnvironment(
variable_start_string="${",
variable_end_string="}",
line_statement_prefix="%",
trim_blocks=True, # do not output newline after blocks
)
mako_template_env.globals.update({
'str': str,
'datetime': datetime,
'len': len,
'abs': abs,
'min': min,
'max': max,
'sum': sum,
'filter': filter,
'reduce': reduce,
'map': map,
'round': round,
})
except ImportError:
_logger.warning("jinja2 not available, templating features will not work!")
class Task(models.Model):
@ -13,34 +51,50 @@ class Task(models.Model):
_description = 'External file task'
name = fields.Char(required=True)
method = fields.Selection(selection='_get_method', required=True,
help='procotol and trasmitting info')
method_type = fields.Char()
method_type = fields.Selection(
[('import', 'Import'), ('export', 'Export')],
required=True)
filename = fields.Char(help='File name which is imported.'
'You can use file pattern like *.txt'
'to import all txt files')
filepath = fields.Char(help='Path to imported file')
filepath = fields.Char(help='Path to imported/exported file')
location_id = fields.Many2one('external.file.location', string='Location',
required=True)
attachment_ids = fields.One2many('ir.attachment.metadata', 'task_id',
string='Attachment')
move_path = fields.Char(string='Move path',
move_path = fields.Char(string='Move Path',
help='Imported File will be moved to this path')
new_name = fields.Char(string='New name',
new_name = fields.Char(string='New Name',
help='Imported File will be renamed to this name'
'Name can use mako template where obj is an '
'ir_attachement. template exemple : '
' ${obj.name}-${obj.create_date}.csv')
md5_check = fields.Boolean(help='Control file integrity after import with'
' a md5 file')
after_import = fields.Selection(selection='_get_action',
help='Action after import a file')
company_id = fields.Many2one(
'res.company', 'Company',
default=lambda self: self.env['res.company']._company_default_get(
'external.file.task'))
file_type = fields.Selection(
selection="_get_file_type",
string="File type",
selection=[],
string="File Type",
help="The file type determines an import method to be used "
"to parse and transform data before their import in ERP")
active = fields.Boolean(default=True)
def _get_action(self):
return [('rename', 'Rename'),
('move', 'Move'),
@ -48,68 +102,111 @@ class Task(models.Model):
('delete', 'Delete'),
]
def _get_file_type(self):
"""This is the method to be inherited for adding file types
The basic import do not apply any parsing or transform of the file.
The file is just added as an attachement
"""
return [('basic_import', 'Basic import')]
def _get_method(self):
res = []
for cls in itersubclasses(AbstractTask):
if not is_module_installed(self.env, get_erp_module(cls)):
continue
if cls._synchronize_type and (
'protocol' not in self._context or
cls._key == self._context['protocol']):
cls_info = (cls._key + '_' + cls._synchronize_type,
cls._name + ' ' + cls._synchronize_type)
res.append(cls_info)
return res
@api.onchange('method')
def onchange_method(self):
if self.method:
if 'import' in self.method:
self.method_type = 'import'
elif 'export' in self.method:
self.method_type = 'export'
@api.multi
def _prepare_attachment_vals(self, datas, filename, md5_datas):
self.ensure_one()
vals = {
'name': filename,
'datas': b64encode(datas),
'datas_fname': filename,
'task_id': self.id,
'external_hash': md5_datas,
'file_type': self.file_type or False,
}
return vals
@api.model
def _template_render(self, template, record):
try:
template = mako_template_env.from_string(tools.ustr(template))
except Exception:
_logger.exception("Failed to load template %r", template)
variables = {'obj': record}
try:
render_result = template.render(variables)
except Exception:
_logger.exception(
"Failed to render template %r using values %r" %
(template, variables))
render_result = u""
if render_result == u"False":
render_result = u""
return render_result
@api.model
def _run(self, domain=None):
if not domain:
def run_task_scheduler(self, domain=None):
if domain is None:
domain = []
tasks = self.env['external.file.task'].search(domain)
tasks.run()
for task in tasks:
if task.method_type == 'import':
task.run_import()
elif task.method_type == 'export':
task.run_export()
@api.multi
def run_import(self):
self.ensure_one()
protocols = self.env['external.file.location']._get_classes()
cls = protocols.get(self.location_id.protocol)[1]
attach_obj = self.env['ir.attachment.metadata']
with cls.connect(self.location_id) as conn:
md5_datas = ''
for file_name in conn.listdir(path=self.filepath,
wildcard=self.filename or '',
files_only=True):
with api.Environment.manage():
with openerp.registry(
self.env.cr.dbname).cursor() as new_cr:
new_env = api.Environment(new_cr, self.env.uid,
self.env.context)
try:
full_path = os.path.join(self.filepath, file_name)
file_data = conn.open(full_path, 'rb')
datas = file_data.read()
if self.md5_check:
md5_file = conn.open(full_path + '.md5', 'rb')
md5_datas = md5_file.read().rstrip('\r\n')
attach_vals = self._prepare_attachment_vals(
datas, file_name, md5_datas)
attachment = attach_obj.with_env(new_env).create(
attach_vals)
new_full_path = False
if self.after_import == 'rename':
new_name = self._template_render(
self.new_name, attachment)
new_full_path = os.path.join(
self.filepath, new_name)
elif self.after_import == 'move':
new_full_path = os.path.join(
self.move_path, file_name)
elif self.after_import == 'move_rename':
new_name = self._template_render(
self.new_name, attachment)
new_full_path = os.path.join(
self.move_path, new_name)
if new_full_path:
conn.rename(full_path, new_full_path)
if self.md5_check:
conn.rename(
full_path + '.md5',
new_full_path + '/md5')
if self.after_import == 'delete':
conn.remove(full_path)
if self.md5_check:
conn.remove(full_path + '.md5')
except Exception, e:
new_env.cr.rollback()
raise e
else:
new_env.cr.commit()
@api.multi
def run(self):
for tsk in self:
for cls in itersubclasses(AbstractTask):
if not is_module_installed(self.env, get_erp_module(cls)):
continue
cls_build = '%s_%s' % (cls._key, cls._synchronize_type)
if cls._synchronize_type and cls_build == tsk.method:
method_class = cls
config = {
'host': tsk.location_id.address,
# ftplib does not support unicode
'user': tsk.location_id.login and\
tsk.location_id.login.encode('utf-8'),
'pwd': tsk.location_id.password and \
tsk.location_id.password.encode('utf-8'),
'port': tsk.location_id.port,
'allow_dir_creation': False,
'file_name': tsk.filename,
'path': tsk.filepath,
'attachment_ids': tsk.attachment_ids,
'task': tsk,
'move_path': tsk.move_path,
'new_name': tsk.new_name,
'after_import': tsk.after_import,
'file_type': tsk.file_type,
'md5_check': tsk.md5_check,
}
conn = method_class(self.env, config)
conn.run()
def run_export(self):
self.ensure_one()
attachment_obj = self.env['ir.attachment.metadata']
attachments = attachment_obj.search(
[('task_id', '=', self.id), ('state', '!=', 'done')])
for attachment in attachments:
attachment.run()

1
external_file_location/tasks/__init__.py
View File

@ -1,4 +1,3 @@
from . import abstract_fs
from . import ftp
from . import sftp
from . import filestore

196
external_file_location/tasks/abstract_fs.py
View File

@ -1,196 +0,0 @@
# coding: utf-8
# Copyright (C) 2014 initOS GmbH & Co. KG (<http://www.initos.com>).
# @ 2015 Valentin CHEMIERE @ Akretion
# ©2016 @author Mourad EL HADJ MIMOUNE <mourad.elhadj.mimoune@akretion.com>
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
import logging
import os
import fnmatch
import datetime
from openerp import tools
from .abstract_task import AbstractTask
_logger = logging.getLogger(__name__)
try:
# We use a jinja2 sandboxed environment to render mako templates.
# Note that the rendering does not cover all the mako syntax, in particular
# arbitrary Python statements are not accepted, and not all expressions are
# allowed: only "public" attributes (not starting with '_') of objects may
# be accessed.
# This is done on purpose: it prevents incidental or malicious execution of
# Python code that may break the security of the server.
from jinja2.sandbox import SandboxedEnvironment
mako_template_env = SandboxedEnvironment(
variable_start_string="${",
variable_end_string="}",
line_statement_prefix="%",
trim_blocks=True, # do not output newline after blocks
)
mako_template_env.globals.update({
'str': str,
'datetime': datetime,
'len': len,
'abs': abs,
'min': min,
'max': max,
'sum': sum,
'filter': filter,
'reduce': reduce,
'map': map,
'round': round,
})
except ImportError:
_logger.warning("jinja2 not available, templating features will not work!")
class AbstractFSTask(AbstractTask):
_name = None
_key = None
_synchronize_type = None
_default_port = None
def __init__(self, env, config):
self.env = env
self.host = config.get('host', '')
self.user = config.get('user', '')
self.pwd = config.get('pwd', '')
self.port = config.get('port', '')
self.allow_dir_creation = config.get('allow_dir_creation', '')
self.file_name = config.get('file_name', '')
self.path = config.get('path') or '.'
self.move_path = config.get('move_path', '')
self.new_name = config.get('new_name', '')
self.after_import = config.get('after_import', False)
self.file_type = config.get('file_type', False)
self.attachment_ids = config.get('attachment_ids', False)
self.task = config.get('task', False)
self.ext_hash = False
self.md5_check = config.get('md5_check', False)
def _handle_new_source(self, fs_conn, download_directory, file_name,
move_directory):
"""open and read given file into create_file method,
move file if move_directory is given"""
with fs_conn.open(self._source_name(download_directory, file_name),
"rb") as fileobj:
data = fileobj.read()
return self.create_file(file_name, data)
def _source_name(self, download_directory, file_name):
"""helper to get the full name"""
return os.path.join(download_directory, file_name)
def _move_file(self, fs_conn, source, target):
"""Moves a file on the server"""
_logger.info('Moving file %s %s' % (source, target))
fs_conn.rename(source, target)
if self.md5_check:
fs_conn.rename(source + '.md5', target + '.md5')
def _delete_file(self, fs_conn, source):
"""Deletes a file from the server"""
_logger.info('Deleting file %s' % source)
fs_conn.remove(source)
if self.md5_check:
fs_conn.remove(source + '.md5')
def _get_hash(self, file_name, fs_conn):
hash_file_name = file_name + '.md5'
with fs_conn.open(hash_file_name, 'rb') as f:
return f.read().rstrip('\r\n')
def _get_files(self, conn, path):
process_files = []
files_list = conn.listdir(path)
pattern = self.file_name
for file_name in fnmatch.filter(files_list, pattern):
source_name = self._source_name(self.path, file_name)
process_files.append((file_name, source_name))
return process_files
def _template_render(self, template, record):
try:
template = mako_template_env.from_string(tools.ustr(template))
except Exception:
_logger.exception("Failed to load template %r", template)
variables = {'obj': record}
try:
render_result = template.render(variables)
except Exception:
_logger.exception(
"Failed to render template %r using values %r" %
(template, variables))
render_result = u""
if render_result == u"False":
render_result = u""
return render_result
def _process_file(self, conn, file_to_process):
if self.md5_check:
self.ext_hash = self._get_hash(file_to_process[1], conn)
att_id = self._handle_new_source(
conn,
self.path,
self.file_name,
self.move_path)
move = False
rename = False
if self.after_import:
move = 'move' in self.after_import
rename = 'rename' in self.after_import
# Move/rename/delete files only after all
# files have been processed.
if self.after_import == 'delete':
self._delete_file(conn, file_to_process[1])
elif rename or move:
new_name = file_to_process[0]
if rename and self.new_name:
new_name_render = self._template_render(
self.new_name, att_id)
if new_name_render:
# Avoid space in file name
new_name = new_name_render.replace(' ', '_')
if self.move_path and not conn.exists(self.move_path):
conn.makedir(self.move_path)
move_path = self.move_path if self.move_path else self.path
self._move_file(
conn,
file_to_process[1],
self._source_name(move_path, new_name))
return att_id
def _handle_existing_target(self, fs_conn, target_name, filedata):
raise Exception("%s already exists" % target_name)
def _handle_new_target(self, fs_conn, target_name, filedata):
try:
with fs_conn.open(target_name, mode='wb') as fileobj:
fileobj.write(filedata)
_logger.info('wrote %s, size %d', target_name, len(filedata))
self.attachment_id.state = 'done'
self.attachment_id.state_message = ''
except IOError:
self.attachment_id.state = 'failed'
self.attachment_id.state_message = (
'The directory doesn\'t exist or had insufficient rights')
def _target_name(self, fs_conn, upload_directory, filename):
return os.path.join(upload_directory, filename)
def _upload_file(self, conn, host, port, user, pwd,
path, filename, filedata):
upload_directory = path or '.'
target_name = self._target_name(conn,
upload_directory,
filename)
if conn.isfile(target_name):
self._handle_existing_target(conn, target_name, filedata)
else:
self._handle_new_target(conn, target_name, filedata)

28
external_file_location/tasks/abstract_task.py
View File

@ -1,28 +0,0 @@
# coding: utf-8
# @ 2015 Valentin CHEMIERE @ Akretion
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
from base64 import b64encode
class AbstractTask(object):
_name = None
_key = None
_synchronize_type = None
_default_port = None
_hide_login = False
_hide_password = False
_hide_port = False
def create_file(self, filename, data):
ir_attachment_id = self.env['ir.attachment.metadata'].create({
'name': filename,
'datas': b64encode(data),
'datas_fname': filename,
'task_id': self.task and self.task.id or False,
'location_id': self.task and self.task.location_id.id or False,
'external_hash': self.ext_hash,
'file_type': self.file_type,
})
return ir_attachment_id

50
external_file_location/tasks/filestore.py
View File

@ -1,53 +1,27 @@
# coding: utf-8
# Copyright (C) 2014 initOS GmbH & Co. KG (<http://www.initos.com>).
# @ 2015 Valentin CHEMIERE @ Akretion
# @ 2016 Florian DA COSTA @ Akretion
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
from .abstract_fs import AbstractFSTask
from base64 import b64decode
from fs import osfs
import logging
_logger = logging.getLogger(__name__)
try:
from fs import osfs
except ImportError:
_logger.debug('Cannot `import fs`.')
class FileStoreTask(AbstractFSTask):
class FileStoreTask(osfs.OSFS):
_key = 'filestore'
_name = 'File Store'
_synchronize_type = None
_default_port = None
_hide_login = True
_hide_password = True
_hide_port = True
class FileStoreImportTask(FileStoreTask):
_synchronize_type = 'import'
def run(self):
att_ids = []
with osfs.OSFS(self.host) as fs_conn:
files_to_process = self._get_files(fs_conn, self.path)
for file_to_process in files_to_process:
att_ids.append(self._process_file(fs_conn, file_to_process))
return att_ids
class FileStoreExportTask(FileStoreTask):
_synchronize_type = 'export'
def run(self, async=True):
for attachment in self.attachment_ids:
if attachment.state in ('pending', 'failed'):
self.attachment_id = attachment
with osfs.OSFS(self.host) as fs_conn:
self._upload_file(fs_conn,
self.host,
self.port,
self.user,
self.pwd,
self.path,
attachment.datas_fname,
b64decode(attachment.datas))
@staticmethod
def connect(location):
rootpath = location.filestore_rootpath or '/'
conn = FileStoreTask(rootpath)
return conn

55
external_file_location/tasks/ftp.py
View File

@ -1,51 +1,30 @@
# coding: utf-8
# Copyright (C) 2014 initOS GmbH & Co. KG (<http://www.initos.com>).
# @ 2015 Valentin CHEMIERE @ Akretion
# @ 2016 Florian DA COSTA @ Akretion
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
from .abstract_fs import AbstractFSTask
from base64 import b64decode
from fs import ftpfs
import logging
_logger = logging.getLogger(__name__)
try:
from fs import ftpfs
except ImportError:
_logger.debug('Cannot `import fs`.')
class FtpTask(AbstractFSTask):
_key = 'ftp'
_name = 'FTP'
class FtpTask(ftpfs.FTPFS):
_key = 'sftp'
_name = 'SFTP'
_synchronize_type = None
_default_port = 21
_default_port = 22
_hide_login = False
_hide_password = False
_hide_port = False
class FtpImportTask(FtpTask):
_synchronize_type = 'import'
def run(self):
att_ids = []
with ftpfs.FTPFS(
self.host, self.user, self.pwd, port=self.port) as ftp_conn:
files_to_process = self._get_files(ftp_conn, self.path)
for file_to_process in files_to_process:
att_ids.append(self._process_file(ftp_conn, file_to_process))
return att_ids
class FtpExportTask(FtpTask):
_synchronize_type = 'export'
def run(self, async=True):
for attachment in self.attachment_ids:
if attachment.state in ('pending', 'failed'):
self.attachment_id = attachment
with ftpfs.FTPFS(self.host, self.user, self.pwd,
port=self.port) as ftp_conn:
self._upload_file(ftp_conn, self.host, self.port,
self.user, self.pwd, self.path,
attachment.datas_fname,
b64decode(attachment.datas))
@staticmethod
def connect(location):
conn = FtpTask(location.address,
location.login,
location.password,
location.port)
return conn

55
external_file_location/tasks/sftp.py
View File

@ -1,16 +1,17 @@
# coding: utf-8
# Copyright (C) 2014 initOS GmbH & Co. KG (<http://www.initos.com>).
# @ 2015 Valentin CHEMIERE @ Akretion
# @ 2016 Florian DA COSTA @ Akretion
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
from .abstract_fs import AbstractFSTask
from base64 import b64decode
from fs import sftpfs
import logging
_logger = logging.getLogger(__name__)
try:
from fs import sftpfs
except ImportError:
_logger.debug('Cannot `import fs`.')
class SftpTask(AbstractFSTask):
class SftpTask(sftpfs.SFTPFS):
_key = 'sftp'
_name = 'SFTP'
@ -20,38 +21,10 @@ class SftpTask(AbstractFSTask):
_hide_password = False
_hide_port = False
class SftpImportTask(SftpTask):
_synchronize_type = 'import'
def run(self):
connection_string = "{}:{}".format(self.host, self.port)
root = "/"
att_ids = []
with sftpfs.SFTPFS(connection=connection_string,
root_path=root,
username=self.user,
password=self.pwd) as sftp_conn:
files_to_process = self._get_files(sftp_conn, self.path)
for file_to_process in files_to_process:
att_ids.append(self._process_file(sftp_conn, file_to_process))
return att_ids
class SftpExportTask(SftpTask):
_synchronize_type = 'export'
def run(self, async=True):
for attachment in self.attachment_ids:
if attachment.state in ('pending', 'failed'):
self.attachment_id = attachment
connection_string = "{}:{}".format(self.host, self.port)
with sftpfs.SFTPFS(connection=connection_string,
username=self.user,
password=self.pwd) as sftp_conn:
datas = b64decode(attachment.datas)
self._upload_file(sftp_conn, self.host, self.port,
self.user, self.pwd, self.path,
attachment.datas_fname, datas)
@staticmethod
def connect(location):
connection_string = "{}:{}".format(location.address, location.port)
conn = SftpTask(connection=connection_string,
username=location.login,
password=location.password)
return conn

2
external_file_location/tests/__init__.py
View File

@ -1,2 +1,4 @@
from . import mock_server
from . import test_ftp
from . import test_sftp
from . import test_filestore

32
external_file_location/tests/common.py
View File

@ -0,0 +1,32 @@
# coding: utf-8
# @ 2016 Florian da Costa @ Akretion
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
import openerp.tests.common as common
from openerp import api
from StringIO import StringIO
class ContextualStringIO(StringIO):
"""
snippet from http://bit.ly/1HfH6uW (stackoverflow)
"""
def __enter__(self):
return self
def __exit__(self, *args):
self.close()
return False
class TestConnection(common.TransactionCase):
def setUp(self):
super(TestConnection, self).setUp()
self.registry.enter_test_mode()
self.env = api.Environment(self.registry.test_cr, self.env.uid,
self.env.context)
def tearDown(self):
self.registry.leave_test_mode()
super(TestConnection, self).tearDown()

19
external_file_location/tests/mock_server.py
View File

@ -54,6 +54,21 @@ class ConnMock(object):
@contextmanager
def server_mock(response):
with mock.patch('fs.sftpfs.SFTPFS', ConnMock(response)) as SFTPFS:
def server_mock_sftp(response):
with mock.patch('openerp.addons.external_file_location.tasks.sftp.'
'SftpTask', ConnMock(response)) as SFTPFS:
yield SFTPFS._calls
@contextmanager
def server_mock_ftp(response):
with mock.patch('openerp.addons.external_file_location.tasks.ftp.'
'FtpTask', ConnMock(response)) as FTPFS:
yield FTPFS._calls
@contextmanager
def server_mock_filestore(response):
with mock.patch('openerp.addons.external_file_location.tasks.filestore.'
'FileStoreTask', ConnMock(response)) as FTPFS:
yield FTPFS._calls

50
external_file_location/tests/test_filestore.py
View File

@ -0,0 +1,50 @@
# coding: utf-8
# @ 2015 Valentin CHEMIERE @ Akretion
# ©2016 @author Mourad EL HADJ MIMOUNE <mourad.elhadj.mimoune@akretion.com>
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
import logging
from base64 import b64decode
from .common import TestConnection, ContextualStringIO
from .mock_server import server_mock_filestore
_logger = logging.getLogger(__name__)
class TestfilestoreConnection(TestConnection):
def setUp(self):
super(TestfilestoreConnection, self).setUp()
self.test_file_filestore = ContextualStringIO()
self.test_file_filestore.write('import filestore')
self.test_file_filestore.seek(0)
def test_00_filestore_import(self):
self.task = self.env.ref(
'external_file_location.filestore_import_task')
with server_mock_filestore(
{'open': self.test_file_filestore,
'listdir': ['test-import-filestore.txt']}):
self.task.run_import()
search_file = self.env['ir.attachment.metadata'].search(
[('name', '=', 'test-import-filestore.txt')])
self.assertEqual(len(search_file), 1)
self.assertEqual(b64decode(search_file[0].datas), 'import filestore')
def test_01_filestore_export(self):
self.task = self.env.ref(
'external_file_location.filestore_export_task')
self.filestore_attachment = self.env.ref(
'external_file_location.ir_attachment_export_file_filestore')
with server_mock_filestore(
{'setcontents': ''}) as Fakefilestore:
self.task.run_export()
if Fakefilestore:
self.assertEqual('setcontents', Fakefilestore[-1]['method'])
self.assertEqual('done', self.filestore_attachment.state)
self.assertEqual(
'/home/user/test/filestore_test_export.txt',
Fakefilestore[-1]['args'][0])
self.assertEqual(
'test filestore file export',
Fakefilestore[-1]['kwargs']['data'])

86
external_file_location/tests/test_ftp.py
View File

@ -0,0 +1,86 @@
# coding: utf-8
# @ 2015 Valentin CHEMIERE @ Akretion
# ©2016 @author Mourad EL HADJ MIMOUNE <mourad.elhadj.mimoune@akretion.com>
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
import logging
from base64 import b64decode
import hashlib
from .common import TestConnection, ContextualStringIO
from .mock_server import server_mock_ftp
from .mock_server import MultiResponse
from openerp.exceptions import UserError
_logger = logging.getLogger(__name__)
class TestFtpConnection(TestConnection):
def setUp(self):
super(TestFtpConnection, self).setUp()
self.test_file_ftp = ContextualStringIO()
self.test_file_ftp.write('import ftp')
self.test_file_ftp.seek(0)
def test_00_ftp_import(self):
self.task = self.env.ref('external_file_location.ftp_import_task')
with server_mock_ftp(
{'open': self.test_file_ftp,
'listdir': ['test-import-ftp.txt']}):
self.task.run_import()
search_file = self.env['ir.attachment.metadata'].search(
[('name', '=', 'test-import-ftp.txt')])
self.assertEqual(len(search_file), 1)
self.assertEqual(b64decode(search_file[0].datas), 'import ftp')
def test_01_ftp_export(self):
self.task = self.env.ref('external_file_location.ftp_export_task')
self.ftp_attachment = self.env.ref(
'external_file_location.ir_attachment_export_file_ftp')
with server_mock_ftp(
{'setcontents': ''}) as FakeFTP:
self.task.run_export()
if FakeFTP:
self.assertEqual('setcontents', FakeFTP[-1]['method'])
self.assertEqual('done', self.ftp_attachment.state)
self.assertEqual(
'/home/user/test/ftp_test_export.txt',
FakeFTP[-1]['args'][0])
self.assertEqual(
'test ftp file export',
FakeFTP[-1]['kwargs']['data'])
def test_02_ftp_import_md5(self):
md5_file = ContextualStringIO()
md5_file.write(hashlib.md5('import ftp').hexdigest())
md5_file.seek(0)
task = self.env.ref('external_file_location.ftp_import_task')
task.md5_check = True
with server_mock_ftp(
{'open': MultiResponse({
1: md5_file,
0: self.test_file_ftp}),
'listdir': [task.filename]}) as Fakeftp:
task.run_import()
search_file = self.env['ir.attachment.metadata'].search(
(('name', '=', task.filename),))
self.assertEqual(len(search_file), 1)
self.assertEqual(b64decode(search_file[0].datas),
'import ftp')
self.assertEqual('open', Fakeftp[-1]['method'])
self.assertEqual(hashlib.md5('import ftp').hexdigest(),
search_file.external_hash)
def test_03_ftp_import_md5_corrupt_file(self):
md5_file = ContextualStringIO()
md5_file.write(hashlib.md5('import test ftp corrupted').hexdigest())
md5_file.seek(0)
task = self.env.ref('external_file_location.ftp_import_task')
task.md5_check = True
with server_mock_ftp(
{'open': MultiResponse({
1: md5_file,
0: self.test_file_ftp}),
'listdir': [task.filename]}):
with self.assertRaises(UserError):
task.run_import()

213
external_file_location/tests/test_sftp.py
View File

@ -3,180 +3,83 @@
# ©2016 @author Mourad EL HADJ MIMOUNE <mourad.elhadj.mimoune@akretion.com>
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
import logging
from StringIO import StringIO
from base64 import b64decode
import hashlib
import openerp.tests.common as common
from ..tasks.sftp import SftpImportTask
from ..tasks.sftp import SftpExportTask
from .mock_server import (server_mock)
from .common import TestConnection, ContextualStringIO
from .mock_server import server_mock_sftp
from .mock_server import MultiResponse
from openerp.exceptions import UserError
_logger = logging.getLogger(__name__)
class ContextualStringIO(StringIO):
"""
snippet from http://bit.ly/1HfH6uW (stackoverflow)
"""
def __enter__(self):
return self
def __exit__(self, *args):
self.close()
return False
class TestSftpConnection(TestConnection):
class TestNewSource(common.TransactionCase):
def setUp(self):
super(TestNewSource, self).setUp()
self.test_file = ContextualStringIO()
self.test_file.write('import')
self.test_file.seek(0)
self.config = \
{'file_name': 'testfile',
'user': 'test',
'password': 'test',
'host': 'test',
'port': 22,
'attachment_ids': self.env['ir.attachment.metadata'].browse(False)
}
super(TestSftpConnection, self).setUp()
self.test_file_sftp = ContextualStringIO()
self.test_file_sftp.write('import sftp')
self.test_file_sftp.seek(0)
def test_00_sftp_import(self):
with server_mock(
{'exists': True,
'makedir': True,
'open': self.test_file,
'listdir': ['testfile']
}):
task = SftpImportTask(self.env, self.config)
task.run()
task = self.env.ref('external_file_location.sftp_import_task')
with server_mock_sftp(
{'open': self.test_file_sftp,
'listdir': [task.filename]}):
task.run_import()
search_file = self.env['ir.attachment.metadata'].search(
(('name', '=', 'testfile'),))
[('name', '=', task.filename)])
self.assertEqual(len(search_file), 1)
self.assertEqual(b64decode(search_file[0].datas), 'import')
self.assertEqual(b64decode(search_file[0].datas), 'import sftp')
def test_01_sftp_export(self):
with server_mock(
{'isfile': False,
'open': self.test_file,
}) as FakeSFTP:
task = SftpExportTask(self.env, self.config)
task.run()
self.task = self.env.ref('external_file_location.sftp_export_task')
self.sftp_attachment = self.env.ref(
'external_file_location.ir_attachment_export_file_sftp')
with server_mock_sftp(
{'setcontents': ''}) as FakeSFTP:
self.task.run_export()
if FakeSFTP:
self.assertEqual('open', FakeSFTP[-1]['method'])
def test_02_sftp_import_delete(self):
with server_mock(
{'exists': True,
'makedir': True,
'open': self.test_file,
'listdir': ['testfile'],
'remove': True
}) as FakeSFTP:
self.config.update({'after_import': 'delete'})
task = SftpImportTask(self.env, self.config)
task.run()
search_file = self.env['ir.attachment.metadata'].search(
(('name', '=', 'testfile'),))
self.assertEqual(len(search_file), 1)
self.assertEqual(b64decode(search_file[0].datas), 'import')
self.assertEqual('remove', FakeSFTP[-1]['method'])
self.assertEqual(
'./testfile', FakeSFTP[-1]['args'][0],
"Delete File must be './testfile'")
def test_03_sftp_import_move(self):
with server_mock(
{'exists': True,
'makedir': True,
'open': self.test_file,
'listdir': ['testfile'],
'rename': True
}) as FakeSFTP:
self.config.update({'after_import': 'move', 'move_path': '/home'})
task = SftpImportTask(self.env, self.config)
task.run()
search_file = self.env['ir.attachment.metadata'].search(
(('name', '=', 'testfile'),))
self.assertEqual(len(search_file), 1)
self.assertEqual(b64decode(search_file[0].datas), 'import')
self.assertEqual('rename', FakeSFTP[-1]['method'])
def test_04_sftp_import_rename(self):
with server_mock(
{'exists': True,
'makedir': True,
'open': self.test_file,
'listdir': ['testfile'],
'rename': True
}) as FakeSFTP:
_logger.info("Test sftp rename file")
self.config.update({
'after_import': 'rename',
'new_name': '${obj.name}.imported',
'path': '/home',
})
task = SftpImportTask(self.env, self.config)
task.run()
search_file = self.env['ir.attachment.metadata'].search(
(('name', '=', 'testfile'),))
self.assertEqual(len(search_file), 1)
self.assertEqual(b64decode(search_file[0].datas), 'import')
self.assertEqual('rename', FakeSFTP[2]['method'])
self.assertEqual('/home/testfile.imported',
FakeSFTP[2]['args'][1],
"File not renamed")
def test_05_sftp_import_move_rename(self):
with server_mock(
{'exists': True,
'makedir': True,
'open': self.test_file,
'listdir': ['testfile'],
'rename': True
}) as FakeSFTP:
_logger.info("Test sftp move and rename file")
self.config.update({
'after_import': 'rename',
'new_name': '${obj.name}.imported',
'path': '/home',
'move_path': '/home/processed',
})
task = SftpImportTask(self.env, self.config)
task.run()
search_file = self.env['ir.attachment.metadata'].search(
(('name', '=', 'testfile'),))
self.assertEqual(len(search_file), 1)
self.assertEqual(b64decode(search_file[0].datas), 'import')
self.assertEqual('rename', FakeSFTP[3]['method'])
self.assertEqual('/home/processed/testfile.imported',
FakeSFTP[3]['args'][1],
"File not renamed and moved")
def test_06_sftp_import_md5(self):
self.assertEqual('setcontents', FakeSFTP[-1]['method'])
self.assertEqual(
'/home/user/test/sftp_test_export.txt',
FakeSFTP[-1]['args'][0])
self.assertEqual(
'test sftp file export',
FakeSFTP[-1]['kwargs']['data'])
def test_02_sftp_import_md5(self):
md5_file = ContextualStringIO()
md5_file.write(hashlib.md5('import').hexdigest())
md5_file.write(hashlib.md5('import sftp').hexdigest())
md5_file.seek(0)
with server_mock(
{'exists': True,
'makedir': True,
'open': MultiResponse({
1: self.test_file,
0: md5_file
}),
'listdir': ['testfile', 'testfile.md5'],
}) as FakeSFTP:
self.config.update({'md5_check': True})
task = SftpImportTask(self.env, self.config)
task.run()
task = self.env.ref('external_file_location.sftp_import_task')
task.md5_check = True
with server_mock_sftp(
{'open': MultiResponse({
1: md5_file,
0: self.test_file_sftp}),
'listdir': [task.filename]}) as FakeSFTP:
task.run_import()
search_file = self.env['ir.attachment.metadata'].search(
(('name', '=', 'testfile'),))
(('name', '=', task.filename),))
self.assertEqual(len(search_file), 1)
self.assertEqual(b64decode(search_file[0].datas), 'import')
self.assertEqual(b64decode(search_file[0].datas),
'import sftp')
self.assertEqual('open', FakeSFTP[-1]['method'])
self.assertEqual('open', FakeSFTP[1]['method'])
self.assertEqual(('./testfile.md5', 'rb'), FakeSFTP[1]['args'])
self.assertEqual(hashlib.md5('import sftp').hexdigest(),
search_file.external_hash)
def test_03_sftp_import_md5_corrupt_file(self):
md5_file = ContextualStringIO()
md5_file.write(hashlib.md5('import test sftp corrupted').hexdigest())
md5_file.seek(0)
task = self.env.ref('external_file_location.sftp_import_task')
task.md5_check = True
with server_mock_sftp(
{'open': MultiResponse({
1: md5_file,
0: self.test_file_sftp}),
'listdir': [task.filename]}):
with self.assertRaises(UserError):
task.run_import()

28
external_file_location/views/attachment_view.xml
View File

@ -4,13 +4,10 @@
<record id="view_attachment_improved_form" model="ir.ui.view">
<field name="model">ir.attachment.metadata</field>
<field name="inherit_id" ref="attachment_metadata.view_attachment_improved_form" />
<field name="inherit_id" ref="attachment_base_synchronize.view_attachment_improved_form" />
<field name="arch" type="xml">
<field name="url" position="after">
<field name="sync_date"/>
<field name="state"/>
<field name="state_message"/>
<field name="task_id"/>
<field name="task_id" attrs="{'required': [('file_type', '=', 'export_external_location')]}"/>
<field name="location_id"/>
</field>
</field>
@ -18,31 +15,12 @@
<record id="view_external_attachment_tree" model="ir.ui.view">
<field name="model">ir.attachment.metadata</field>
<field name="inherit_id" ref="attachment_metadata.view_external_attachment_tree" />
<field name="inherit_id" ref="attachment_base_synchronize.view_external_attachment_tree" />
<field name="arch" type="xml">
<field name="file_type" position="after">
<field name="task_id"/>
<field name="location_id"/>
<field name="type"/>
<field name="create_date"/>
<field name="state"/>
</field>
</field>
</record>
<record id="view_external_attachment_search" model="ir.ui.view">
<field name="model">ir.attachment.metadata</field>
<field name="inherit_id" ref="attachment_metadata.view_external_attachment_search" />
<field name="arch" type="xml">
<field name="type" position="after">
<filter string="Pending" domain="[('state', '=', 'pending')]"/>
<filter string="Failed" domain="[('state', '=', 'failed')]"/>
<filter string="Done" domain="[('state', '=', 'done')]"/>
</field>
<filter string="Creation Month" position="after">
<filter string="State" domain="[]"
context="{'group_by': 'state'}"/>
</filter>
</field>
</record>

7
external_file_location/views/location_view.xml
View File

@ -15,8 +15,10 @@
<newline/>
<field name="protocol" colspan="2"/>
<newline/>
<field name="address" colspan="2"/>
<field name="address" colspan="2" attrs="{'required': [('protocol', '!=', 'file_store')], 'invisible': [('protocol', '=', 'file_store')]}"/>
<field name="filestore_rootpath" colspan="2" attrs="{'required': [('protocol', '=', 'file_store')], 'invisible': [('protocol', '!=', 'file_store')]}"/>
<field name="port" colspan="2" attrs="{'invisible': [('hide_port', '=', True)], 'required': [('hide_port', '=', False)]}"/>
<field name="company_id" colspan="2"/>
<field name="login" colspan="2" attrs="{'invisible': [('hide_login', '=', True)], 'required': [('hide_login', '=', False)]}"/>
<field name="password" password="1" colspan="2" attrs="{'invisible': [('hide_password', '=', True)]}"/>
<separator string="Tasks" colspan="4"/>
@ -24,10 +26,9 @@
<tree>
<field name="name"/>
<field name="name"/>
<field name="method"/>
<field name="method_type"/>
<field name="filename"/>
<field name="filepath"/>
<button name="run" type="object" string="Run" icon="gtk-execute"/>
</tree>
</field>
<field name="hide_login" invisible="1"/>

12
external_file_location/views/task_view.xml
View File

@ -10,7 +10,8 @@
<field name="arch" type="xml">
<form string="Tasks" version="7.0">
<header>
<button name="run" type="object" string="Run" icon="gtk-execute"/>
<button name="run_import" type="object" string="Run" icon="gtk-execute" attrs="{'invisible': [('method_type', '!=', 'import')]}"/>
<button name="run_export" type="object" string="Run" icon="gtk-execute" attrs="{'invisible': [('method_type', '!=', 'export')]}"/>
</header>
<sheet>
<field name="method_type" invisible="1"/>
@ -19,10 +20,11 @@
<label class="oe_edit_only" for="name" string="Name"/>
<h1><field name="name" class="oe_inline"/></h1>
</div>
<field name="method" colspan="2"/>
<field name="method_type" colspan="2"/>
<span colspan="2"/>
<field name="filename" colspan="4" attrs="{'invisible':[('method_type','!=','import')], 'required':[('method_type', '=', 'import')]}"/>
<field name="filepath" colspan="4" />
<field name="company_id" colspan="4"/>
</group>
<group col="6">
<field name="after_import" attrs="{'invisible':[('method_type','!=','import')]}"/>
@ -41,7 +43,8 @@
<field name="md5_check" colspan="2"
attrs="{'invisible':
[('method_type','!=','import')]}"
/>
/>
<field name="active" colspan="2"/>
</group>
<group string="Data importation setting">
<field name="file_type" attrs="{'invisible':[('method_type','!=','import')]}"/>
@ -56,10 +59,9 @@
<field name="arch" type="xml">
<tree string="Tasks" >
<field name="name" select="1"/>
<field name="method"/>
<field name="method_type"/>
<field name="filename"/>
<field name="filepath"/>
<button name="run" type="object" string="Run" icon="gtk-execute"/>
</tree>
</field>
</record>

1
requirements.txt
View File

@ -5,3 +5,4 @@ IPy
validate_email
pysftp
pyotp
fs==0.5.4
Write Preview
Loading…
Cancel
Save