Browse Source

[auto_backup] Refactor.

- Follow template README.
- Remove HTML README.
- Move models to models folder.
- Model and view file names follow guidelines.
- Unused methods cleanup.
- Remove unneeded `.pot` file.
- Fix permissons.
- Follow PEP8 in names everywhere.
- Set more descriptive field names.
- Disable backups for other databases, for security.
- Remove db name from generated file, for easier cleanup.
- EAFP logic everywhere.
- More descriptive name.
- Data files moved to YAML, with cleaner ir.cron record creation.
- Add permissions for db.backup model.
- Icons.
- Update tests with new format.
- Storage method is a selectable, for easier extensibility.
- Instead of custom mailing, it just has a mail thread where you can subscribe.
- Should fix almost all comments in https://github.com/OCA/server-tools/pull/203.
pull/620/head
Jairo Llopis 9 years ago
committed by Dave Lasley
parent
commit
0a7dd278e0
  1. 42
      auto_backup/README.rst
  2. 3
      auto_backup/__init__.py
  3. 14
      auto_backup/__openerp__.py
  4. 18
      auto_backup/data/backup_data.xml
  5. 28
      auto_backup/data/backup_data.yml
  6. 141
      auto_backup/i18n/auto_backup.pot
  7. 376
      auto_backup/model/backup_scheduler.py
  8. 2
      auto_backup/models/__init__.py
  9. 282
      auto_backup/models/db_backup.py
  10. 3
      auto_backup/security/ir.model.access.csv
  11. BIN
      auto_backup/static/description/icon.png
  12. 51
      auto_backup/static/description/icon.svg
  13. 100
      auto_backup/static/description/no_index.html
  14. 37
      auto_backup/tests/test_auto_backup.py
  15. 100
      auto_backup/view/bkp_conf_view.xml
  16. 94
      auto_backup/view/db_backup_view.xml

42
auto_backup/README.rst

@ -1,40 +1,40 @@
.. image:: https://img.shields.io/badge/licence-AGPL--3-blue.svg .. image:: https://img.shields.io/badge/licence-AGPL--3-blue.svg
:alt: License: AGPL-3
:target: http://www.gnu.org/licenses/agpl-3.0-standalone.html
:alt: License: AGPL-3
=================
Automated backups
=================
====================
Database Auto-Backup
====================
A tool for all your back-ups, internal and external! A tool for all your back-ups, internal and external!
Installation Installation
============ ============
Before to install this module, you need to:
Before installing this module, you need to execute::
instal pysftp via pip.
pip install pysftp
Configuration Configuration
============= =============
Go to Settings -> Configuration -> Configure Backup
Go to *Settings -> Configuration -> Configure Backup* to
create your configurations for each database that you needed create your configurations for each database that you needed
to backups. to backups.
Usage Usage
===== =====
Keep your Odoo data safe with this module. Take automated back-ups, Keep your Odoo data safe with this module. Take automated back-ups,
remove them automatically and even write them to an external server remove them automatically and even write them to an external server
through an encrypted tunnel. You can even specify how long local backups through an encrypted tunnel. You can even specify how long local backups
and external backups should be kept, automatically! and external backups should be kept, automatically!
Connect with an FTP Server Connect with an FTP Server
-------------------------- --------------------------
#### Keep your data safe, through an SSH tunnel!
Keep your data safe, through an SSH tunnel!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Want to go even further and write your backups to an external server? Want to go even further and write your backups to an external server?
You can with this module! Specify the credentials to the server, specify You can with this module! Specify the credentials to the server, specify
@ -45,7 +45,8 @@ safe!
Test connection Test connection
--------------- ---------------
#### Checks your credentials in one click
Checks your credentials in one click
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Want to make sure if the connection details are correct and if Odoo can Want to make sure if the connection details are correct and if Odoo can
automatically write them to the remote server? Simply click on the ‘Test automatically write them to the remote server? Simply click on the ‘Test
@ -55,16 +56,15 @@ everything is OK, or what is wrong!
E-mail on backup failure E-mail on backup failure
------------------------ ------------------------
#### Stay informed of problems, automatically!
Do you want to know if the database backup failed? Check the checkbox 'Auto. E-mail on backup fail' and fill in your e-mail.
Every time a backup fails you will get an e-mail in your mailbox with technical details.
Stay informed of problems, automatically!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Do you want to know if the database backup succeeded or failed? Subscribe to
the corresponding backup setting notification type.
Known issues / Roadmap
======================
* ...
.. image:: https://odoo-community.org/website/image/ir.attachment/5784_f2813bd/datas
:alt: Try me on Runbot
:target: https://runbot.odoo-community.org/runbot/149/8.0
Bug Tracker Bug Tracker
=========== ===========
@ -74,7 +74,6 @@ In case of trouble, please check there if your issue has already been reported.
If you spotted it first, help us smashing it by providing a detailed and welcomed feedback If you spotted it first, help us smashing it by providing a detailed and welcomed feedback
`here <https://github.com/OCA/server-tools/issues/new?body=module:%20auto_backup%0Aversion:%208.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**>`_. `here <https://github.com/OCA/server-tools/issues/new?body=module:%20auto_backup%0Aversion:%208.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**>`_.
Credits Credits
======= =======
@ -83,6 +82,7 @@ Contributors
* Yenthe Van Ginneken <yenthe.vanginneken@vanroey.be> * Yenthe Van Ginneken <yenthe.vanginneken@vanroey.be>
* Alessio Gerace <alessio.gerace@agilebg.com> * Alessio Gerace <alessio.gerace@agilebg.com>
* Jairo Llopis <yajo.sk8@gmail.com>
Maintainer Maintainer
---------- ----------
@ -97,4 +97,4 @@ OCA, or the Odoo Community Association, is a nonprofit organization whose
mission is to support the collaborative development of Odoo features and mission is to support the collaborative development of Odoo features and
promote its widespread use. promote its widespread use.
To contribute to this module, please visit http://odoo-community.org.
To contribute to this module, please visit https://odoo-community.org.

3
auto_backup/__init__.py

@ -20,5 +20,4 @@
# #
############################################################################## ##############################################################################
from . import model
from . import tests
from . import models

14
auto_backup/__openerp__.py

@ -22,22 +22,26 @@
{ {
"name": "Database Auto-Backup", "name": "Database Auto-Backup",
"summary": "Backups database",
"version": "8.0.1.0.0", "version": "8.0.1.0.0",
"author": ( "author": (
"VanRoey.be - Yenthe Van Ginneken, Agile Business Group," "VanRoey.be - Yenthe Van Ginneken, Agile Business Group,"
" Grupo ESOC Ingeniería de Servicios,"
" Odoo Community Association (OCA)" " Odoo Community Association (OCA)"
), ),
'license': "AGPL-3", 'license': "AGPL-3",
"website": "http://www.vanroey.be/applications/bedrijfsbeheer/odoo", "website": "http://www.vanroey.be/applications/bedrijfsbeheer/odoo",
"category": "Tools", "category": "Tools",
"summary": "Backups data base",
"depends": ['email_template'], "depends": ['email_template'],
"demo": [], "demo": [],
"data": [ "data": [
"view/bkp_conf_view.xml",
"data/backup_data.xml",
"security/ir.model.access.csv"
"data/backup_data.yml",
"security/ir.model.access.csv",
"view/db_backup_view.xml",
], ],
"application": True, "application": True,
"installable": True
"installable": True,
"external_dependencies": {
"python": ["pysftp"],
},
} }

18
auto_backup/data/backup_data.xml

@ -1,18 +0,0 @@
<?xml version="1.0"?>
<openerp>
<data noupdate="1">
<record id="ir_cron_backupscheduler0" model="ir.cron">
<field eval="&quot;&quot;&quot;schedule_backup&quot;&quot;&quot;" name="function"/>
<field eval="&quot;&quot;&quot;work_days&quot;&quot;&quot;" name="interval_type"/>
<field name="user_id" ref="base.user_root"/>
<field eval="&quot;&quot;&quot;Backup scheduler&quot;&quot;&quot;" name="name"/>
<field eval="-1" name="numbercall"/>
<field eval="&quot;&quot;&quot;2015-07-07 11:35:28&quot;&quot;&quot;" name="nextcall"/>
<field eval="5" name="priority"/>
<field eval="0" name="doall"/>
<field eval="False" name="active"/>
<field eval="1" name="interval_number"/>
<field eval="&quot;&quot;&quot;db.backup&quot;&quot;&quot;" name="model"/>
</record>
</data>
</openerp>

28
auto_backup/data/backup_data.yml

@ -0,0 +1,28 @@
# -*- coding: utf-8 -*-
# © 2016 Grupo ESOC Ingeniería de Servicios, S.L.U. - Jairo Llopis
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
# Cron job
- !record {model: ir.cron, id: ir_cron_backupscheduler0}:
name: Backup scheduler
user_id: base.user_root
interval_number: 1
interval_type: days
numbercall: -1
nextcall: !eval
(datetime.now() + timedelta(days=1)).strftime("%Y-%m-%d 02:00:00")
model: db.backup
function: action_backup_all
# New message subtypes
- !record {model: mail.message.subtype, id: success}:
name: Backup successful
res_model: db.backup
default: False
description: Database backup succeeded.
- !record {model: mail.message.subtype, id: failure}:
name: Backup failed
res_model: db.backup
default: True
description: Database backup failed.

141
auto_backup/i18n/auto_backup.pot

@ -1,141 +0,0 @@
# Translation of OpenERP Server.
# This file contains the translation of the following modules:
# * auto_backup
#
msgid ""
msgstr ""
"Project-Id-Version: OpenERP Server 5.0.6\n"
"Report-Msgid-Bugs-To: support@openerp.com\n"
"POT-Creation-Date: 2009-11-24 13:49:51+0000\n"
"PO-Revision-Date: 2009-11-24 13:49:51+0000\n"
"Last-Translator: <>\n"
"Language-Team: \n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: \n"
"Plural-Forms: \n"
#. module: auto_backup
#: help:db.backup,name:0
msgid "Database you want to schedule backups for"
msgstr ""
#. module: auto_backup
#: constraint:ir.model:0
msgid "The Object name must start with x_ and not contain any special character !"
msgstr ""
#. module: auto_backup
#: constraint:ir.actions.act_window:0
msgid "Invalid model name in the action definition."
msgstr ""
#. module: auto_backup
#: model:ir.model,name:auto_backup.model_db_backup
msgid "db.backup"
msgstr ""
#. module: auto_backup
#: view:db.backup:0
msgid "1) Go to Administration / Configuration / Scheduler / Scheduled Actions"
msgstr ""
#. module: auto_backup
#: model:ir.actions.act_window,name:auto_backup.action_backup_conf_form
#: model:ir.ui.menu,name:auto_backup.backup_conf_menu
msgid "Configure Backup"
msgstr ""
#. module: auto_backup
#: view:db.backup:0
msgid "Test"
msgstr ""
#. module: auto_backup
#: view:db.backup:0
msgid "IP Configuration"
msgstr ""
#. module: auto_backup
#: help:db.backup,bkp_dir:0
msgid "Absolute path for storing the backups"
msgstr ""
#. module: auto_backup
#: model:ir.module.module,shortdesc:auto_backup.module_meta_information
msgid "Database Auto-Backup"
msgstr ""
#. module: auto_backup
#: view:db.backup:0
msgid "Database Configuration"
msgstr ""
#. module: auto_backup
#: view:db.backup:0
msgid "4) Set other values as per your preference"
msgstr ""
#. module: auto_backup
#: field:db.backup,host:0
msgid "Host"
msgstr ""
#. module: auto_backup
#: view:db.backup:0
msgid "Automatic backup of all the databases under this can be scheduled as follows: "
msgstr ""
#. module: auto_backup
#: constraint:ir.ui.view:0
msgid "Invalid XML for View Architecture!"
msgstr ""
#. module: auto_backup
#: field:db.backup,bkp_dir:0
msgid "Backup Directory"
msgstr ""
#. module: auto_backup
#: field:db.backup,name:0
msgid "Database"
msgstr ""
#. module: auto_backup
#: view:db.backup:0
msgid "2) Schedule new action(create a new record)"
msgstr ""
#. module: auto_backup
#: model:ir.module.module,description:auto_backup.module_meta_information
msgid "The generic Open ERP Database Auto-Backup system enables the user to make configurations for the automatic backup of the database.\n"
"User simply requires to specify host & port under IP Configuration & database(on specified host running at specified port) and backup directory(in which all the backups of the specified database will be stored) under Database Configuration.\n"
"\n"
"Automatic backup for all such configured databases under this can then be scheduled as follows: \n"
" \n"
"1) Go to Administration / Configuration / Scheduler / Scheduled Actions\n"
"2) Schedule new action(create a new record)\n"
"3) Set 'Object' to 'db.backup' and 'Function' to 'schedule_backup' under page 'Technical Data'\n"
"4) Set other values as per your preference"
msgstr ""
#. module: auto_backup
#: view:db.backup:0
msgid "3) Set 'Object' to 'db.backup' and 'Function' to 'schedule_backup' under page 'Technical Data'"
msgstr ""
#. module: auto_backup
#: view:db.backup:0
msgid "Help"
msgstr ""
#. module: auto_backup
#: view:db.backup:0
msgid "This configures the scheduler for automatic backup of the given database running on given host at given port on regular intervals."
msgstr ""
#. module: auto_backup
#: field:db.backup,port:0
msgid "Port"
msgstr ""

376
auto_backup/model/backup_scheduler.py

@ -1,376 +0,0 @@
# -*- coding: utf-8 -*-
##############################################################################
# OpenERP, Open Source Management Solution
# Copyright (C) 2004-2009 Tiny SPRL (<http://tiny.be>). All Rights Reserved
# Copyright 2015 Agile Business Group <http://www.agilebg.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
##############################################################################
import socket
import os
import time
import datetime
import re
from openerp import models, fields, api, _
from openerp.exceptions import except_orm, Warning as UserError
from openerp import tools
from openerp.service import db
import logging
_logger = logging.getLogger(__name__)
try:
import pysftp
except ImportError:
_logger.debug('Can not import pysftp')
def execute(connector, method, *args):
res = False
try:
res = getattr(connector, method)(*args)
except socket.error as e:
raise e
return res
class DbBackup(models.Model):
_name = 'db.backup'
@api.model
def _get_db_name(self):
return self.env.cr.dbname
name = fields.Char(
string='Database', size=100, required=True,
default=_get_db_name,
help='Database you want to schedule backups for'
)
bkp_dir = fields.Char(
string='Backup Directory', size=100,
default='/odoo/backups',
help='Absolute path for storing the backups',
required=True
)
autoremove = fields.Boolean(
string='Auto. Remove Backups',
help=(
"If you check this option you can choose to "
"automaticly remove the backup after xx days"
)
)
daystokeep = fields.Integer(
string='Remove after x days',
default=30,
help=(
"Choose after how many days the backup should be "
"deleted. For example:\nIf you fill in 5 the backups "
"will be removed after 5 days."
), required=True
)
sftpwrite = fields.Boolean(
string='Write to external server with sftp',
help=(
"If you check this option you can specify the details "
"needed to write to a remote server with SFTP."
)
)
sftppath = fields.Char(
string='Path external server',
help=(
"The location to the folder where the dumps should be "
"written to. For example /odoo/backups/.\nFiles will then"
" be written to /odoo/backups/ on your remote server."
)
)
sftpip = fields.Char(
string='IP Address SFTP Server',
help=(
"The IP address from your remote"
" server. For example 192.168.0.1"
)
)
sftpport = fields.Integer(
string="SFTP Port",
default=22,
help="The port on the FTP server that accepts SSH/SFTP calls."
)
sftpusername = fields.Char(
string='Username SFTP Server',
help=(
"The username where the SFTP connection "
"should be made with. This is the user on the external server."
)
)
sftppassword = fields.Char(
string='Password User SFTP Server',
help=(
"The password from the user where the SFTP connection "
"should be made with. This is the password from the user"
" on the external server."
)
)
daystokeepsftp = fields.Integer(
string='Remove SFTP after x days',
default=30,
help=(
"Choose after how many days the backup should be deleted "
"from the FTP server. For example:\nIf you fill in 5 the "
"backups will be removed after 5 days from the FTP server."
)
)
sendmailsftpfail = fields.Boolean(
string='Auto. E-mail on backup fail',
help=(
"If you check this option you can choose to automaticly"
" get e-mailed when the backup to the external server failed."
)
)
emailtonotify = fields.Char(
string='E-mail to notify',
help=(
"Fill in the e-mail where you want to be"
" notified that the backup failed on the FTP."
)
)
lasterrorlog = fields.Text(
string='E-mail to notify',
help=(
"Fill in the e-mail where you want to be"
" notified that the backup failed on the FTP."
)
)
@api.multi
def _check_db_exist(self):
for rec in self:
db_list = db.exp_list(True)
if rec.name in db_list:
return True
return False
_constraints = [
(
_check_db_exist,
_('Error ,No such database exists'), ['name']
)
]
@api.multi
def test_sftp_connection(self):
confs = self.search([])
# Check if there is a success or fail and write messages
messageTitle = ""
messageContent = ""
conn_success = False
for rec in confs:
try:
conn_success = True
ipHost = rec.sftpip
portHost = rec.sftpport
usernameLogin = rec.sftpusername
passwordLogin = rec.sftppassword
# Connect with external server over SFTP, so we know sure that
# everything works.
srv = pysftp.Connection(host=ipHost, username=usernameLogin,
password=passwordLogin, port=portHost)
srv.close()
# We have a success.
messageTitle = _("Connection Test Succeeded!")
messageContent = _(
"Everything seems properly set up for FTP back-ups!")
except Exception as e:
conn_success = False
messageTitle = _("Connection Test Failed!")
if len(rec.sftpip) < 8:
messageContent += _(
"\nYour IP address seems to be too short.\n")
messageContent += _("Here is what we got instead:\n")
if not conn_success:
raise except_orm(
_(messageTitle), _(
messageContent + "%s") %
tools.ustr(e))
else:
raise UserError(_(messageTitle), _(messageContent))
@api.model
def schedule_backup(self):
for rec in self.search([]):
if not os.path.isdir(rec.bkp_dir):
os.makedirs(rec.bkp_dir)
# Create name for dumpfile.
bkp_file = '%s_%s.dump.zip' % (
time.strftime('%d_%m_%Y_%H_%M_%S'),
rec.name)
file_path = os.path.join(rec.bkp_dir, bkp_file)
fbk = open(file_path, 'wb')
db.dump_db(rec.name, fbk)
fbk.close()
# Check if user wants to write to SFTP or not.
if rec.sftpwrite is True:
try:
# Store all values in variables
dir = rec.bkp_dir
pathToWriteTo = rec.sftppath
ipHost = rec.sftpip
portHost = rec.sftpport
usernameLogin = rec.sftpusername
passwordLogin = rec.sftppassword
# Connect with external server over SFTP
srv = pysftp.Connection(
host=ipHost,
username=usernameLogin,
password=passwordLogin,
port=portHost)
# Move to the correct directory on external server. If the
# user made a typo in his path with multiple slashes
# (/odoo//backups/) it will be fixed by this regex.
pathToWriteTo = re.sub('/+', '/', pathToWriteTo)
_logger.debug(
'Start to copy files..'
)
try:
srv.chdir(pathToWriteTo)
except IOError:
# Create directory and subdirs if they do not exist.
currentDir = ''
for dirElement in pathToWriteTo.split('/'):
currentDir += dirElement + '/'
try:
srv.chdir(currentDir)
except:
_logger.info(
_(
'(Part of the) path didn\'t exist. '
'Creating it now at %s'
) % currentDir
)
# Make directory and then navigate into it
srv.mkdir(currentDir, mode=777)
srv.chdir(currentDir)
pass
srv.chdir(pathToWriteTo)
# Loop over all files in the directory.
for f in os.listdir(dir):
fullpath = os.path.join(dir, f)
if os.path.isfile(fullpath):
srv.put(fullpath)
# Navigate in to the correct folder.
srv.chdir(pathToWriteTo)
# Loop over all files in the directory from the back-ups.
# We will check the creation date of every back-up.
for file in srv.listdir(pathToWriteTo):
# Get the full path
fullpath = os.path.join(pathToWriteTo, file)
if srv.isfile(fullpath) and ".dump.zip" in file:
# Get the timestamp from the file on the external
# server
timestamp = srv.stat(fullpath).st_atime
createtime = (
datetime.datetime.fromtimestamp(timestamp)
)
now = datetime.datetime.now()
delta = now - createtime
# If the file is older than the daystokeepsftp (the
# days to keep that the user filled in on the Odoo
# form it will be removed.
if (
rec.daystokeepsftp > 0 and
delta.days >= rec.daystokeepsftp
):
# Only delete files, no directories!
srv.unlink(file)
# Close the SFTP session.
srv.close()
except Exception as e:
_logger.debug(
'Exception We couldn\'t back '
'up to the FTP server..'
)
# At this point the SFTP backup failed.
# We will now check if the user wants
# an e-mail notification about this.
if rec.sendmailsftpfail:
self.send_notification(rec, e)
# Remove all old files (on local server)
# in case this is configured..
if rec.autoremove is True:
try:
self.remove_folder(rec)
except Exception as e:
_logger.debug(
'Exception when try to remove file'
)
return True
def send_notification(self, rec, e):
try:
ir_mail_server = self.env['ir.mail_server']
message = (
"Dear,\n\nThe backup for the server %s"
" (IP: %s) failed.Please check"
" the following details:\n\n"
"IP address SFTP server: %s \nUsername: %s"
"\nPassword: %s"
"\n\nError details: %s \n\nWith kind regards"
) % (
rec.host, rec.sftpip, rec.sftpip,
rec.sftpusername, rec.sftppassword,
tools.ustr(e)
)
msg = ir_mail_server.build_email(
"auto_backup@%s.com" % rec.name,
[rec.emailtonotify],
"Backup from %s ( %s ) failed" % (
rec.host, rec.sftpip),
message)
ir_mail_server.send_email(msg)
except Exception as e:
_logger.debug(
'Exception %s' % tools.ustr(e)
)
# This is done after the SFTP writing to prevent unusual behaviour:
# If the user would set local back-ups to be kept 0 days and the SFTP
# to keep backups xx days there wouldn't be any new back-ups added
# to the SFTP.
# If we'd remove the dump files before they're writen to the SFTP
# there willbe nothing to write. Meaning that if an user doesn't want
# to keep back-ups locally and only wants them on the SFTP
# (NAS for example) there wouldn't be any writing to the
# remote server if this if statement was before the SFTP write method
# right above this comment.
def remove_folder(self, rec):
dir = rec.bkp_dir
# Loop over all files in the directory.
for f in os.listdir(dir):
fullpath = os.path.join(dir, f)
if os.path.isfile(fullpath) and ".dump.zip" in f:
timestamp = os.stat(fullpath).st_ctime
createtime = (
datetime.datetime.fromtimestamp(timestamp)
)
now = datetime.datetime.now()
delta = now - createtime
if delta.days >= rec.daystokeep:
_logger.debug(
'Remove local file...'
)
os.remove(fullpath)

2
auto_backup/model/__init__.py → auto_backup/models/__init__.py

@ -20,4 +20,4 @@
# #
############################################################################## ##############################################################################
from . import backup_scheduler
from . import db_backup

282
auto_backup/models/db_backup.py

@ -0,0 +1,282 @@
# -*- coding: utf-8 -*-
##############################################################################
# OpenERP, Open Source Management Solution
# Copyright (C) 2004-2009 Tiny SPRL (<http://tiny.be>). All Rights Reserved
# Copyright 2015 Agile Business Group <http://www.agilebg.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
##############################################################################
import os
import shutil
import tempfile
import traceback
from contextlib import contextmanager
from datetime import datetime, timedelta
from glob import iglob
from openerp import exceptions, models, fields, api, _, tools
from openerp.service import db
import logging
_logger = logging.getLogger(__name__)
try:
import pysftp
except ImportError:
_logger.warning('Cannot import pysftp')
class DbBackup(models.Model):
_name = 'db.backup'
_inherit = "mail.thread"
_sql_constraints = [
("days_to_keep_positive", "CHECK(days_to_keep >= 0)",
"I cannot remove backups from the future. Ask Doc for that."),
]
name = fields.Char(
string="Name",
compute="_compute_name",
store=True,
help="Summary of this backup process",
)
folder = fields.Char(
default=lambda self: self._default_folder(),
oldname="bkp_dir",
help='Absolute path for storing the backups',
required=True
)
days_to_keep = fields.Integer(
oldname="daystokeep",
required=True,
default=0,
help="Backups older than this will be deleted automatically. "
"Set 0 to disable autodeletion.",
)
method = fields.Selection(
selection=[("local", "Local disk"), ("sftp", "Remote SFTP server")],
default="local",
help="Choose the storage method for this backup.",
)
sftp_host = fields.Char(
string='SFTP Server',
oldname="sftpip",
help=(
"The host name or IP address from your remote"
" server. For example 192.168.0.1"
)
)
sftp_port = fields.Integer(
string="SFTP Port",
default=22,
oldname="sftpport",
help="The port on the FTP server that accepts SSH/SFTP calls."
)
sftp_user = fields.Char(
string='Username in the SFTP Server',
oldname="sftpusername",
help=(
"The username where the SFTP connection "
"should be made with. This is the user on the external server."
)
)
sftp_password = fields.Char(
string="SFTP Password",
oldname="sftppassword",
help="The password for the SFTP connection. If you specify a private "
"key file, then this is the password to decrypt it.",
)
sftp_private_key = fields.Char(
string="Private key location",
help="Path to the private key file. Only the Odoo user should have "
"read permissions for that file.",
)
@api.model
def _default_folder(self):
"""Default to ``backups`` folder inside current database datadir."""
return os.path.join(
tools.config.filestore(self.env.cr.dbname),
"backups")
@api.multi
@api.depends("folder", "method", "sftp_host", "sftp_port", "sftp_user")
def _compute_name(self):
"""Get the right summary for this job."""
for rec in self:
if rec.method == "local":
rec.name = "%s @ localhost" % rec.folder
elif rec.method == "sftp":
rec.name = "sftp://%s@%s:%d%s" % (
rec.sftp_user, rec.sftp_host, rec.sftp_port, rec.folder)
@api.multi
def action_sftp_test_connection(self):
"""Check if the SFTP settings are correct."""
try:
# Just open and close the connection
with self.sftp_connection():
raise exceptions.Warning(_("Connection Test Succeeded!"))
except (pysftp.CredentialException, pysftp.ConnectionException):
_logger.info("Connection Test Failed!", exc_info=True)
raise exceptions.Warning(_("Connection Test Failed!"))
@api.multi
def action_backup(self):
"""Run selected backups."""
backup = None
filename = self.filename(datetime.now())
successful = self.browse()
# Start with local storage
for rec in self.filtered(lambda r: r.method == "local"):
with rec.backup_log():
# Directory must exist
try:
os.makedirs(rec.folder)
except OSError:
pass
with open(os.path.join(rec.folder, filename),
'wb') as destiny:
# Copy the cached backup
if backup:
with open(backup) as cached:
shutil.copyfileobj(cached, destiny)
# Generate new backup
else:
db.dump_db(self.env.cr.dbname, destiny)
backup = backup or destiny.name
successful |= rec
# Ensure a local backup exists if we are going to write it remotely
sftp = self.filtered(lambda r: r.method == "sftp")
if sftp:
if backup:
cached = open(backup)
else:
cached = tempfile.TemporaryFile()
db.dump_db(self.env.cr.dbname, cached)
with cached:
for rec in sftp:
with rec.backup_log():
with rec.sftp_connection() as remote:
# Directory must exist
try:
remote.makedirs(rec.folder)
except pysftp.ConnectionException:
pass
# Copy cached backup to remote server
with remote.open(
os.path.join(rec.folder, filename),
"wb") as destiny:
shutil.copyfileobj(cached, destiny)
successful |= rec
# Remove old files for successful backups
successful.cleanup()
@api.model
def action_backup_all(self):
"""Run all scheduled backups."""
return self.search([]).action_backup()
@api.multi
@contextmanager
def backup_log(self):
"""Log a backup result."""
try:
_logger.info("Starting database backup: %s", self.name)
yield
except:
_logger.exception("Database backup failed: %s", self.name)
escaped_tb = tools.html_escape(traceback.format_exc())
self.message_post(
"<p>%s</p><pre>%s</pre>" % (
_("Database backup failed."),
escaped_tb),
subtype=self.env.ref("auto_backup.failure"))
else:
_logger.info("Database backup succeeded: %s", self.name)
self.message_post(_("Database backup succeeded."))
@api.multi
def cleanup(self):
"""Clean up old backups."""
now = datetime.now()
for rec in self.filtered("days_to_keep"):
with rec.cleanup_log():
oldest = self.filename(now - timedelta(days=rec.days_to_keep))
if rec.method == "local":
for name in iglob(os.path.join(rec.folder,
"*.dump.zip")):
if name < oldest:
os.unlink(name)
elif rec.method == "sftp":
with rec.sftp_connection() as remote:
for name in remote.listdir(rec.folder):
if name.endswith(".dump.zip") and name < oldest:
remote.unlink(name)
@api.multi
@contextmanager
def cleanup_log(self):
"""Log a possible cleanup failure."""
try:
_logger.info("Starting cleanup process after database backup: %s",
self.name)
yield
except:
_logger.exception("Cleanup of old database backups failed: %s")
escaped_tb = tools.html_escape(traceback.format_exc())
self.message_post(
"<p>%s</p><pre>%s</pre>" % (
_("Cleanup of old database backups failed."),
escaped_tb),
subtype=self.env.ref("auto_backup.failure"))
else:
_logger.info("Cleanup of old database backups succeeded: %s",
self.name)
@api.model
def filename(self, when):
"""Generate a file name for a backup.
:param datetime.datetime when:
Use this datetime instead of :meth:`datetime.datetime.now`.
"""
return "{:%Y_%m_%d_%H_%M_%S}.dump.zip".format(when)
@api.multi
def sftp_connection(self):
"""Return a new SFTP connection with found parameters."""
params = {
"host": self.sftp_host,
"username": self.sftp_user,
"port": self.sftp_port,
}
_logger.debug(
"Trying to connect to sftp://%(username)s@%(host)s:%(port)d",
extra=params)
if self.sftp_private_key:
params["private_key"] = self.stfpprivatekey
if self.sftp_password:
params["private_key_pass"] = self.sftp_password
else:
params["password"] = self.sftp_password
return pysftp.Connection(**params)

3
auto_backup/security/ir.model.access.csv

@ -1,2 +1,3 @@
id,name,model_id:id,group_id:id,perm_read,perm_write,perm_create,perm_unlink id,name,model_id:id,group_id:id,perm_read,perm_write,perm_create,perm_unlink
access_db_backup,access_db_backup,model_db_backup,,1,0,0,0
access_db_backup_read,Read db.backup,model_db_backup,base.group_erp_manager,1,0,0,0
access_db_backup_write,Write db.backup,model_db_backup,base.group_system,1,1,1,1

BIN
auto_backup/static/description/icon.png

After

Width: 100  |  Height: 100  |  Size: 5.9 KiB

51
auto_backup/static/description/icon.svg

@ -0,0 +1,51 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:cc="http://creativecommons.org/ns#"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:svg="http://www.w3.org/2000/svg"
xmlns="http://www.w3.org/2000/svg"
version="1.1"
id="svg2"
viewBox="0 0 99.999997 99.999997"
height="100"
width="100">
<defs
id="defs4" />
<metadata
id="metadata7">
<rdf:RDF>
<cc:Work
rdf:about="">
<dc:format>image/svg+xml</dc:format>
<dc:type
rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
<dc:title></dc:title>
</cc:Work>
</rdf:RDF>
</metadata>
<g
transform="translate(0,-952.36223)"
id="layer1">
<text
id="text3336"
y="1031.6924"
x="40.411446"
style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:88.29121399px;font-family:'Nuosu SIL';-inkscape-font-specification:'Nuosu SIL';text-align:center;text-anchor:middle;fill:#1a1a1a;fill-opacity:1;stroke:#000000;stroke-width:1.7658242;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
xml:space="preserve"><tspan
style="fill:#1a1a1a;stroke-width:1.7658242"
y="1031.6924"
x="40.411446"
id="tspan3338"></tspan></text>
<text
id="text3340"
y="1050.2731"
x="74.752251"
style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:56.87847137px;font-family:'Nuosu SIL';-inkscape-font-specification:'Nuosu SIL';text-align:center;text-anchor:middle;fill:#b3b3b3;fill-opacity:1;stroke:#000000;stroke-width:1.13756943;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
xml:space="preserve"><tspan
style="stroke-width:1.13756943"
y="1050.2731"
x="74.752251"
id="tspan3342"></tspan></text>
</g>
</svg>

100
auto_backup/static/description/no_index.html

@ -1,100 +0,0 @@
<section class="oe_container">
<div class="oe_row oe_spaced">
<div class="oe_span12">
<h2 class="oe_slogan">Automated backups</h2>
<h3 class="oe_slogan">A tool for all your back-ups, internal and external!</h3>
</div>
<div class="oe_span6">
<div class="oe_demo oe_picture oe_screenshot">
<img src="overview.png">
</div>
</div>
<div class="oe_span6">
<p class="oe_mt32">
Keep your Odoo data safe with this module. Take automated back-ups, remove them automatically
and even write them to an external server through an encrypted tunnel.
You can even specify how long local backups and external backups should be kept, automatically!
</p>
<div class="oe_centeralign oe_websiteonly">
<a href="http://www.openerp.com/start?app=mail" class="oe_button oe_big oe_tacky">Start your <span class="oe_emph">free</span> trial</a>
</div>
</div>
</div>
</section>
<!-- Second block -->
<section class="oe_container oe_dark">
<div class="oe_row oe_spaced">
<h2 class="oe_slogan">Connect with an FTP Server</h2>
<h3 class="oe_slogan">Keep your data safe, through an SSH tunnel!</h3>
<div class="oe_span6">
<p class="oe_mt32">
Want to go even further and write your backups to an external server?
You can with this module! Specify the credentials to the server, specify a path and everything will be backed up automatically. This is done through an SSH (encrypted) tunnel, thanks to pysftp, so your data is safe!
</p>
</div>
<div class="oe_span6">
<div class="oe_row_img oe_centered">
<img class="oe_picture oe_screenshot" src="terminalssh.png">
</div>
</div>
</div>
</section>
<!--Third block -->
<section class="oe_container">
<div class="oe_row oe_spaced">
<div class="oe_span12">
<h2 class="oe_slogan">Test connection</h2>
<h3 class="oe_slogan">Checks your credentials in one click</h3>
</div>
<div class="oe_span6">
<div class="oe_demo oe_picture oe_screenshot">
<img src="testconnection.png">
<img src="testconnectionfailed.png">
</div>
</div>
<div class="oe_span6">
<p class="oe_mt32">
Want to make sure if the connection details are correct and if Odoo can automatically write them to the remote server? Simply click on the 'Test SFTP Connection' button and you will get message telling you if everything is OK, or what is wrong!
</p>
</div>
</div>
</section>
<!-- Fourth block -->
<section class="oe_container oe_dark">
<div class="oe_row oe_spaced">
<h2 class="oe_slogan">E-mail on backup failure</h2>
<h3 class="oe_slogan">Stay informed of problems, automatically!</h3>
<div class="oe_span6">
<p class="oe_mt32">
Do you want to know if the database backup failed? Check the checkbox 'Auto. E-mail on backup fail' and fill in your e-mail.
Every time a backup fails you will get an e-mail in your mailbox with technical details.
</p>
</div>
<div class="oe_span6">
<div class="oe_row_img oe_centered">
<img class="oe_picture oe_screenshot" src="emailnotification.png">
</div>
</div>
</div>
</section>
<!--Fifth block -->
<section class="oe_container">
<div class="oe_row oe_spaced">
<div class="oe_span12">
<h2 class="oe_slogan">Contact / Support</h2>
<h3 class="oe_slogan">Need help or want extra features?</h3>
</div>
<div class="oe_span6">
<div class="oe_demo oe_picture oe_screenshot">
<a href="http://www.vanroey.be/appplications/bedrijfsbeheer/odoo"><img src="logo.png"></a>
</div>
</div>
<div class="oe_span6">
<p class="oe_mt32">
Need help with the configuration or want this module to have more functionalities?
Contact us through e-mail at <a href="mailto:"yenthe.vanginneken@vanroey.be">yenthe.vanginneken@vanroey.be</a> or <a href="mailto:"tony.crols@vanroey.be">tony.crols@vanroey.be</a>
</p>
</div>
</div>
</section>

37
auto_backup/tests/test_auto_backup.py

@ -19,38 +19,25 @@
# #
############################################################################## ##############################################################################
from openerp.tests import common
from openerp.exceptions import except_orm
import os import os
import time
from datetime import datetime
from openerp.tests import common
class TestsAutoBackup(common.TransactionCase): class TestsAutoBackup(common.TransactionCase):
def setUp(self): def setUp(self):
super(TestsAutoBackup, self).setUp() super(TestsAutoBackup, self).setUp()
self.abk_model = self.env["db.backup"]
self.cron_model = self.env["ir.cron"]
def test_0(self):
with self.assertRaises(except_orm):
self.abk_model.create(
{
'name': 'abcd',
'adminpassword': 'admin'
}
)
def test_1(self):
this = self.abk_model.create(
self.abk = self.env["db.backup"].create(
{ {
'bkp_dir': '/tmp'
'name': u'Têst backup',
} }
) )
self.assertEqual(this.bkp_dir, '/tmp')
bkp_file = '%s_%s.dump.zip' % (
time.strftime('%d_%m_%Y_%H_%M_%S'),
this.name)
file_path = os.path.join(this.bkp_dir, bkp_file)
this.schedule_backup()
self.assertTrue(os.path.isfile(file_path))
def test_local(self):
"""A local database is backed up."""
filename = self.abk.filename(datetime.now())
self.abk.action_backup()
generated_backup = [f for f in os.listdir(self.abk.folder)
if f >= filename]
self.assertEqual(len(generated_backup), 1)

100
auto_backup/view/bkp_conf_view.xml

@ -1,100 +0,0 @@
<?xml version="1.0"?>
<openerp>
<data>
<record model="ir.ui.view" id="view_backup_conf_form">
<field name="name">Configure Backup</field>
<field name="model">db.backup</field>
<field name="type">form</field>
<field name="arch" type="xml">
<form string="Test">
<group col="4" colspan="4">
<separator col="2" string="Local backup configuration"/>
</group>
<group>
<field name="name"/>
<field name="bkp_dir"/>
<field name="autoremove"/>
<field name="daystokeep" attrs="{'invisible': [('autoremove','=',False)]}"/>
</group>
<group col="4" colspan="4">
<separator col="2" string="SFTP"/>
</group>
<div
style="width:50%;border-radius:10px;
margin: 10px 0px;
padding:15px 10px 15px 10px;
background-repeat:
no-repeat;background-position: 10px center;
color: #9F6000;background-color: #FEEFB3;"
attrs="{'invisible': [('sftpwrite','=',False)]}">
<b>Warning:</b>
Use SFTP with caution! This writes files to external servers under the path you specify.
</div>
<group>
<field name="sftpwrite"/>
<field name="sftpip" attrs="{'invisible':[('sftpwrite', '==', False)],'required':[('sftpwrite', '==', True)]}"/>
<field name="sftpport" attrs="{'invisible':[('sftpwrite', '==', False)],'required':[('sftpwrite', '==', True)]}"/>
<field name="sftpusername" attrs="{'invisible':[('sftpwrite', '==', False)],'required':[('sftpwrite', '==', True)]}"/>
<field name="sftppassword" attrs="{'invisible':[('sftpwrite', '==', False)],'required':[('sftpwrite', '==', True)]}" password="True"/>
<field name="sftppath" attrs="{'invisible':[('sftpwrite', '==', False)],'required':[('sftpwrite', '==', True)]}" placeholder="For example: /odoo/backups/"/>
<field name="daystokeepsftp" attrs="{'invisible':[('sftpwrite', '==', False)],'required':[('sftpwrite', '==', True)]}"/>
<field name="sendmailsftpfail" attrs="{'invisible': [('sftpwrite','=',False)]}"/>
<field name="emailtonotify" attrs="{'invisible':['|',('sendmailsftpfail', '==', False),
('sftpwrite', '==', False)],'required':[('sendmailsftpfail', '==', True)]}"/>
<button name="test_sftp_connection" type="object" attrs="{'invisible': [('sftpwrite','=',False)]}" string="Test SFTP Connection" icon="gtk-network"/>
</group>
<separator string="Help" colspan="2"/>
<div>
This configures the scheduler for automatic backup of the given database running on given host at given port on regular intervals.
<br/>
Automatic backups of the database can be scheduled as follows:
<ol>
<li>Go to Settings / Technical / Automation / Scheduled Actions.</li>
<li>Search the action named 'Backup scheduler'.</li>
<li>Set the scheduler to active and fill in how often you want backups generated.</li>
</ol>
</div>
</form>
</field>
</record>
<record model="ir.ui.view" id="view_backup_conf_tree">
<field name="name">Configure Backup</field>
<field name="model">db.backup</field>
<field name="type">tree</field>
<field name="arch" type="xml">
<tree string="Backups">
<field name='name'/>
<field name='bkp_dir'/>
<field name="autoremove"/>
<field name="sftpip"/>
</tree>
</field>
</record>
<record model="ir.ui.view" id="view_backup_conf_search">
<field name="name">Configure Backup</field>
<field name="model">db.backup</field>
<field name="type">search</field>
<field name="arch" type="xml">
<search string="Search options">
<field name='name'/>
<field name='bkp_dir'/>
<field name="autoremove"/>
<field name="sftpip"/>
</search>
</field>
</record>
<record model="ir.actions.act_window" id="action_backup_conf_form">
<field name="name">Configure Backup</field>
<field name="res_model">db.backup</field>
<field name="view_type">form</field>
<field name='view_mode'>tree,form</field>
<field name='view_id' ref='view_backup_conf_tree'/>
</record>
<menuitem parent="base.menu_config" action="action_backup_conf_form" id="backup_conf_menu"/>
</data>
</openerp>

94
auto_backup/view/db_backup_view.xml

@ -0,0 +1,94 @@
<?xml version="1.0"?>
<openerp>
<data>
<record model="ir.ui.view" id="view_backup_conf_form">
<field name="name">Automated Backups</field>
<field name="model">db.backup</field>
<field name="type">form</field>
<field name="arch" type="xml">
<form>
<h1><field name="name"/></h1>
<group string="Local backup configuration">
<field name="folder"/>
<field name="days_to_keep"/>
<field name="method"/>
</group>
<group
string="SFTP Settings"
attrs="{
'invisible': [
('method', '!=', 'sftp'),
],
}">
<div class="bg-warning text-warning">
<h3>Warning:</h3>
Use SFTP with caution! This writes files to external servers under the path you specify.
</div>
<field name="sftp_host" placeholder="sftp.example.com"/>
<field name="sftp_port"/>
<field name="sftp_user" placeholder="john"/>
<field name="sftp_password"/>
<field
name="sftp_private_key"
placeholder="/home/odoo/.ssh/id_rsa"/>
<button
name="action_sftp_test_connection"
type="object"
string="Test SFTP Connection"
icon="gtk-network"/>
</group>
<separator string="Help" colspan="2"/>
<div>
This configures the scheduler for automatic backup of the given database running on given host at given port on regular intervals.
<br/>
Automatic backups of the database can be scheduled as follows:
<ol>
<li>Go to Settings / Technical / Automation / Scheduled Actions.</li>
<li>Search the action named 'Backup scheduler'.</li>
<li>Set the scheduler to active and fill in how often you want backups generated.</li>
</ol>
</div>
</form>
</field>
</record>
<record model="ir.ui.view" id="view_backup_conf_tree">
<field name="name">Automated Backups</field>
<field name="model">db.backup</field>
<field name="type">tree</field>
<field name="arch" type="xml">
<tree string="Backups">
<field name='name'/>
<field name='folder'/>
<field name="sftp_host"/>
</tree>
</field>
</record>
<record model="ir.ui.view" id="view_backup_conf_search">
<field name="name">Automated Backups</field>
<field name="model">db.backup</field>
<field name="type">search</field>
<field name="arch" type="xml">
<search string="Search options">
<field name='name'/>
<field name='folder'/>
<field name="sftp_host"/>
</search>
</field>
</record>
<act_window
name="Automated Backups"
id="action_backup_conf_form"
res_model="db.backup"/>
<menuitem
parent="base.menu_config"
action="action_backup_conf_form"
id="backup_conf_menu"/>
</data>
</openerp>
Loading…
Cancel
Save