[3/4] Add support for 'django-dbbackup'

Message ID 20181006214506.22720-4-stephen@that.guru
State Accepted
Headers show
Series
  • Add dbbackup, dbrestore commands
Related show

Commit Message

Stephen Finucane Oct. 6, 2018, 9:45 p.m.
'parsemail' and 'parsearchive' are slow. When messing with models and
migrations, it can be very useful to backup the database in its current
state and restore it if/when you mess up. Currently, we've documented
how to do this via some commands run in the shell of the container, but
we can do things easier via an application designed for this purpose:
'django-dbshell'.

Signed-off-by: Stephen Finucane <stephen@that.guru>
---
 .gitignore                |  3 +++
 patchwork/settings/dev.py | 13 +++++++++++++
 requirements-dev.txt      |  1 +
 3 files changed, 17 insertions(+)

Comments

Daniel Axtens Oct. 9, 2018, 1:54 a.m. | #1
Stephen Finucane <stephen@that.guru> writes:

> 'parsemail' and 'parsearchive' are slow. When messing with models and
> migrations, it can be very useful to backup the database in its current
> state and restore it if/when you mess up. Currently, we've documented
> how to do this via some commands run in the shell of the container, but
> we can do things easier via an application designed for this purpose:
> 'django-dbshell'.

You think it's better to include this dependency than to add a script to
tools/ that wraps mysqldump/mysqlrestore and the postgres equivalents?
I'm happy if you think this is the way to go, I just want to know you've
thought about it.

>
> Signed-off-by: Stephen Finucane <stephen@that.guru>
> ---
>  .gitignore                |  3 +++
>  patchwork/settings/dev.py | 13 +++++++++++++
>  requirements-dev.txt      |  1 +

Does this need docs added somewhere? I hadn't heard of django-dbshell
before this...

>  3 files changed, 17 insertions(+)
>
> diff --git a/.gitignore b/.gitignore
> index 57376bdd..a33d1029 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -54,3 +54,6 @@ patchwork/settings/production.py
>  
>  # docker-compose configuration files
>  /.env
> +
> +# django-dbbackup files
Also, here you've called it django-dbbackup, in the commit message it's
django-dbshell. It that correct?

Apart from that, I'm very much an ACK for the idea of making backups
easier.

Regards,
Daniel
> +/.backups
> diff --git a/patchwork/settings/dev.py b/patchwork/settings/dev.py
> index 4bb5a93c..711177c9 100644
> --- a/patchwork/settings/dev.py
> +++ b/patchwork/settings/dev.py
> @@ -9,6 +9,11 @@ Design based on:
>  
>  from .base import *  # noqa
>  
> +try:
> +    import dbbackup
> +except ImportError:
> +    dbbackup = None
> +
>  #
>  # Core settings
>  # https://docs.djangoproject.com/en/1.11/ref/settings/#core-settings
> @@ -72,6 +77,14 @@ INTERNAL_IPS = [
>      '172.18.0.1'
>  ]
>  
> +# django-dbbackup
> +
> +if dbbackup:
> +    INSTALLED_APPS += [
> +        'dbbackup',
> +    ]
> +
> +    DBBACKUP_STORAGE_OPTIONS = {'location': '.backups'}
>  
>  #
>  # Patchwork settings
> diff --git a/requirements-dev.txt b/requirements-dev.txt
> index b0cdd0de..ed98c30e 100644
> --- a/requirements-dev.txt
> +++ b/requirements-dev.txt
> @@ -3,4 +3,5 @@ Django==1.11.15; python_version < '3.0'  # pyup: ignore
>  djangorestframework==3.8.2
>  django-filter==2.0.0; python_version >= '3.4'
>  django-filter==1.1.0; python_version < '3.0'  # pyup: ignore
> +django-dbbackup==3.2.0
>  -r requirements-test.txt
> -- 
> 2.17.1
>
> _______________________________________________
> Patchwork mailing list
> Patchwork@lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/patchwork
Stephen Finucane Oct. 9, 2018, 9:46 a.m. | #2
On Tue, 2018-10-09 at 12:54 +1100, Daniel Axtens wrote:
> Stephen Finucane <stephen@that.guru> writes:
> 
> > 'parsemail' and 'parsearchive' are slow. When messing with models and
> > migrations, it can be very useful to backup the database in its current
> > state and restore it if/when you mess up. Currently, we've documented
> > how to do this via some commands run in the shell of the container, but
> > we can do things easier via an application designed for this purpose:
> > 'django-dbshell'.
> 
> You think it's better to include this dependency than to add a script to
> tools/ that wraps mysqldump/mysqlrestore and the postgres equivalents?
> I'm happy if you think this is the way to go, I just want to know you've
> thought about it.

Yeah, I was going to write a new management command until I decided to
go with this. This is only installed and enabled as part of the dev
configuration so I figure the impact is minimal enough and we benefit
from someone else testing this for us :)

> > 
> > Signed-off-by: Stephen Finucane <stephen@that.guru>
> > ---
> >  .gitignore                |  3 +++
> >  patchwork/settings/dev.py | 13 +++++++++++++
> >  requirements-dev.txt      |  1 +
> 
> Does this need docs added somewhere? I hadn't heard of django-dbshell
> before this...

Hmm, possibly? I purposefully avoided noting this in the release notes
since it doesn't really affect real-world users. That said, I did note
when we added django-debug-toolbar (though not any subsequent updates)
so I could be talked around. In any case, I will add a follow-on patch
to update the docs to refer to this tool instead of manual
backups/restores.

> >  3 files changed, 17 insertions(+)
> > 
> > diff --git a/.gitignore b/.gitignore
> > index 57376bdd..a33d1029 100644
> > --- a/.gitignore
> > +++ b/.gitignore
> > @@ -54,3 +54,6 @@ patchwork/settings/production.py
> >  
> >  # docker-compose configuration files
> >  /.env
> > +
> > +# django-dbbackup files
> 
> Also, here you've called it django-dbbackup, in the commit message it's
> django-dbshell. It that correct?

Yeah, the commit message is wrong. Can fix if/when applying.

Stephen

> Apart from that, I'm very much an ACK for the idea of making backups
> easier.
> 
> Regards,
> Daniel
> > +/.backups
> > diff --git a/patchwork/settings/dev.py b/patchwork/settings/dev.py
> > index 4bb5a93c..711177c9 100644
> > --- a/patchwork/settings/dev.py
> > +++ b/patchwork/settings/dev.py
> > @@ -9,6 +9,11 @@ Design based on:
> >  
> >  from .base import *  # noqa
> >  
> > +try:
> > +    import dbbackup
> > +except ImportError:
> > +    dbbackup = None
> > +
> >  #
> >  # Core settings
> >  # https://docs.djangoproject.com/en/1.11/ref/settings/#core-settings
> > @@ -72,6 +77,14 @@ INTERNAL_IPS = [
> >      '172.18.0.1'
> >  ]
> >  
> > +# django-dbbackup
> > +
> > +if dbbackup:
> > +    INSTALLED_APPS += [
> > +        'dbbackup',
> > +    ]
> > +
> > +    DBBACKUP_STORAGE_OPTIONS = {'location': '.backups'}
> >  
> >  #
> >  # Patchwork settings
> > diff --git a/requirements-dev.txt b/requirements-dev.txt
> > index b0cdd0de..ed98c30e 100644
> > --- a/requirements-dev.txt
> > +++ b/requirements-dev.txt
> > @@ -3,4 +3,5 @@ Django==1.11.15; python_version < '3.0'  # pyup: ignore
> >  djangorestframework==3.8.2
> >  django-filter==2.0.0; python_version >= '3.4'
> >  django-filter==1.1.0; python_version < '3.0'  # pyup: ignore
> > +django-dbbackup==3.2.0
> >  -r requirements-test.txt
> > -- 
> > 2.17.1
> > 
> > _______________________________________________
> > Patchwork mailing list
> > Patchwork@lists.ozlabs.org
> > https://lists.ozlabs.org/listinfo/patchwork
Daniel Axtens Oct. 9, 2018, 1:10 p.m. | #3
Stephen Finucane <stephen@that.guru> writes:

> On Tue, 2018-10-09 at 12:54 +1100, Daniel Axtens wrote:
>> Stephen Finucane <stephen@that.guru> writes:
>> 
>> > 'parsemail' and 'parsearchive' are slow. When messing with models and
>> > migrations, it can be very useful to backup the database in its current
>> > state and restore it if/when you mess up. Currently, we've documented
>> > how to do this via some commands run in the shell of the container, but
>> > we can do things easier via an application designed for this purpose:
>> > 'django-dbshell'.
>> 
>> You think it's better to include this dependency than to add a script to
>> tools/ that wraps mysqldump/mysqlrestore and the postgres equivalents?
>> I'm happy if you think this is the way to go, I just want to know you've
>> thought about it.
>
> Yeah, I was going to write a new management command until I decided to
> go with this. This is only installed and enabled as part of the dev
> configuration so I figure the impact is minimal enough and we benefit
> from someone else testing this for us :)
>
>> > 
>> > Signed-off-by: Stephen Finucane <stephen@that.guru>
>> > ---
>> >  .gitignore                |  3 +++
>> >  patchwork/settings/dev.py | 13 +++++++++++++
>> >  requirements-dev.txt      |  1 +
>> 
>> Does this need docs added somewhere? I hadn't heard of django-dbshell
>> before this...
>
> Hmm, possibly? I purposefully avoided noting this in the release notes
> since it doesn't really affect real-world users. That said, I did note
> when we added django-debug-toolbar (though not any subsequent updates)
> so I could be talked around. In any case, I will add a follow-on patch
> to update the docs to refer to this tool instead of manual
> backups/restores.

All I wanted was something in the docs, thanks for the follow-up.

Regards,
Daniel

>
>> >  3 files changed, 17 insertions(+)
>> > 
>> > diff --git a/.gitignore b/.gitignore
>> > index 57376bdd..a33d1029 100644
>> > --- a/.gitignore
>> > +++ b/.gitignore
>> > @@ -54,3 +54,6 @@ patchwork/settings/production.py
>> >  
>> >  # docker-compose configuration files
>> >  /.env
>> > +
>> > +# django-dbbackup files
>> 
>> Also, here you've called it django-dbbackup, in the commit message it's
>> django-dbshell. It that correct?
>
> Yeah, the commit message is wrong. Can fix if/when applying.
>
> Stephen
>
>> Apart from that, I'm very much an ACK for the idea of making backups
>> easier.
>> 
>> Regards,
>> Daniel
>> > +/.backups
>> > diff --git a/patchwork/settings/dev.py b/patchwork/settings/dev.py
>> > index 4bb5a93c..711177c9 100644
>> > --- a/patchwork/settings/dev.py
>> > +++ b/patchwork/settings/dev.py
>> > @@ -9,6 +9,11 @@ Design based on:
>> >  
>> >  from .base import *  # noqa
>> >  
>> > +try:
>> > +    import dbbackup
>> > +except ImportError:
>> > +    dbbackup = None
>> > +
>> >  #
>> >  # Core settings
>> >  # https://docs.djangoproject.com/en/1.11/ref/settings/#core-settings
>> > @@ -72,6 +77,14 @@ INTERNAL_IPS = [
>> >      '172.18.0.1'
>> >  ]
>> >  
>> > +# django-dbbackup
>> > +
>> > +if dbbackup:
>> > +    INSTALLED_APPS += [
>> > +        'dbbackup',
>> > +    ]
>> > +
>> > +    DBBACKUP_STORAGE_OPTIONS = {'location': '.backups'}
>> >  
>> >  #
>> >  # Patchwork settings
>> > diff --git a/requirements-dev.txt b/requirements-dev.txt
>> > index b0cdd0de..ed98c30e 100644
>> > --- a/requirements-dev.txt
>> > +++ b/requirements-dev.txt
>> > @@ -3,4 +3,5 @@ Django==1.11.15; python_version < '3.0'  # pyup: ignore
>> >  djangorestframework==3.8.2
>> >  django-filter==2.0.0; python_version >= '3.4'
>> >  django-filter==1.1.0; python_version < '3.0'  # pyup: ignore
>> > +django-dbbackup==3.2.0
>> >  -r requirements-test.txt
>> > -- 
>> > 2.17.1
>> > 
>> > _______________________________________________
>> > Patchwork mailing list
>> > Patchwork@lists.ozlabs.org
>> > https://lists.ozlabs.org/listinfo/patchwork

Patch

diff --git a/.gitignore b/.gitignore
index 57376bdd..a33d1029 100644
--- a/.gitignore
+++ b/.gitignore
@@ -54,3 +54,6 @@  patchwork/settings/production.py
 
 # docker-compose configuration files
 /.env
+
+# django-dbbackup files
+/.backups
diff --git a/patchwork/settings/dev.py b/patchwork/settings/dev.py
index 4bb5a93c..711177c9 100644
--- a/patchwork/settings/dev.py
+++ b/patchwork/settings/dev.py
@@ -9,6 +9,11 @@  Design based on:
 
 from .base import *  # noqa
 
+try:
+    import dbbackup
+except ImportError:
+    dbbackup = None
+
 #
 # Core settings
 # https://docs.djangoproject.com/en/1.11/ref/settings/#core-settings
@@ -72,6 +77,14 @@  INTERNAL_IPS = [
     '172.18.0.1'
 ]
 
+# django-dbbackup
+
+if dbbackup:
+    INSTALLED_APPS += [
+        'dbbackup',
+    ]
+
+    DBBACKUP_STORAGE_OPTIONS = {'location': '.backups'}
 
 #
 # Patchwork settings
diff --git a/requirements-dev.txt b/requirements-dev.txt
index b0cdd0de..ed98c30e 100644
--- a/requirements-dev.txt
+++ b/requirements-dev.txt
@@ -3,4 +3,5 @@  Django==1.11.15; python_version < '3.0'  # pyup: ignore
 djangorestframework==3.8.2
 django-filter==2.0.0; python_version >= '3.4'
 django-filter==1.1.0; python_version < '3.0'  # pyup: ignore
+django-dbbackup==3.2.0
 -r requirements-test.txt