| Message ID | 20181006214506.22720-4-stephen@that.guru |
|---|---|
| State | Accepted |
| Headers | show |
| Series | Add dbbackup, dbrestore commands | expand |
Stephen Finucane <stephen@that.guru> writes: > 'parsemail' and 'parsearchive' are slow. When messing with models and > migrations, it can be very useful to backup the database in its current > state and restore it if/when you mess up. Currently, we've documented > how to do this via some commands run in the shell of the container, but > we can do things easier via an application designed for this purpose: > 'django-dbshell'. You think it's better to include this dependency than to add a script to tools/ that wraps mysqldump/mysqlrestore and the postgres equivalents? I'm happy if you think this is the way to go, I just want to know you've thought about it. > > Signed-off-by: Stephen Finucane <stephen@that.guru> > --- > .gitignore | 3 +++ > patchwork/settings/dev.py | 13 +++++++++++++ > requirements-dev.txt | 1 + Does this need docs added somewhere? I hadn't heard of django-dbshell before this... > 3 files changed, 17 insertions(+) > > diff --git a/.gitignore b/.gitignore > index 57376bdd..a33d1029 100644 > --- a/.gitignore > +++ b/.gitignore > @@ -54,3 +54,6 @@ patchwork/settings/production.py > > # docker-compose configuration files > /.env > + > +# django-dbbackup files Also, here you've called it django-dbbackup, in the commit message it's django-dbshell. It that correct? Apart from that, I'm very much an ACK for the idea of making backups easier. Regards, Daniel > +/.backups > diff --git a/patchwork/settings/dev.py b/patchwork/settings/dev.py > index 4bb5a93c..711177c9 100644 > --- a/patchwork/settings/dev.py > +++ b/patchwork/settings/dev.py > @@ -9,6 +9,11 @@ Design based on: > > from .base import * # noqa > > +try: > + import dbbackup > +except ImportError: > + dbbackup = None > + > # > # Core settings > # https://docs.djangoproject.com/en/1.11/ref/settings/#core-settings > @@ -72,6 +77,14 @@ INTERNAL_IPS = [ > '172.18.0.1' > ] > > +# django-dbbackup > + > +if dbbackup: > + INSTALLED_APPS += [ > + 'dbbackup', > + ] > + > + DBBACKUP_STORAGE_OPTIONS = {'location': '.backups'} > > # > # Patchwork settings > diff --git a/requirements-dev.txt b/requirements-dev.txt > index b0cdd0de..ed98c30e 100644 > --- a/requirements-dev.txt > +++ b/requirements-dev.txt > @@ -3,4 +3,5 @@ Django==1.11.15; python_version < '3.0' # pyup: ignore > djangorestframework==3.8.2 > django-filter==2.0.0; python_version >= '3.4' > django-filter==1.1.0; python_version < '3.0' # pyup: ignore > +django-dbbackup==3.2.0 > -r requirements-test.txt > -- > 2.17.1 > > _______________________________________________ > Patchwork mailing list > Patchwork@lists.ozlabs.org > https://lists.ozlabs.org/listinfo/patchwork
On Tue, 2018-10-09 at 12:54 +1100, Daniel Axtens wrote: > Stephen Finucane <stephen@that.guru> writes: > > > 'parsemail' and 'parsearchive' are slow. When messing with models and > > migrations, it can be very useful to backup the database in its current > > state and restore it if/when you mess up. Currently, we've documented > > how to do this via some commands run in the shell of the container, but > > we can do things easier via an application designed for this purpose: > > 'django-dbshell'. > > You think it's better to include this dependency than to add a script to > tools/ that wraps mysqldump/mysqlrestore and the postgres equivalents? > I'm happy if you think this is the way to go, I just want to know you've > thought about it. Yeah, I was going to write a new management command until I decided to go with this. This is only installed and enabled as part of the dev configuration so I figure the impact is minimal enough and we benefit from someone else testing this for us :) > > > > Signed-off-by: Stephen Finucane <stephen@that.guru> > > --- > > .gitignore | 3 +++ > > patchwork/settings/dev.py | 13 +++++++++++++ > > requirements-dev.txt | 1 + > > Does this need docs added somewhere? I hadn't heard of django-dbshell > before this... Hmm, possibly? I purposefully avoided noting this in the release notes since it doesn't really affect real-world users. That said, I did note when we added django-debug-toolbar (though not any subsequent updates) so I could be talked around. In any case, I will add a follow-on patch to update the docs to refer to this tool instead of manual backups/restores. > > 3 files changed, 17 insertions(+) > > > > diff --git a/.gitignore b/.gitignore > > index 57376bdd..a33d1029 100644 > > --- a/.gitignore > > +++ b/.gitignore > > @@ -54,3 +54,6 @@ patchwork/settings/production.py > > > > # docker-compose configuration files > > /.env > > + > > +# django-dbbackup files > > Also, here you've called it django-dbbackup, in the commit message it's > django-dbshell. It that correct? Yeah, the commit message is wrong. Can fix if/when applying. Stephen > Apart from that, I'm very much an ACK for the idea of making backups > easier. > > Regards, > Daniel > > +/.backups > > diff --git a/patchwork/settings/dev.py b/patchwork/settings/dev.py > > index 4bb5a93c..711177c9 100644 > > --- a/patchwork/settings/dev.py > > +++ b/patchwork/settings/dev.py > > @@ -9,6 +9,11 @@ Design based on: > > > > from .base import * # noqa > > > > +try: > > + import dbbackup > > +except ImportError: > > + dbbackup = None > > + > > # > > # Core settings > > # https://docs.djangoproject.com/en/1.11/ref/settings/#core-settings > > @@ -72,6 +77,14 @@ INTERNAL_IPS = [ > > '172.18.0.1' > > ] > > > > +# django-dbbackup > > + > > +if dbbackup: > > + INSTALLED_APPS += [ > > + 'dbbackup', > > + ] > > + > > + DBBACKUP_STORAGE_OPTIONS = {'location': '.backups'} > > > > # > > # Patchwork settings > > diff --git a/requirements-dev.txt b/requirements-dev.txt > > index b0cdd0de..ed98c30e 100644 > > --- a/requirements-dev.txt > > +++ b/requirements-dev.txt > > @@ -3,4 +3,5 @@ Django==1.11.15; python_version < '3.0' # pyup: ignore > > djangorestframework==3.8.2 > > django-filter==2.0.0; python_version >= '3.4' > > django-filter==1.1.0; python_version < '3.0' # pyup: ignore > > +django-dbbackup==3.2.0 > > -r requirements-test.txt > > -- > > 2.17.1 > > > > _______________________________________________ > > Patchwork mailing list > > Patchwork@lists.ozlabs.org > > https://lists.ozlabs.org/listinfo/patchwork
Stephen Finucane <stephen@that.guru> writes: > On Tue, 2018-10-09 at 12:54 +1100, Daniel Axtens wrote: >> Stephen Finucane <stephen@that.guru> writes: >> >> > 'parsemail' and 'parsearchive' are slow. When messing with models and >> > migrations, it can be very useful to backup the database in its current >> > state and restore it if/when you mess up. Currently, we've documented >> > how to do this via some commands run in the shell of the container, but >> > we can do things easier via an application designed for this purpose: >> > 'django-dbshell'. >> >> You think it's better to include this dependency than to add a script to >> tools/ that wraps mysqldump/mysqlrestore and the postgres equivalents? >> I'm happy if you think this is the way to go, I just want to know you've >> thought about it. > > Yeah, I was going to write a new management command until I decided to > go with this. This is only installed and enabled as part of the dev > configuration so I figure the impact is minimal enough and we benefit > from someone else testing this for us :) > >> > >> > Signed-off-by: Stephen Finucane <stephen@that.guru> >> > --- >> > .gitignore | 3 +++ >> > patchwork/settings/dev.py | 13 +++++++++++++ >> > requirements-dev.txt | 1 + >> >> Does this need docs added somewhere? I hadn't heard of django-dbshell >> before this... > > Hmm, possibly? I purposefully avoided noting this in the release notes > since it doesn't really affect real-world users. That said, I did note > when we added django-debug-toolbar (though not any subsequent updates) > so I could be talked around. In any case, I will add a follow-on patch > to update the docs to refer to this tool instead of manual > backups/restores. All I wanted was something in the docs, thanks for the follow-up. Regards, Daniel > >> > 3 files changed, 17 insertions(+) >> > >> > diff --git a/.gitignore b/.gitignore >> > index 57376bdd..a33d1029 100644 >> > --- a/.gitignore >> > +++ b/.gitignore >> > @@ -54,3 +54,6 @@ patchwork/settings/production.py >> > >> > # docker-compose configuration files >> > /.env >> > + >> > +# django-dbbackup files >> >> Also, here you've called it django-dbbackup, in the commit message it's >> django-dbshell. It that correct? > > Yeah, the commit message is wrong. Can fix if/when applying. > > Stephen > >> Apart from that, I'm very much an ACK for the idea of making backups >> easier. >> >> Regards, >> Daniel >> > +/.backups >> > diff --git a/patchwork/settings/dev.py b/patchwork/settings/dev.py >> > index 4bb5a93c..711177c9 100644 >> > --- a/patchwork/settings/dev.py >> > +++ b/patchwork/settings/dev.py >> > @@ -9,6 +9,11 @@ Design based on: >> > >> > from .base import * # noqa >> > >> > +try: >> > + import dbbackup >> > +except ImportError: >> > + dbbackup = None >> > + >> > # >> > # Core settings >> > # https://docs.djangoproject.com/en/1.11/ref/settings/#core-settings >> > @@ -72,6 +77,14 @@ INTERNAL_IPS = [ >> > '172.18.0.1' >> > ] >> > >> > +# django-dbbackup >> > + >> > +if dbbackup: >> > + INSTALLED_APPS += [ >> > + 'dbbackup', >> > + ] >> > + >> > + DBBACKUP_STORAGE_OPTIONS = {'location': '.backups'} >> > >> > # >> > # Patchwork settings >> > diff --git a/requirements-dev.txt b/requirements-dev.txt >> > index b0cdd0de..ed98c30e 100644 >> > --- a/requirements-dev.txt >> > +++ b/requirements-dev.txt >> > @@ -3,4 +3,5 @@ Django==1.11.15; python_version < '3.0' # pyup: ignore >> > djangorestframework==3.8.2 >> > django-filter==2.0.0; python_version >= '3.4' >> > django-filter==1.1.0; python_version < '3.0' # pyup: ignore >> > +django-dbbackup==3.2.0 >> > -r requirements-test.txt >> > -- >> > 2.17.1 >> > >> > _______________________________________________ >> > Patchwork mailing list >> > Patchwork@lists.ozlabs.org >> > https://lists.ozlabs.org/listinfo/patchwork
diff --git a/.gitignore b/.gitignore index 57376bdd..a33d1029 100644 --- a/.gitignore +++ b/.gitignore @@ -54,3 +54,6 @@ patchwork/settings/production.py # docker-compose configuration files /.env + +# django-dbbackup files +/.backups diff --git a/patchwork/settings/dev.py b/patchwork/settings/dev.py index 4bb5a93c..711177c9 100644 --- a/patchwork/settings/dev.py +++ b/patchwork/settings/dev.py @@ -9,6 +9,11 @@ Design based on: from .base import * # noqa +try: + import dbbackup +except ImportError: + dbbackup = None + # # Core settings # https://docs.djangoproject.com/en/1.11/ref/settings/#core-settings @@ -72,6 +77,14 @@ INTERNAL_IPS = [ '172.18.0.1' ] +# django-dbbackup + +if dbbackup: + INSTALLED_APPS += [ + 'dbbackup', + ] + + DBBACKUP_STORAGE_OPTIONS = {'location': '.backups'} # # Patchwork settings diff --git a/requirements-dev.txt b/requirements-dev.txt index b0cdd0de..ed98c30e 100644 --- a/requirements-dev.txt +++ b/requirements-dev.txt @@ -3,4 +3,5 @@ Django==1.11.15; python_version < '3.0' # pyup: ignore djangorestframework==3.8.2 django-filter==2.0.0; python_version >= '3.4' django-filter==1.1.0; python_version < '3.0' # pyup: ignore +django-dbbackup==3.2.0 -r requirements-test.txt
'parsemail' and 'parsearchive' are slow. When messing with models and migrations, it can be very useful to backup the database in its current state and restore it if/when you mess up. Currently, we've documented how to do this via some commands run in the shell of the container, but we can do things easier via an application designed for this purpose: 'django-dbshell'. Signed-off-by: Stephen Finucane <stephen@that.guru> --- .gitignore | 3 +++ patchwork/settings/dev.py | 13 +++++++++++++ requirements-dev.txt | 1 + 3 files changed, 17 insertions(+)