[RFC,0/4] Add 'Event.payload' field

The '/event' API endpoint is really slow due to the amount of JOINs
necessary to retrieve records from the database. Resolve this by
a static JSON representation of any embedded data in the database. This
has some disadvantages, noted in the patches, but the performance
improvement is huge and users will not notice a thing.

There are a couple of TODOs in here that I'd like to work on. In
addition, this uses Django REST Framework's serializers to generate
the embedded payload. This means we have to do one of the following:

- Require 'django-rest-framework' as a dependency, even if the actual
  API is turned off
- Disable events when the REST API is disabled

I welcome input on the above along with the patches themselves.

Stephen Finucane (4):
  Automatically remove old events
  REST: Support embedded serializers without context
  models: Migrate event fields to JSON field
  REST: Only fetch required fields event filtering

 docs/deployment/configuration.rst                  |   7 ++
 docs/usage/overview.rst                            |   2 +
 patchwork/api/embedded.py                          |  60 ++++++++---
 patchwork/api/event.py                             | 110 +++++++++++++--------
 patchwork/api/filters.py                           |   7 ++
 patchwork/fields.py                                |  32 ++++++
 patchwork/management/commands/cron.py              |   2 +
 .../migrations/0021_add_event_payload_field.py     |  21 ++++
 ...22_migrate_data_from_event_fields_to_payload.py |  67 +++++++++++++
 .../migrations/0023_remove_old_event_fields.py     |  43 ++++++++
 patchwork/models.py                                |  28 +-----
 patchwork/notifications.py                         |  18 +++-
 patchwork/settings/base.py                         |   3 +
 patchwork/signals.py                               |  48 +++++++++
 patchwork/tests/test_events.py                     | 110 +++++++++++++--------
 ...tically-remove-old-events-4ee222aaf4a6ea6c.yaml |   8 ++
 16 files changed, 443 insertions(+), 123 deletions(-)
 create mode 100644 patchwork/migrations/0021_add_event_payload_field.py
 create mode 100644 patchwork/migrations/0022_migrate_data_from_event_fields_to_payload.py
 create mode 100644 patchwork/migrations/0023_remove_old_event_fields.py
 create mode 100644 releasenotes/notes/automatically-remove-old-events-4ee222aaf4a6ea6c.yaml

Stephen Finucane <stephen@that.guru> writes:

> The '/event' API endpoint is really slow due to the amount of JOINs
> necessary to retrieve records from the database. Resolve this by
> a static JSON representation of any embedded data in the database. This
> has some disadvantages, noted in the patches, but the performance
> improvement is huge and users will not notice a thing.
Interesing approach - let's have a look!
>
> There are a couple of TODOs in here that I'd like to work on. In
> addition, this uses Django REST Framework's serializers to generate
> the embedded payload. This means we have to do one of the following:
>
> - Require 'django-rest-framework' as a dependency, even if the actual
>   API is turned off
> - Disable events when the REST API is disabled

This second one would be the way to go.
>
> I welcome input on the above along with the patches themselves.
>
Looking through them now.

Regards,
Daniel

> Stephen Finucane (4):
>   Automatically remove old events
>   REST: Support embedded serializers without context
>   models: Migrate event fields to JSON field
>   REST: Only fetch required fields event filtering
>
>  docs/deployment/configuration.rst                  |   7 ++
>  docs/usage/overview.rst                            |   2 +
>  patchwork/api/embedded.py                          |  60 ++++++++---
>  patchwork/api/event.py                             | 110 +++++++++++++--------
>  patchwork/api/filters.py                           |   7 ++
>  patchwork/fields.py                                |  32 ++++++
>  patchwork/management/commands/cron.py              |   2 +
>  .../migrations/0021_add_event_payload_field.py     |  21 ++++
>  ...22_migrate_data_from_event_fields_to_payload.py |  67 +++++++++++++
>  .../migrations/0023_remove_old_event_fields.py     |  43 ++++++++
>  patchwork/models.py                                |  28 +-----
>  patchwork/notifications.py                         |  18 +++-
>  patchwork/settings/base.py                         |   3 +
>  patchwork/signals.py                               |  48 +++++++++
>  patchwork/tests/test_events.py                     | 110 +++++++++++++--------
>  ...tically-remove-old-events-4ee222aaf4a6ea6c.yaml |   8 ++
>  16 files changed, 443 insertions(+), 123 deletions(-)
>  create mode 100644 patchwork/migrations/0021_add_event_payload_field.py
>  create mode 100644 patchwork/migrations/0022_migrate_data_from_event_fields_to_payload.py
>  create mode 100644 patchwork/migrations/0023_remove_old_event_fields.py
>  create mode 100644 releasenotes/notes/automatically-remove-old-events-4ee222aaf4a6ea6c.yaml
>
> -- 
> 2.14.3
>
> _______________________________________________
> Patchwork mailing list
> Patchwork@lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/patchwork