From patchwork Tue Apr 14 06:21:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Axtens X-Patchwork-Id: 1270128 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 491b432gHbz9sSt for ; Tue, 14 Apr 2020 16:21:55 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=axtens.net Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=axtens.net header.i=@axtens.net header.a=rsa-sha256 header.s=google header.b=WKGoiyxZ; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 491b426ZL3zDqWW for ; Tue, 14 Apr 2020 16:21:54 +1000 (AEST) X-Original-To: patchwork@lists.ozlabs.org Delivered-To: patchwork@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=axtens.net (client-ip=2607:f8b0:4864:20::632; helo=mail-pl1-x632.google.com; envelope-from=dja@axtens.net; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=axtens.net Authentication-Results: lists.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=axtens.net header.i=@axtens.net header.a=rsa-sha256 header.s=google header.b=WKGoiyxZ; dkim-atps=neutral Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 491b3F4NPRzDqWg for ; Tue, 14 Apr 2020 16:21:13 +1000 (AEST) Received: by mail-pl1-x632.google.com with SMTP id t4so4278072plq.12 for ; Mon, 13 Apr 2020 23:21:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=axtens.net; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7p8NqHWlqn/8EVKJIVOnuBcZ58K34zBz1eRiBPP/1IM=; b=WKGoiyxZ6Hzld/ehp3r8J9XHcqYt1GncoMvEIwC4ITgF+EeSMKSCpaQgwbrTDyMRlB SDxQ4hbfSpcGBD+65a8t4O9cg7r7qh89SN6MS03fn046cjqcHMkCyguZ9pfE35wbQcT9 6TT+cH4dp7DMaHyKWoaBPccNHRI6tRDy0QpH0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7p8NqHWlqn/8EVKJIVOnuBcZ58K34zBz1eRiBPP/1IM=; b=W2JwdIAoPDhdlddl8H1gHBOMU0ylYieR54gi1GGZPkI4ZvhuS35o0eF3yzrgTvLGPg 5cEPKODqYrMx4ngqsyc8d4lNzDlUh9QOeF88BGvC2SGX0fOqnL1Gh6+6iRMRaI5KXcHK KjVp8HeCcmTq9CUbxT6hGRTc8Vq5tWZUQzGz/YQnxGSvhzgO34XDJFJvvFVEPLFXrzh+ YEF7hTXztditW3SQQ/23adv0XSEsVIbgJnvdSm2Mqlf1joFaRKZ4OfE4Jjaf+gMQJEEG Hhx7H8EIgDXLEbRbadmuVqiU+s8dm2mlaI43k4e597KsTPBD5Pv1nZqWFx+xkKExrJ8d L0Kw== X-Gm-Message-State: AGi0PuZBzV1qgDI9sRSlwPaxslGX6+oHMBeOqhi5qbAn4aoEvGWEcrCd +0It75aHRgaF0fqhNBfPSyNaxXsOmeg= X-Google-Smtp-Source: APiQypJYg/I2NNjgnbZJtC7oiAaMcw8wd9B2WbGL7Uz3o6q6Mla3fQIiatD4CUbFgCt2SauLyHWiyg== X-Received: by 2002:a17:902:8506:: with SMTP id bj6mr21236597plb.106.1586845270364; Mon, 13 Apr 2020 23:21:10 -0700 (PDT) Received: from localhost (2001-44b8-1113-6700-65ed-ce52-e9b0-0e8c.static.ipv6.internode.on.net. [2001:44b8:1113:6700:65ed:ce52:e9b0:e8c]) by smtp.gmail.com with ESMTPSA id x16sm7702964pfm.146.2020.04.13.23.21.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Apr 2020 23:21:09 -0700 (PDT) From: Daniel Axtens To: patchwork@lists.ozlabs.org Subject: [PATCH 1/2] api: do not fetch every patch in a patch detail view 404 Date: Tue, 14 Apr 2020 16:21:01 +1000 Message-Id: <20200414062102.6798-2-dja@axtens.net> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200414062102.6798-1-dja@axtens.net> References: <20200414062102.6798-1-dja@axtens.net> MIME-Version: 1.0 X-BeenThere: patchwork@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Patchwork development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Ellerman , Stephen Rothwell , jk@ozlabs.org Errors-To: patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Patchwork" mpe and jk and sfr found that the OzLabs server was melting due to some queries downloading every patch. Turns out if you 404 the patch detail view in the API, d-r-f attempts to render a listbox with every single patch to fill in the 'related' field. The bundle API also has a similar field. Replace the multiple selection box with a text field. You can still (AIUI) populate the relevant patch IDs manually. Reported-by: Jeremy Kerr Reported-by: Michael Ellerman Reported-by: Stephen Rothwell Signed-off-by: Daniel Axtens Tested-by: Jeremy Kerr Reviewed-by: Stephen Finucane --- patchwork/api/bundle.py | 3 ++- patchwork/api/embedded.py | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/patchwork/api/bundle.py b/patchwork/api/bundle.py index b8c0f1781786..54a9266e7d73 100644 --- a/patchwork/api/bundle.py +++ b/patchwork/api/bundle.py @@ -62,7 +62,8 @@ class BundleSerializer(BaseHyperlinkedModelSerializer): project = ProjectSerializer(read_only=True) mbox = SerializerMethodField() owner = UserSerializer(read_only=True) - patches = PatchSerializer(many=True, required=True) + patches = PatchSerializer(many=True, required=True, + style={'base_template': 'input.html'}) def get_web_url(self, instance): request = self.context.get('request') diff --git a/patchwork/api/embedded.py b/patchwork/api/embedded.py index 85a30cae1cdf..cb3f07e6b998 100644 --- a/patchwork/api/embedded.py +++ b/patchwork/api/embedded.py @@ -141,7 +141,8 @@ class PatchSerializer(SerializedRelatedField): class PatchRelationSerializer(BaseHyperlinkedModelSerializer): """Hide the PatchRelation model, just show the list""" - patches = PatchSerializer(many=True) + patches = PatchSerializer(many=True, + style={'base_template': 'input.html'}) def to_internal_value(self, data): if not isinstance(data, type([])): From patchwork Tue Apr 14 06:21:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Axtens X-Patchwork-Id: 1270129 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 491b4T4gh0z9s71 for ; Tue, 14 Apr 2020 16:22:17 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=axtens.net Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=axtens.net header.i=@axtens.net header.a=rsa-sha256 header.s=google header.b=iPX+Yz5D; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 491b4R4Vg8zDqWw for ; Tue, 14 Apr 2020 16:22:15 +1000 (AEST) X-Original-To: patchwork@lists.ozlabs.org Delivered-To: patchwork@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=axtens.net (client-ip=2607:f8b0:4864:20::1041; helo=mail-pj1-x1041.google.com; envelope-from=dja@axtens.net; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=axtens.net Authentication-Results: lists.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=axtens.net header.i=@axtens.net header.a=rsa-sha256 header.s=google header.b=iPX+Yz5D; dkim-atps=neutral Received: from mail-pj1-x1041.google.com (mail-pj1-x1041.google.com [IPv6:2607:f8b0:4864:20::1041]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 491b3K3xjzzDqTL for ; Tue, 14 Apr 2020 16:21:17 +1000 (AEST) Received: by mail-pj1-x1041.google.com with SMTP id mn19so4798786pjb.0 for ; Mon, 13 Apr 2020 23:21:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=axtens.net; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mi1ZClZg5nzBCZ/0N8U9CBWAZlsb9xwB81QDZX+eBuI=; b=iPX+Yz5DoQFXNjLneM+UykOQc7uejn+FtjwMRHLtgut+e/jxCJbtNemmrkVA/38U2c t1X02TT81sNaymW1DemjmjPoPxu0sSLAt2iXzpVmIRzRyPXCWZ9JLZtM6ZNk0S/+5nSY 3ba8ICmCig4O8cjqP1Q0GGrlXRwLpMkFQFnyc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mi1ZClZg5nzBCZ/0N8U9CBWAZlsb9xwB81QDZX+eBuI=; b=pMOQ7Z/fBuci5O1AIGPRG8yrxE8aVhppZhlTq0PBcpzTS4OHOQYxj4Kkt6Kt0/YAx2 8XyObKi9pd24dnQXT+rjCPJwOSJmdgsebkAlxqLAjJTJ4uIn6GAgHxFCVGBhzfQn5Vyo sWnv/0TT/ZgykLij+fxJwEr/HusblwcAz9QHZG+4xpNsiEjZbe2Xzt7t2soKQCoOeTBn j8aMFjsKvIrXEonqMP5QnLmKQgv7A0PWb5i/x7zXnJCps5nuP+OyPAhfg1f1zAM36/J6 bj5LRKvmwj5EpXoWsD96DmepKSX1MS/7ghTbhe/8dTqQ91YB0cIiOyMNYVg0VMDxmfcs OnBA== X-Gm-Message-State: AGi0PuZugW2vb33+enhMGegPT26lJe7zS5X5y8YwO52uTM97AgxG5N9T T6DB7BZ3tVaeAdPV+jx7coDZlOO7wTY= X-Google-Smtp-Source: APiQypJl2Ras5wETAIEw9orW1mrS/xrrpA+5lVhQCCpl5FtdnQf/WL3Ye4PbUbW9btVApAmLXuaqPg== X-Received: by 2002:a17:90a:364c:: with SMTP id s70mr25652975pjb.143.1586845274133; Mon, 13 Apr 2020 23:21:14 -0700 (PDT) Received: from localhost (2001-44b8-1113-6700-65ed-ce52-e9b0-0e8c.static.ipv6.internode.on.net. [2001:44b8:1113:6700:65ed:ce52:e9b0:e8c]) by smtp.gmail.com with ESMTPSA id q63sm1127445pfb.178.2020.04.13.23.21.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Apr 2020 23:21:13 -0700 (PDT) From: Daniel Axtens To: patchwork@lists.ozlabs.org Subject: [PATCH 2/2] api: allow filtering patches and covers by msgid Date: Tue, 14 Apr 2020 16:21:02 +1000 Message-Id: <20200414062102.6798-3-dja@axtens.net> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200414062102.6798-1-dja@axtens.net> References: <20200414062102.6798-1-dja@axtens.net> MIME-Version: 1.0 X-BeenThere: patchwork@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Patchwork development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Ellerman , jk@ozlabs.org Errors-To: patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Patchwork" In the process of fixing the previous bug, I realised that: a) /api/patches/msgid is a perfectly reasonable thing to attempt b) We have no way of finding a patch by message id in the API We can't actualy make /api/patches/msgid work because it may not be unique, but we can add a filter. I'm shoehorning this into stable/2.2, even though it's technically an API change: it's minor, not incompatible and in hindsight a glaring hole. Cc: Michael Ellerman Signed-off-by: Daniel Axtens Tested-by: Jeremy Kerr Reviewed-by: Andrew Donnellan Reviewed-by: Stephen Finucane --- docs/api/schemas/latest/patchwork.yaml | 16 ++++++++++++++++ docs/api/schemas/patchwork.j2 | 18 ++++++++++++++++++ docs/api/schemas/v1.2/patchwork.yaml | 16 ++++++++++++++++ patchwork/api/filters.py | 14 ++++++++++---- patchwork/tests/api/test_cover.py | 12 ++++++++++++ patchwork/tests/api/test_patch.py | 12 ++++++++++++ .../rest-filter-msgid-41f693cd4e53cf93.yaml | 6 ++++++ 7 files changed, 90 insertions(+), 4 deletions(-) create mode 100644 releasenotes/notes/rest-filter-msgid-41f693cd4e53cf93.yaml diff --git a/docs/api/schemas/latest/patchwork.yaml b/docs/api/schemas/latest/patchwork.yaml index 13cdc9cd78fd..cc0d97e696b6 100644 --- a/docs/api/schemas/latest/patchwork.yaml +++ b/docs/api/schemas/latest/patchwork.yaml @@ -246,6 +246,14 @@ paths: schema: title: '' type: string + - in: query + name: msgid + description: > + The cover message-id as a case-sensitive string, without leading or + trailing angle brackets, to filter by. + schema: + title: '' + type: string responses: '200': description: '' @@ -474,6 +482,14 @@ paths: schema: title: '' type: string + - in: query + name: msgid + description: > + The patch message-id as a case-sensitive string, without leading or + trailing angle brackets, to filter by. + schema: + title: '' + type: string responses: '200': description: '' diff --git a/docs/api/schemas/patchwork.j2 b/docs/api/schemas/patchwork.j2 index bd714d5e7a2a..f5618d41faa0 100644 --- a/docs/api/schemas/patchwork.j2 +++ b/docs/api/schemas/patchwork.j2 @@ -251,6 +251,16 @@ paths: schema: title: '' type: string +{% if version >= (1, 2) %} + - in: query + name: msgid + description: > + The cover message-id as a case-sensitive string, without leading or + trailing angle brackets, to filter by. + schema: + title: '' + type: string +{% endif %} responses: '200': description: '' @@ -488,6 +498,14 @@ paths: schema: title: '' type: string + - in: query + name: msgid + description: > + The patch message-id as a case-sensitive string, without leading or + trailing angle brackets, to filter by. + schema: + title: '' + type: string {% endif %} responses: '200': diff --git a/docs/api/schemas/v1.2/patchwork.yaml b/docs/api/schemas/v1.2/patchwork.yaml index db2ed122eec2..7bdbe66997c0 100644 --- a/docs/api/schemas/v1.2/patchwork.yaml +++ b/docs/api/schemas/v1.2/patchwork.yaml @@ -246,6 +246,14 @@ paths: schema: title: '' type: string + - in: query + name: msgid + description: > + The cover message-id as a case-sensitive string, without leading or + trailing angle brackets, to filter by. + schema: + title: '' + type: string responses: '200': description: '' @@ -474,6 +482,14 @@ paths: schema: title: '' type: string + - in: query + name: msgid + description: > + The patch message-id as a case-sensitive string, without leading or + trailing angle brackets, to filter by. + schema: + title: '' + type: string responses: '200': description: '' diff --git a/patchwork/api/filters.py b/patchwork/api/filters.py index deb5ace11880..93e6281bf5e6 100644 --- a/patchwork/api/filters.py +++ b/patchwork/api/filters.py @@ -184,6 +184,10 @@ class SeriesFilterSet(TimestampMixin, BaseFilterSet): fields = ('submitter', 'project') +def msgid_filter(queryset, name, value): + return queryset.filter(**{name: '<' + value + '>'}) + + class CoverLetterFilterSet(TimestampMixin, BaseFilterSet): project = ProjectFilter(queryset=Project.objects.all(), distinct=False) @@ -192,6 +196,7 @@ class CoverLetterFilterSet(TimestampMixin, BaseFilterSet): series = BaseFilter(queryset=Project.objects.all(), widget=MultipleHiddenInput, distinct=False) submitter = PersonFilter(queryset=Person.objects.all(), distinct=False) + msgid = CharFilter(method=msgid_filter) class Meta: model = CoverLetter @@ -210,17 +215,18 @@ class PatchFilterSet(TimestampMixin, BaseFilterSet): delegate = UserFilter(queryset=User.objects.all(), distinct=False) state = StateFilter(queryset=State.objects.all(), distinct=False) hash = CharFilter(lookup_expr='iexact') + msgid = CharFilter(method=msgid_filter) class Meta: model = Patch - # NOTE(dja): ideally we want to version the hash field, but I cannot - # find a way to do that which is reliable and not extremely ugly. + # NOTE(dja): ideally we want to version the hash/msgid field, but I + # can't find a way to do that which is reliable and not extremely ugly. # The best I can come up with is manually working with request.GET # which seems to rather defeat the point of using django-filters. fields = ('project', 'series', 'submitter', 'delegate', - 'state', 'archived', 'hash') + 'state', 'archived', 'hash', 'msgid') versioned_fields = { - '1.2': ('hash', ), + '1.2': ('hash', 'msgid'), } diff --git a/patchwork/tests/api/test_cover.py b/patchwork/tests/api/test_cover.py index 5eeb1902e1d1..1b19ded1b4d5 100644 --- a/patchwork/tests/api/test_cover.py +++ b/patchwork/tests/api/test_cover.py @@ -111,6 +111,18 @@ class TestCoverLetterAPI(utils.APITestCase): 'submitter': 'test@example.org'}) self.assertEqual(0, len(resp.data)) + def test_list_filter_msgid(self): + """Filter covers by msgid.""" + cover = create_cover() + + resp = self.client.get(self.api_url(), {'msgid': cover.url_msgid}) + self.assertEqual([cover.id], [x['id'] for x in resp.data]) + + # empty response if nothing matches + resp = self.client.get(self.api_url(), { + 'msgid': 'fishfish@fish.fish'}) + self.assertEqual(0, len(resp.data)) + @utils.store_samples('cover-list-1-0') def test_list_version_1_0(self): create_cover() diff --git a/patchwork/tests/api/test_patch.py b/patchwork/tests/api/test_patch.py index b24c5ab28947..da2dd6e9084b 100644 --- a/patchwork/tests/api/test_patch.py +++ b/patchwork/tests/api/test_patch.py @@ -199,6 +199,18 @@ class TestPatchAPI(utils.APITestCase): {'hash': 'garbagevalue'}) self.assertEqual(1, len(resp.data)) + def test_list_filter_msgid(self): + """Filter patches by msgid.""" + patch = self._create_patch() + + resp = self.client.get(self.api_url(), {'msgid': patch.url_msgid}) + self.assertEqual([patch.id], [x['id'] for x in resp.data]) + + # empty response if nothing matches + resp = self.client.get(self.api_url(), { + 'msgid': 'fishfish@fish.fish'}) + self.assertEqual(0, len(resp.data)) + @utils.store_samples('patch-list-1-0') def test_list_version_1_0(self): """List patches using API v1.0.""" diff --git a/releasenotes/notes/rest-filter-msgid-41f693cd4e53cf93.yaml b/releasenotes/notes/rest-filter-msgid-41f693cd4e53cf93.yaml new file mode 100644 index 000000000000..0fcbbeb8a736 --- /dev/null +++ b/releasenotes/notes/rest-filter-msgid-41f693cd4e53cf93.yaml @@ -0,0 +1,6 @@ +--- +api: + - | + The REST API now supports filtering patches and cover letters by message + ID, using the ``msgid`` query parameter. Don't include leading or trailing + angle brackets.