| Message ID | 20181030111916.7342-1-stephen@that.guru |
|---|---|
| Headers | show |
| Series | Add OpenAPI 3.0.0 REST API spec | expand |
Stephen Finucane <stephen@that.guru> writes: > Round 2. When we first added the REST API, attempts were made to > document this using OpenAPI 2.0 (f.k.a. Swagger). These floundered owing > to the tediousness of writing said spec from scratch. Thankfully, tools > have emerged [1][2] that allow us to generate specs from the code. The > spec introduced here is based on the combined output of these tools, > with some additional tweaks to work around gaps in the generated output > (the embedded serializers in particular caused a lot of fuss). > So I love the idea of generating code, because without users of the generated spec it's very easy to forget to do the manual work. But, you've mentioned that there is work that needs to be done on top of the generated code. How far down the auto-generation path are we, and what would need to be different in order to have it fully automated? I will have a look at the rest of the series. Regards, Daniel PS. I am travelling next week and then on holiday for another week and a bit, so I may either get lots done or nothing at all. Sorry! > The end result is something that we can use to validate the API, > validate clients and generate documentation from. To that effect, there > are a couple of patches at the beginning of the series that clean up > issues identified by running the tool. These should merge (and be > backported) regardless of the progress on the spec itself. > > [1] https://www.django-rest-framework.org/api-guide/schemas/ > [2] https://github.com/axnsan12/drf-yasg/ > > Stephen Finucane (17): > REST: Add additional documentation > REST: Show 'web_url' in embedded series responses > REST: Ensure patch exists for check creation > REST: Ensure submission exists for comment listing > docs: Start documenting API using OpenAPI > docs: Document the '/users' resource > docs: Document the '/people' resource > docs: Document the '/projects' resource > docs: Document the '/bundles' resource > docs: Document the '/covers' resource > docs: Document the '/patches' resource > docs: Document the '/series' resource > docs: Document the '/covers/{coverID}/comments' resource > docs: Document the '/patch/{patchID}/comments' resource > docs: Document the '/patches/{patchID}/checks' resource > docs: Document the '/events' resource > docs: Make API document versioned > > .gitignore | 1 + > docs/api/schemas/generate_schema.py | 29 + > docs/api/schemas/patchwork.j2 | 2207 +++++++++++++++++ > patchwork/api/check.py | 16 +- > patchwork/api/comment.py | 5 + > patchwork/api/embedded.py | 3 +- > patchwork/api/index.py | 1 + > patchwork/api/patch.py | 10 +- > patchwork/api/project.py | 11 +- > patchwork/api/user.py | 11 +- > patchwork/tests/api/test_check.py | 31 +- > patchwork/tests/api/test_comment.py | 13 + > .../notes/issue-224-8f1c4207aa273ac6.yaml | 5 + > .../notes/issue-225-94215600c1b23f6e.yaml | 6 + > .../notes/issue-226-27ea72266d3ee9ac.yaml | 7 + > 15 files changed, 2349 insertions(+), 7 deletions(-) > create mode 100644 docs/api/schemas/generate_schema.py > create mode 100644 docs/api/schemas/patchwork.j2 > create mode 100644 releasenotes/notes/issue-224-8f1c4207aa273ac6.yaml > create mode 100644 releasenotes/notes/issue-225-94215600c1b23f6e.yaml > create mode 100644 releasenotes/notes/issue-226-27ea72266d3ee9ac.yaml > > -- > 2.17.2 > > _______________________________________________ > Patchwork mailing list > Patchwork@lists.ozlabs.org > https://lists.ozlabs.org/listinfo/patchwork
On Fri, 2018-11-09 at 00:13 +1100, Daniel Axtens wrote: > Stephen Finucane <stephen@that.guru> writes: > > > Round 2. When we first added the REST API, attempts were made to > > document this using OpenAPI 2.0 (f.k.a. Swagger). These floundered owing > > to the tediousness of writing said spec from scratch. Thankfully, tools > > have emerged [1][2] that allow us to generate specs from the code. The > > spec introduced here is based on the combined output of these tools, > > with some additional tweaks to work around gaps in the generated output > > (the embedded serializers in particular caused a lot of fuss). > > > > So I love the idea of generating code, because without users of the > generated spec it's very easy to forget to do the manual work. But, > you've mentioned that there is work that needs to be done on top of the > generated code. How far down the auto-generation path are we, and what > would need to be different in order to have it fully automated? I noted this in patch 17, but I plan to build on this by integrating schema validation into the tests and using the schema for client API code generation. As such, there are three more series needed to really complete this: * Integrate the examples generated by the "Start generating API examples from tests" series into the schema. * Generate documentation for the schema (we have no proper docs for the API right now). * Integrate openapi-core into tests so we can validate the schema against the actual code. I have the first two of these completed locally and the final one is in development (I've been side-tracked by real work and label/tag support but soon). There's also the client side of things but that's not a blocker for any of the above. Does all that make sense? > I will have a look at the rest of the series. > > Regards, > Daniel > > PS. I am travelling next week and then on holiday for another week and a > bit, so I may either get lots done or nothing at all. Sorry! All good :) I am also gone for the next three weeks (\o/) so expect nothing from me until December. Stephen > > The end result is something that we can use to validate the API, > > validate clients and generate documentation from. To that effect, there > > are a couple of patches at the beginning of the series that clean up > > issues identified by running the tool. These should merge (and be > > backported) regardless of the progress on the spec itself. > > > > [1] https://www.django-rest-framework.org/api-guide/schemas/ > > [2] https://github.com/axnsan12/drf-yasg/ > > > > Stephen Finucane (17): > > REST: Add additional documentation > > REST: Show 'web_url' in embedded series responses > > REST: Ensure patch exists for check creation > > REST: Ensure submission exists for comment listing > > docs: Start documenting API using OpenAPI > > docs: Document the '/users' resource > > docs: Document the '/people' resource > > docs: Document the '/projects' resource > > docs: Document the '/bundles' resource > > docs: Document the '/covers' resource > > docs: Document the '/patches' resource > > docs: Document the '/series' resource > > docs: Document the '/covers/{coverID}/comments' resource > > docs: Document the '/patch/{patchID}/comments' resource > > docs: Document the '/patches/{patchID}/checks' resource > > docs: Document the '/events' resource > > docs: Make API document versioned > > > > .gitignore | 1 + > > docs/api/schemas/generate_schema.py | 29 + > > docs/api/schemas/patchwork.j2 | 2207 +++++++++++++++++ > > patchwork/api/check.py | 16 +- > > patchwork/api/comment.py | 5 + > > patchwork/api/embedded.py | 3 +- > > patchwork/api/index.py | 1 + > > patchwork/api/patch.py | 10 +- > > patchwork/api/project.py | 11 +- > > patchwork/api/user.py | 11 +- > > patchwork/tests/api/test_check.py | 31 +- > > patchwork/tests/api/test_comment.py | 13 + > > .../notes/issue-224-8f1c4207aa273ac6.yaml | 5 + > > .../notes/issue-225-94215600c1b23f6e.yaml | 6 + > > .../notes/issue-226-27ea72266d3ee9ac.yaml | 7 + > > 15 files changed, 2349 insertions(+), 7 deletions(-) > > create mode 100644 docs/api/schemas/generate_schema.py > > create mode 100644 docs/api/schemas/patchwork.j2 > > create mode 100644 releasenotes/notes/issue-224-8f1c4207aa273ac6.yaml > > create mode 100644 releasenotes/notes/issue-225-94215600c1b23f6e.yaml > > create mode 100644 releasenotes/notes/issue-226-27ea72266d3ee9ac.yaml > > > > -- > > 2.17.2 > > > > _______________________________________________ > > Patchwork mailing list > > Patchwork@lists.ozlabs.org > > https://lists.ozlabs.org/listinfo/patchwork
Stephen Finucane <stephen@that.guru> writes: > On Fri, 2018-11-09 at 00:13 +1100, Daniel Axtens wrote: >> Stephen Finucane <stephen@that.guru> writes: >> >> > Round 2. When we first added the REST API, attempts were made to >> > document this using OpenAPI 2.0 (f.k.a. Swagger). These floundered owing >> > to the tediousness of writing said spec from scratch. Thankfully, tools >> > have emerged [1][2] that allow us to generate specs from the code. The >> > spec introduced here is based on the combined output of these tools, >> > with some additional tweaks to work around gaps in the generated output >> > (the embedded serializers in particular caused a lot of fuss). >> > >> >> So I love the idea of generating code, because without users of the >> generated spec it's very easy to forget to do the manual work. But, >> you've mentioned that there is work that needs to be done on top of the >> generated code. How far down the auto-generation path are we, and what >> would need to be different in order to have it fully automated? > > I noted this in patch 17, but I plan to build on this by integrating > schema validation into the tests and using the schema for client API > code generation. As such, there are three more series needed to really > complete this: > > * Integrate the examples generated by the "Start generating API > examples from tests" series into the schema. > * Generate documentation for the schema (we have no proper docs for > the API right now). > * Integrate openapi-core into tests so we can validate the schema > against the actual code. > > I have the first two of these completed locally and the final one is in > development (I've been side-tracked by real work and label/tag support > but soon). There's also the client side of things but that's not a > blocker for any of the above. > > Does all that make sense? Mostly, I think. I'm keen on the validation especially. I don't object to this going in (modulo the things we've previously identified) so you can work on those series - if it ends up being more trouble than it's worth we can remove it later. Regards, Daniel > >> I will have a look at the rest of the series. >> >> Regards, >> Daniel >> >> PS. I am travelling next week and then on holiday for another week and a >> bit, so I may either get lots done or nothing at all. Sorry! > > All good :) I am also gone for the next three weeks (\o/) so expect > nothing from me until December. > > Stephen > >> > The end result is something that we can use to validate the API, >> > validate clients and generate documentation from. To that effect, there >> > are a couple of patches at the beginning of the series that clean up >> > issues identified by running the tool. These should merge (and be >> > backported) regardless of the progress on the spec itself. >> > >> > [1] https://www.django-rest-framework.org/api-guide/schemas/ >> > [2] https://github.com/axnsan12/drf-yasg/ >> > >> > Stephen Finucane (17): >> > REST: Add additional documentation >> > REST: Show 'web_url' in embedded series responses >> > REST: Ensure patch exists for check creation >> > REST: Ensure submission exists for comment listing >> > docs: Start documenting API using OpenAPI >> > docs: Document the '/users' resource >> > docs: Document the '/people' resource >> > docs: Document the '/projects' resource >> > docs: Document the '/bundles' resource >> > docs: Document the '/covers' resource >> > docs: Document the '/patches' resource >> > docs: Document the '/series' resource >> > docs: Document the '/covers/{coverID}/comments' resource >> > docs: Document the '/patch/{patchID}/comments' resource >> > docs: Document the '/patches/{patchID}/checks' resource >> > docs: Document the '/events' resource >> > docs: Make API document versioned >> > >> > .gitignore | 1 + >> > docs/api/schemas/generate_schema.py | 29 + >> > docs/api/schemas/patchwork.j2 | 2207 +++++++++++++++++ >> > patchwork/api/check.py | 16 +- >> > patchwork/api/comment.py | 5 + >> > patchwork/api/embedded.py | 3 +- >> > patchwork/api/index.py | 1 + >> > patchwork/api/patch.py | 10 +- >> > patchwork/api/project.py | 11 +- >> > patchwork/api/user.py | 11 +- >> > patchwork/tests/api/test_check.py | 31 +- >> > patchwork/tests/api/test_comment.py | 13 + >> > .../notes/issue-224-8f1c4207aa273ac6.yaml | 5 + >> > .../notes/issue-225-94215600c1b23f6e.yaml | 6 + >> > .../notes/issue-226-27ea72266d3ee9ac.yaml | 7 + >> > 15 files changed, 2349 insertions(+), 7 deletions(-) >> > create mode 100644 docs/api/schemas/generate_schema.py >> > create mode 100644 docs/api/schemas/patchwork.j2 >> > create mode 100644 releasenotes/notes/issue-224-8f1c4207aa273ac6.yaml >> > create mode 100644 releasenotes/notes/issue-225-94215600c1b23f6e.yaml >> > create mode 100644 releasenotes/notes/issue-226-27ea72266d3ee9ac.yaml >> > >> > -- >> > 2.17.2 >> > >> > _______________________________________________ >> > Patchwork mailing list >> > Patchwork@lists.ozlabs.org >> > https://lists.ozlabs.org/listinfo/patchwork
Round 2. When we first added the REST API, attempts were made to document this using OpenAPI 2.0 (f.k.a. Swagger). These floundered owing to the tediousness of writing said spec from scratch. Thankfully, tools have emerged [1][2] that allow us to generate specs from the code. The spec introduced here is based on the combined output of these tools, with some additional tweaks to work around gaps in the generated output (the embedded serializers in particular caused a lot of fuss). The end result is something that we can use to validate the API, validate clients and generate documentation from. To that effect, there are a couple of patches at the beginning of the series that clean up issues identified by running the tool. These should merge (and be backported) regardless of the progress on the spec itself. [1] https://www.django-rest-framework.org/api-guide/schemas/ [2] https://github.com/axnsan12/drf-yasg/ Stephen Finucane (17): REST: Add additional documentation REST: Show 'web_url' in embedded series responses REST: Ensure patch exists for check creation REST: Ensure submission exists for comment listing docs: Start documenting API using OpenAPI docs: Document the '/users' resource docs: Document the '/people' resource docs: Document the '/projects' resource docs: Document the '/bundles' resource docs: Document the '/covers' resource docs: Document the '/patches' resource docs: Document the '/series' resource docs: Document the '/covers/{coverID}/comments' resource docs: Document the '/patch/{patchID}/comments' resource docs: Document the '/patches/{patchID}/checks' resource docs: Document the '/events' resource docs: Make API document versioned .gitignore | 1 + docs/api/schemas/generate_schema.py | 29 + docs/api/schemas/patchwork.j2 | 2207 +++++++++++++++++ patchwork/api/check.py | 16 +- patchwork/api/comment.py | 5 + patchwork/api/embedded.py | 3 +- patchwork/api/index.py | 1 + patchwork/api/patch.py | 10 +- patchwork/api/project.py | 11 +- patchwork/api/user.py | 11 +- patchwork/tests/api/test_check.py | 31 +- patchwork/tests/api/test_comment.py | 13 + .../notes/issue-224-8f1c4207aa273ac6.yaml | 5 + .../notes/issue-225-94215600c1b23f6e.yaml | 6 + .../notes/issue-226-27ea72266d3ee9ac.yaml | 7 + 15 files changed, 2349 insertions(+), 7 deletions(-) create mode 100644 docs/api/schemas/generate_schema.py create mode 100644 docs/api/schemas/patchwork.j2 create mode 100644 releasenotes/notes/issue-224-8f1c4207aa273ac6.yaml create mode 100644 releasenotes/notes/issue-225-94215600c1b23f6e.yaml create mode 100644 releasenotes/notes/issue-226-27ea72266d3ee9ac.yaml