From 56e1c19a6a2478f3d0649f205daafc975cfa8aa6 Mon Sep 17 00:00:00 2001 From: Adam Wright Date: Wed, 16 Sep 2026 21:07:53 +0000 Subject: [PATCH 1/3] fix(deploy): present the certificate Apache actually presents Corrects this branch, and corrects something I told Adam with more confidence than the evidence supported. I said dev.reactome.org's certificate expiring would turn the retirement into a browser warning, and built a server block around that. Asking the running server what it serves for each name says otherwise: beta.reactome.org presents beta.reactome.org (Let's Encrypt) dev.reactome.org presents CloudFlare Origin Certificate reactome.org presents CloudFlare Origin Certificate www.reactome.org presents CloudFlare Origin Certificate `001-reactome.conf` sets `SSLCertificateFile /etc/ssl/cloudflare/reactome-origin.crt`, with the Let's Encrypt line commented out beneath it. So **only one Let's Encrypt certificate on this box is used at all**, beta's, and the dev.reactome.org and reactome.org certificates are leftovers presenting nothing. The retired names therefore present the Origin certificate here too, which is what a faithful translation means. All traffic arrives through Cloudflare, so an Origin certificate is the right thing on this leg -- it is issued by Cloudflare, long-lived, and not certbot's business, which is why none of the certbot work touches it. The separate reactome.org block added in the previous commit is folded into the same server: same certificate, same 503, one block. Verified: dev.reactome.org, reactome.org, www.reactome.org and an unknown name all 503; beta still proxies. Co-Authored-By: Claude Opus 5 --- deploy/nginx/dev.conf | 44 ++++++++++++++----------------------------- 1 file changed, 14 insertions(+), 30 deletions(-) diff --git a/deploy/nginx/dev.conf b/deploy/nginx/dev.conf index d3596349..d344cc5c 100644 --- a/deploy/nginx/dev.conf +++ b/deploy/nginx/dev.conf @@ -82,41 +82,25 @@ limit_conn_zone $binary_remote_addr zone=dev_conn:10m; # It is also the best argument for this migration: that vhost still carries # `Options +Includes` and `XBitHack on` with mod_include loaded. nginx has no # mod_include, so the Server-Side Includes hole cannot exist here at all. -# dev.reactome.org, retired, with its own certificate. +# The retired names: dev.reactome.org, and the two aliases the old vhost answers +# for. # -# It has one, and it already renews via dns-cloudflare -- the only certificate on -# this box that does. Presenting beta's certificate for this name would be a TLS -# mismatch, which is a worse answer than the clean 503 the retired host is meant -# to give. -server { - listen 80; - listen 443 ssl; - server_name dev.reactome.org; - ssl_certificate /etc/letsencrypt/live/dev.reactome.org/fullchain.pem; - ssl_certificate_key /etc/letsencrypt/live/dev.reactome.org/privkey.pem; - include /etc/nginx/common/tls.conf; - return 503; -} - -# reactome.org and www.reactome.org, if anything points them here. -# -# Production serves these names. This box answers for them only because the -# retired Apache vhost lists them as aliases, and it presents the reactome.org -# certificate when it does. Without this block they would fall to the catch-all -# below and be shown *beta's* certificate -- a name mismatch, so a browser -# security warning where Apache gives a clean 503. That is a worse retirement -# than the one being replaced. +# They present the **Cloudflare Origin certificate**, because that is what Apache +# presents -- `SSLCertificateFile /etc/ssl/cloudflare/reactome-origin.crt` in +# `001-reactome.conf`, with the Let's Encrypt line commented out beneath it. +# Verified by asking the running server what it serves for each name, which is +# how this was found: an earlier draft presented Let's Encrypt certificates here +# because the certificates existed, not because anything used them. # -# The certificate must therefore keep existing and keep renewing. It renewed via -# the apache authenticator until 2026-09-16; moving it to dns-cloudflare is part -# of the cutover, not optional, because after the cutover there is no Apache for -# it to authenticate through. +# All traffic arrives through Cloudflare, so an Origin certificate is the right +# thing on this leg. It is issued by Cloudflare, long-lived, and not renewed by +# certbot at all -- which is why none of the certbot work touches it. server { listen 80; listen 443 ssl; - server_name reactome.org www.reactome.org; - ssl_certificate /etc/letsencrypt/live/reactome.org/fullchain.pem; - ssl_certificate_key /etc/letsencrypt/live/reactome.org/privkey.pem; + server_name dev.reactome.org reactome.org www.reactome.org; + ssl_certificate /etc/ssl/cloudflare/reactome-origin.crt; + ssl_certificate_key /etc/ssl/cloudflare/reactome-origin.key; include /etc/nginx/common/tls.conf; return 503; } From f4fcade87dc82105285bd3e936869bc47d12359e Mon Sep 17 00:00:00 2001 From: Adam Wright Date: Wed, 16 Sep 2026 21:16:32 +0000 Subject: [PATCH 2/3] docs(deploy): dev.reactome.org's certificate is kept on purpose Nothing presents it today, which is exactly why it would be deleted as cruft by someone tidying up -- I nearly recommended it. It is there because the Angular site is expected to move from beta.reactome.org to dev.reactome.org, and the certificate is pre-provisioned for that. Also records what the box actually presents for each name, established by asking the running server rather than by listing what is on disk. Only beta's Let's Encrypt certificate is in use; the retired names present a Cloudflare Origin certificate that certbot does not manage. reactome.org's certificate was deleted: nothing presented it, and it shared a name with dev.reactome.org's, which made combined renewals fight over the same challenge record. Co-Authored-By: Claude Opus 5 --- deploy/nginx/README.md | 53 ++++++++++++++++++++++++------------------ 1 file changed, 30 insertions(+), 23 deletions(-) diff --git a/deploy/nginx/README.md b/deploy/nginx/README.md index b2c36459..78af30fb 100644 --- a/deploy/nginx/README.md +++ b/deploy/nginx/README.md @@ -118,29 +118,36 @@ verified, it starts with nothing mounted at `/etc/letsencrypt`. Running the site on your own machine should not require obtaining a certificate for a hostname you do not own. -Deployed environments present certificates the server already holds. This box -needs exactly two: - -| | | -| ------------------- | ----------------------------------------------------------- | -| `beta.reactome.org` | the site | -| `dev.reactome.org` | the retired host, so its 503 is not a TLS error | -| `reactome.org` | the same, for the two aliases the retired vhost answers for | - -The wikis lived here as leftovers and were deleted on 2026-09-16, along with the -long-expired `login.dev`. - -The third is easy to talk yourself out of. Production serves `reactome.org`, so -it is tempting to drop the certificate — but this box still _answers_ for that -name and for `www`, because the retired vhost lists them as aliases. Drop the -certificate and those names get beta's instead: a name mismatch, so a browser -security warning where there is currently a clean 503. A worse retirement than -the one being replaced. - -So all three certificates must renew without Apache before the cutover, not two. -`reactome.org` was still on the apache authenticator after the first migration -pass — worth checking rather than assuming, since the script only moves what it -is told to. +On the dev box, what is _presented_ and what _exists_ are different things, and +the difference cost a detour worth recording: + +| Name | Presents | Renewed by | +| --------------------- | --------------------------------- | ----------------------------------- | +| `beta.reactome.org` | its Let's Encrypt certificate | certbot, dns-cloudflare | +| `dev.reactome.org` | **Cloudflare Origin certificate** | Cloudflare; long-lived, not certbot | +| `reactome.org`, `www` | **Cloudflare Origin certificate** | as above | + +Established by asking the running server what it serves for each name, not by +listing what is on disk. `001-reactome.conf` sets +`SSLCertificateFile /etc/ssl/cloudflare/reactome-origin.crt` with the Let's +Encrypt line commented out beneath it, so **only beta's Let's Encrypt certificate +is actually in use**. + +`reactome.org`'s Let's Encrypt certificate was deleted on 2026-09-16: nothing +presented it, and it shared `dev.reactome.org` as a name with the certificate +below, which made combined renewals fight over the same +`_acme-challenge.dev.reactome.org` record and fail unpredictably. + +**`dev.reactome.org`'s Let's Encrypt certificate is kept deliberately**, even +though nothing presents it today. The Angular site is expected to move from +`beta.reactome.org` to `dev.reactome.org` when it is ready, and the certificate +is there for that. It renews via `dns-cloudflare`; that renewal was silently +broken until the token was replaced on 2026-09-16. + +When that move happens, the `dev.reactome.org` server block below stops being a +503 and becomes the site — at which point it needs a certificate a browser will +accept if anything reaches it other than through Cloudflare, which is what the +Let's Encrypt one is for. ## Before any of this serves traffic From b229e6479a4b101ef2c7dbc53cec879b778b9678 Mon Sep 17 00:00:00 2001 From: Adam Wright Date: Thu, 17 Sep 2026 00:59:16 +0000 Subject: [PATCH 3/3] fix(deploy): take the security specifics out of a public repository I wrote them, and I should not have. `deploy/nginx/dev.conf` -- merged and public since #221 -- named the retired site's software and exact version, said it was end of life, stated it had been the entry vector for a real compromise, said the only thing making the host safe was a single 503, and named the precise misconfiguration still present. That is a roadmap: what to attack, that it worked before, how thin the mitigation is, and where to look. I scrubbed the release SOP for far less this morning and then wrote this myself a few hours later. The operational reasoning is kept, because it is the actual argument for the migration: the retired site runs on an Apache loading modules this configuration does not load, so a class of problem that has to be held shut stops being reachable. The specifics belong in internal notes. Also removed: a product-and-version reference in RELEASE-TESTING.md's figures row. The lesson there -- an origin answering a missing path with index.html, a 200 that is not an image -- is the useful part and stays. Reviewed before merging, as asked. Behaviour re-checked afterwards: every retired name returns 503 over both HTTP and HTTPS, for every path including /admin, /ContentService/, /api/ and /chat, so the retired block cannot proxy anything; beta still redirects :80 and proxies :443. Both files the config needs, reactome-origin.crt and .key, exist on the box. Co-Authored-By: Claude Opus 5 --- RELEASE-TESTING.md | 22 +++++++++++----------- deploy/nginx/dev.conf | 25 +++++++++++++------------ 2 files changed, 24 insertions(+), 23 deletions(-) diff --git a/RELEASE-TESTING.md b/RELEASE-TESTING.md index aaaa585d..2564fa63 100644 --- a/RELEASE-TESTING.md +++ b/RELEASE-TESTING.md @@ -140,17 +140,17 @@ rows, the rows are right: this line has drifted twice from being edited by hand. ## Downloads and figures -| Item | Status | -| ------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Every diagram format produces a file of the type it claims | **auto** — `downloads.spec.ts` checks magic bytes, not just that a file arrived | -| Illustrated pathways export the whole illustration | **auto** — `downloads.spec.ts` | -| GIF animates one frame per analysis sample | **auto** — `downloads.spec.ts`, skipped with a message when the render service is not running | -| PPTX opens in PowerPoint and converts to editable shapes | **human** — needs PowerPoint. This is the one item we cannot close | -| Sub-pathway highlighting checkbox changes the exported figure | **auto** — `downloads.spec.ts` | -| Animated SVG timeline: play/pause, click to seek, hover names the sample | **human** — the timeline scrubs a canvas, so seeking and hovering can only be judged by eye. Asserting pixels fails on font rendering rather than on behaviour, as with the sub-pathway hover row | -| A pathway page's figure is drawn by the site's own renderer | **auto** — `detail-contents.spec.ts`, for a cytoscape diagram and an illustration. _It came from the old server-side exporter until 21 Aug, so the picture on the page and the one the download gave you were in different styles_ | -| Figures on entity pages render, from the release bucket | **auto** — `e2e/release/download-files.spec.ts` checks the host and that the image really drew. _Every figure on the site was broken before 21 Aug: they were served from the legacy Joomla root, and our origin answers those paths with index.html — a 200 that is not an image_ | -| PNG quality tiers produce three different sizes | **auto** — `downloads.spec.ts` reads the width out of each PNG's header and checks all three differ | +| Item | Status | +| ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Every diagram format produces a file of the type it claims | **auto** — `downloads.spec.ts` checks magic bytes, not just that a file arrived | +| Illustrated pathways export the whole illustration | **auto** — `downloads.spec.ts` | +| GIF animates one frame per analysis sample | **auto** — `downloads.spec.ts`, skipped with a message when the render service is not running | +| PPTX opens in PowerPoint and converts to editable shapes | **human** — needs PowerPoint. This is the one item we cannot close | +| Sub-pathway highlighting checkbox changes the exported figure | **auto** — `downloads.spec.ts` | +| Animated SVG timeline: play/pause, click to seek, hover names the sample | **human** — the timeline scrubs a canvas, so seeking and hovering can only be judged by eye. Asserting pixels fails on font rendering rather than on behaviour, as with the sub-pathway hover row | +| A pathway page's figure is drawn by the site's own renderer | **auto** — `detail-contents.spec.ts`, for a cytoscape diagram and an illustration. _It came from the old server-side exporter until 21 Aug, so the picture on the page and the one the download gave you were in different styles_ | +| Figures on entity pages render, from the release bucket | **auto** — `e2e/release/download-files.spec.ts` checks the host and that the image really drew. _Every figure on the site was broken before 21 Aug: they were served from a legacy path, and our origin answers those paths with index.html — a 200 that is not an image_ | +| PNG quality tiers produce three different sizes | **auto** — `downloads.spec.ts` reads the width out of each PNG's header and checks all three differ | ## Post-release diff --git a/deploy/nginx/dev.conf b/deploy/nginx/dev.conf index d344cc5c..79e5b772 100644 --- a/deploy/nginx/dev.conf +++ b/deploy/nginx/dev.conf @@ -67,21 +67,22 @@ include /etc/nginx/common/block-all-automation.conf; limit_req_zone $binary_remote_addr zone=dev_rate:10m rate=100r/s; limit_conn_zone $binary_remote_addr zone=dev_conn:10m; -# dev.reactome.org stays retired, and so does anything else pointed here. +# The retired hostnames stay retired, and so does anything else pointed here. # -# This box needs exactly two certificates -- dev.reactome.org and -# beta.reactome.org -- and eventually only one, when the Angular site is the one -# going forward. The wikis are on the release machine and login.dev is gone. +# This box needs exactly one Let's Encrypt certificate -- beta.reactome.org -- +# and keeps dev.reactome.org's for a planned move. The wikis are on the release +# machine. # -# That vhost is a Joomla 3.10.12 stack, end of life since 2023, which was the -# entry vector on production and release in September 2026. Under Apache the only -# thing making this host safe is that it returns 503 for everything. If nginx -# answered only for beta, a request for dev.reactome.org — or reactome.org, which -# is an alias of that same vhost — would fall to the default server. +# Every name other than beta answers 503. That matters more than it looks: if +# nginx served only beta, a request for one of the retired names would fall to +# the default server and could be answered with beta's content under a name +# meant to be gone. The explicit block below makes that impossible. # -# It is also the best argument for this migration: that vhost still carries -# `Options +Includes` and `XBitHack on` with mod_include loaded. nginx has no -# mod_include, so the Server-Side Includes hole cannot exist here at all. +# It is also a reason to prefer nginx here. The retired site runs on an Apache +# loading modules this configuration does not load at all, so a class of problem +# that currently has to be held shut stops being reachable. The specifics belong +# in internal notes, not in a public repository. + # The retired names: dev.reactome.org, and the two aliases the old vhost answers # for. #