diff --git a/skills/README.md b/skills/README.md new file mode 100644 index 000000000..1f2aa1a0b --- /dev/null +++ b/skills/README.md @@ -0,0 +1,67 @@ + + +# Roller developer skills + +Optional developer tooling. Nothing here is part of the Roller build, the +source or binary distributions, or the runtime; the source assembly does not +include this directory. You can ignore it entirely and work on Roller normally. + +Each subdirectory is a "skill": a `SKILL.md` describing a project procedure, +plus supporting reference notes, templates and helper scripts. They exist so +that recurring project chores are written down in one reviewable place instead +of living in one committer's head. + +| Skill | Covers | +|---|---| +| [`roller-release`](roller-release/SKILL.md) | Preparing, building, signing, verifying, staging, voting on and publishing a release, plus the release website and announcement drafts. | +| [`roller-security`](roller-security/SKILL.md) | Security report triage, private case tracking, CVE preparation, reporter coordination and disclosure planning. | + +## Reading them + +`SKILL.md` and the files under `references/` are plain Markdown. Read them +directly for a written account of how a release is put together. No tooling is +required for this. + +## Using them with an AI coding assistant + +The layout follows the convention used by Claude Code and similar agent tools, +which discover skills under `.claude/skills/`. To make them available without +duplicating the files: + +```sh +mkdir -p .claude/skills +ln -s ../../skills/roller-release .claude/skills/roller-release +``` + +`.claude/` is not tracked by this repository, so this is a local choice that +affects only your checkout. Copying the directories instead of symlinking works +equally well. + +## Scope and limits + +These skills describe procedure; they do not grant authority. The PMC owns +release decisions, and ASF policy governs what a release requires. Where a skill +and current ASF policy disagree, policy wins and the skill needs fixing. + +The helper scripts under `scripts/` are deliberately conservative: they preview +by default, they do not commit, tag, push, sign on your behalf, or send mail, +and they are not a substitute for reading the diff and verifying a candidate +yourself. Verify current ASF policy from its authoritative sources rather than +trusting a summary here. + +Corrections are welcome as ordinary pull requests. diff --git a/skills/roller-release/SKILL.md b/skills/roller-release/SKILL.md new file mode 100644 index 000000000..43ca49ec2 --- /dev/null +++ b/skills/roller-release/SKILL.md @@ -0,0 +1,95 @@ +--- +name: roller-release +description: Prepare, build, sign, verify, stage, vote on and publish Apache Roller releases, including release website updates and announcement drafts. +--- + + + +# Apache Roller releases + +Use the current checkout, project release decisions and +[ASF policy sources](references/asf-policy.md). Keep release execution notes, +personal signing setup and undisclosed security scope outside this skill. +Loading the skill does not authorize sending mail, pushing tags or publishing. + +## Establish release inputs + +Record the source checkout, target branch and exact commit; release version; +RC number and tag; release-manager ASF ID; signing-key fingerprint; distribution +working copies; and website checkout. Supply these from the actual release, +not defaults from an earlier session. Use a clean tree or isolated checkout. + +Check build documentation and CI for the branch's supported JDK. Confirm Maven, +GnuPG, SVN and archive tools are available. Python 3 is used by the version helper. +Use [signing-key guidance](references/signing-key.md) before building a candidate. + +## Prepare and verify + +1. Agree scope and version, keeping undisclosed vulnerability details private. +2. Inspect version-bearing POMs and packaging scripts. Run + `scripts/bump-version.sh ` from this skill for a preview; use + `--write` only to apply the reviewed scope. It does not commit. Update release + notes and other appropriate references manually, preserving historical entries. +3. Commit the release inputs and record an RC tag at the exact source revision. + Build from that revision so the embedded revision matches the candidate. + Preserve candidate history; changed artifacts normally require a new candidate + and vote. Do not silently replace artifacts already being voted on. +4. Build and test from the root using the branch's documented Maven invocation + (normally `mvn -V -ntp clean install`). Build `assembly-release` separately + with `mvn -f assembly-release/pom.xml package`; it is outside the root reactor. +5. Inspect the source and binary tar/zip archives. Sign and generate checksums + using [signing instructions](references/signing-key.md). +6. Run `scripts/check-release.sh ` from this skill. Its optional + `--build` compiles the source archive with tests skipped; it does not replace + the test suite, a full license review, or independent voter verification. +7. Stage and download the candidate using [distribution guidance](references/dist-svn.md). + Verify signatures, checksums, contents and source build from the downloaded files. + +## Vote and publish + +Use [vote and announcement templates](references/vote-and-announce.md). Normally +allow at least 72 hours and require at least three positive binding votes and +more positive than negative binding votes. Check current policy for exceptions. +Record the result and the exact approved candidate. + +Promote the approved artifact bytes; do not rebuild. If filenames lose the RC +suffix, detached signatures still verify those unchanged bytes, but checksum +filenames must be updated and verified. Point the final Git tag at the approved +RC commit. Inspect remote destination paths before distribution changes. + +Update and publish the [website](references/website.md), verify public download +links and propagation, then announce. Prune superseded distributions only after +the website points to the new release and archive availability is confirmed. +Keep historical signing keys available for verification of old releases. + +For security releases, coordinate advisory timing with the PMC and ASF Security; +use the companion `roller-security` skill when available. Public vote material +must not expose undisclosed case details. The operator upgrade must be available +when the advisory is published. + +## References and helpers + +- [ASF policy](references/asf-policy.md): authoritative sources and review boundaries. +- [Signing](references/signing-key.md): explicit signer selection and verification. +- [Distribution](references/dist-svn.md): staging, promotion and pruning. +- [Vote and announcement](references/vote-and-announce.md): reusable message drafts. +- [Website](references/website.md): source edits, rebuild and link verification. +- [Release tooling](references/atr.md): evaluating Apache Trusted Releases. + +Resolve helper paths relative to this skill; when using the repository copy, +they are under `skills/roller-release/scripts/`. Helpers do not commit or publish. diff --git a/skills/roller-release/references/asf-policy.md b/skills/roller-release/references/asf-policy.md new file mode 100644 index 000000000..2adca93d0 --- /dev/null +++ b/skills/roller-release/references/asf-policy.md @@ -0,0 +1,45 @@ + + +# Authoritative release policy + +Consult these sources when cutting a release; this skill is not a policy snapshot: + +- [Release policy](https://www.apache.org/legal/release-policy.html): source artifacts, + binding vote requirements, licensing and publication. +- [Distribution policy](https://infra.apache.org/release-distribution.html): official + distribution channels, signatures, digests, download links and archive handling. +- [Release signing](https://infra.apache.org/release-signing.html): key and signature guidance. +- [Release publishing](https://infra.apache.org/release-publishing.html): operational steps. +- [Roller release process](https://cwiki.apache.org/confluence/spaces/ROLLER/pages/75651/Release+Process): + project instructions; verify paths and commands against the current checkout. + +A source release is the approved artifact; binary packages are convenience +artifacts. Release approval requires at least three positive binding votes and +more positive binding votes than negative ones. The normal review period is at +least 72 hours; consult the policy for exceptional expedited releases. + +Independently validate signed source packages, compile and test them, and review +licensing. `check-release.sh` checks selected mechanical properties only. Presence +of LICENSE and NOTICE does not prove that bundled material is correctly licensed. +Inspect compiled artifacts, third-party content and exclusions in the actual +source archive. Do not treat a historical release as an exemption from policy. + +Publish the artifacts that were approved. Rebuilding changes what voters reviewed +and requires renewed approval. Use the official channels and verify public links +before announcing availability. Keep per-release findings and corrective actions +in release notes or the private coordination workspace as appropriate, not here. diff --git a/skills/roller-release/references/atr.md b/skills/roller-release/references/atr.md new file mode 100644 index 000000000..328c6b4c6 --- /dev/null +++ b/skills/roller-release/references/atr.md @@ -0,0 +1,36 @@ + + +# Evaluating Apache Trusted Releases + +[Apache Trusted Releases](https://releases.apache.org) is an alternative release +workflow to evaluate with the PMC. Check its current documentation, availability, +project enrollment and supported packaging before adoption; this skill does not +pin a platform status or participating-project list. + +Try a non-publishing rehearsal and establish how the chosen tool handles: + +- Source and convenience binary archives, signatures and checksums. +- Existing project distribution layout and KEYS handling. +- Candidate identity, vote evidence, cancellation and replacement. +- Promotion of the exact approved bytes and website coordination. +- Access control and the separation of private security coordination from + public release artifacts and messages. + +Agree the workflow and ownership before a release deadline. Keep the evaluation +and any per-release decision in project planning records. Do not switch an active +release between tools without reconciling candidate identity and approval evidence. diff --git a/skills/roller-release/references/dist-svn.md b/skills/roller-release/references/dist-svn.md new file mode 100644 index 000000000..25af16340 --- /dev/null +++ b/skills/roller-release/references/dist-svn.md @@ -0,0 +1,59 @@ + + +# Distribution staging and promotion + +Confirm the current project layout before preparing paths: + +```sh +svn ls https://dist.apache.org/repos/dist/dev/roller/ +svn ls https://dist.apache.org/repos/dist/release/roller/ +``` + +The conventional layout is `roller-/v/` below each project +root. RC suffixes are typically in archive filenames. Verify this convention for +the target branch; do not infer paths from an older wiki example. + +Use separate user-selected working directories for dev and release SVN checkouts. +Stage only the intended candidate archives and signature/checksum sidecars. Review +`svn status` and `svn diff` before committing. Download staged files to a fresh +directory and verify them before starting the vote. Development staging is publicly +accessible even though it is not an official release; do not upload private notes. + +## Approved-candidate promotion + +Record the passed vote, source SVN revision, candidate filenames and destination. +Use an SVN working copy or a single reviewed repository transaction to promote +only that candidate. Preserve archive bytes and detached signatures. If removing +an RC suffix from filenames, update the filename references in checksum sidecars +and verify each digest against the unchanged archive. + +Do not blindly promote everything in a version directory: it may contain cancelled +candidates or unrelated files. Verify the final inventory, signature fingerprints +and checksum checks after promotion. Never rebuild to remove an RC suffix. + +Wait for distribution propagation and check the public download URLs, not merely +SVN success. Follow the current +[release publishing guidance](https://infra.apache.org/release-publishing.html) +for timing. Update the website and verify its links before announcing. + +## Cleanup + +Remove cancelled candidate files only within the agreed cleanup scope, checking +exact filenames first. For superseded official releases, confirm archive +availability and update website links before pruning. Preserve the KEYS history. +A successful delete does not remove copies already downloaded from public staging. diff --git a/skills/roller-release/references/signing-key.md b/skills/roller-release/references/signing-key.md new file mode 100644 index 000000000..27fe1677e --- /dev/null +++ b/skills/roller-release/references/signing-key.md @@ -0,0 +1,54 @@ + + +# Release signing + +Read the current [ASF signing guidance](https://infra.apache.org/release-signing.html) +and [distribution policy](https://infra.apache.org/release-distribution.html). +Select the acting release manager's key explicitly. Never copy another person's +fingerprint or assume the default local key is suitable. + +Check ownership, expiry, revocation, algorithm and strength, including the signing +subkey. For a new RSA signing key, use at least 4096 bits and an appropriate ASF +identity. Keep private keys, passphrases, keyring inventories and local configuration +backups outside the checkout. Let GnuPG prompt through the user's normal pinentry; +do not capture a passphrase in shell commands or agent output. + +Publish the public key through the project's KEYS process and the release manager's +ASF profile as applicable. The project KEYS file is available at +. Add new public keys without removing +historical signing keys. Verify fingerprints independently before trusting an import. + +Set `ROLLER_SIGNING_KEY` to the selected fingerprint and `VERSION` / `RC_SUFFIX` +to the release inputs. Inspect the checkout's `assembly-release/sign-release.sh` +usage; in versions supporting positional inputs, run from the repository root: + +```sh +assembly-release/sign-release.sh "$VERSION" "$RC_SUFFIX" +``` + +The current script expects `ROLLER_SIGNING_KEY` in its environment. Export it +explicitly in the release session. Confirm that the script signs with the intended +key and uses an approved digest. Do not transplant an old signing script from +release notes. Generate armored detached signatures and SHA-256 and/or SHA-512 +checksum sidecars; avoid legacy digest formats. + +Verify every archive's signature against its bytes and every checksum from the +artifact directory. Check the expected fingerprint, not just GnuPG's exit status. +A cryptographically valid signature does not itself establish signer identity or +full release compliance. Keep the signer fingerprint in release execution notes +and the vote draft, not as a fixed default in this skill. diff --git a/skills/roller-release/references/vote-and-announce.md b/skills/roller-release/references/vote-and-announce.md new file mode 100644 index 000000000..512a69f72 --- /dev/null +++ b/skills/roller-release/references/vote-and-announce.md @@ -0,0 +1,76 @@ + + +# Vote and announcement drafts + +Replace bracketed values from the actual release record. Verify URLs, tag commit, +artifact checksums and signer fingerprint. Do not send messages merely because a +template has been filled. Keep undisclosed security characterization out of public +vote threads and release notes before coordinated announcement. + +## Vote + +Subject: [VOTE] Release Apache Roller [version] ([candidate]) + +Please review and vote on Apache Roller [version], candidate [candidate]. + +Source and convenience binary artifacts: [staging URL] +Source tag: [tag URL and commit] +KEYS: https://downloads.apache.org/roller/KEYS +Signing fingerprint: [release manager's verified fingerprint] +Release notes: [appropriate public release notes URL] + +The vote will remain open until at least [date, time and timezone], allowing +at least 72 hours for review under the normal process. + +[ ] +1 Release this candidate +[ ] +0 No opinion +[ ] -1 Do not release, because [reason] + +[Release manager name] + +## Result + +Subject: [RESULT][VOTE] Release Apache Roller [version] ([candidate]) + +The vote [passed/did not pass]. + +Binding votes: [positive, neutral and negative counts; voters] +Non-binding votes: [counts; voters] +Vote thread: [public permalink] +Approved candidate, if passed: [exact tag, commit and artifact location] + +[Next step consistent with the result] + +[Release manager name] + +## Release announcement + +Subject: [ANNOUNCE] Apache Roller [version] released + +The Apache Roller project is pleased to announce Apache Roller [version]. + +[Concise description of the release and publicly releasable changes] + +Downloads: https://roller.apache.org/downloads/downloads.html +Release notes: [public URL] +[Upgrade guidance and coordinated advisory links, when ready for disclosure] + +[Release manager name], on behalf of the Apache Roller project + +Check the current ASF and project announcement guidance for recipients and +formatting. Tally binding votes by PMC membership; do not count every +1 as binding. diff --git a/skills/roller-release/references/website.md b/skills/roller-release/references/website.md new file mode 100644 index 000000000..be8622281 --- /dev/null +++ b/skills/roller-release/references/website.md @@ -0,0 +1,39 @@ + + +# Release website update + +Use the user's chosen checkout of the Apache Roller website repository: +. +Inspect its README and `.asf.yaml` to confirm the publishing branch and build +instructions; do not assume that a sibling checkout exists or is up to date. + +The JBake layout uses `sources/src/main/jbake/content/` for source pages and +`content/` for rendered output. Find the download page and edit its source version, +download URLs and verification guidance. Build with the repository's documented +command (normally `mvn package` under `sources/`) and review both source and +rendered changes before publishing. + +Use HTTPS and current ASF download infrastructure. Fetch signatures, checksums +and KEYS from the official distribution site rather than an arbitrary mirror. +Verify versioned archive links and all verification sidecars after propagation. +A redirect response alone is not evidence that its destination archive exists. + +Publish the website only when the approved release artifacts are available. +Verify the live page and links before announcement, then prune superseded +releases within the agreed scope. Keep release-specific website defects in the +release work record, not as permanent instructions in this reference. diff --git a/skills/roller-release/scripts/bump-version.sh b/skills/roller-release/scripts/bump-version.sh new file mode 100755 index 000000000..f45389822 --- /dev/null +++ b/skills/roller-release/scripts/bump-version.sh @@ -0,0 +1,68 @@ +#!/usr/bin/env bash +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Preview or update Roller module versions; never commits. +# Usage: bump-version.sh [--write] +# Requires Git and Python 3. Run from any directory in the target checkout. +set -euo pipefail +root=$(git rev-parse --show-toplevel) || exit 1 +python3 - "$root" "$@" <<'PY' +import argparse +from pathlib import Path +import re +import subprocess +import sys + +root = Path(sys.argv[1]) +parser = argparse.ArgumentParser(description="Preview or update Roller module versions.") +parser.add_argument("old") +parser.add_argument("new") +parser.add_argument("--write", action="store_true") +args = parser.parse_args(sys.argv[2:]) +for value in (args.old, args.new): + if not re.fullmatch(r"[0-9]+\.[0-9]+\.[0-9]+(?:-[A-Za-z0-9.-]+)?", value): + parser.error("versions must be major.minor.patch with an optional suffix") +expected = ["pom.xml", "app/pom.xml", "db-utils/pom.xml", + "assembly-release/pom.xml", "it-selenium/pom.xml"] +missing = [name for name in expected if not (root / name).is_file()] +if missing: + sys.exit("Missing expected files; no changes made: " + ", ".join(missing)) +# Match complete version element contents, not prefixes of other versions. +pattern = re.compile(r"(?P\s*)" + re.escape(args.old) + + r"(?P\s*)") +changes = [] +for name in expected: + path = root / name + original = path.read_bytes() + text = original.decode("utf-8") + updated, count = pattern.subn(lambda m: m['start'] + args.new + m['end'], text) + if count: + changes.append((path, updated.encode("utf-8"))) + print(f"{'update' if args.write else 'would update'} {name}: {count} version element(s)") +if args.write: + for path, content in changes: + path.write_bytes(content) +result = subprocess.run(["git", "grep", "-l", "-I", "-F", "--", args.old], + cwd=root, capture_output=True, text=True) +if result.returncode not in (0, 1): + sys.exit(result.stderr) +others = [name for name in result.stdout.splitlines() if name not in expected] +print("Other tracked references to review manually:") +print("\n".join(others) if others else "(none)") +print("Review dependency version elements, release notes and packaging inputs manually.") +print("The signing script takes runtime version/RC inputs; it is not rewritten.") +print("Review git diff." if args.write else "Dry run; pass --write to apply.") +PY diff --git a/skills/roller-release/scripts/check-release.sh b/skills/roller-release/scripts/check-release.sh new file mode 100755 index 000000000..63e8e1010 --- /dev/null +++ b/skills/roller-release/scripts/check-release.sh @@ -0,0 +1,205 @@ +#!/usr/bin/env bash +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Verify a staged Roller release candidate the way a voter will. +# +# check-release.sh [--build] +# +# Checks every archive in for: a valid detached signature, a matching +# checksum, an RSA signing key (>= 2048; warns below 4096), a +# signature digest that is not SHA-1, and LICENSE/NOTICE inside the archive. +# The source archive is additionally checked for stray compiled artifacts, and +# with --build it is unpacked and built on the current JDK. +# +# Exit status is the number of failed checks, so it is usable in a gate. + +set -uo pipefail + +dir="${1:?usage: check-release.sh [--build]}" +do_build="" +[ "${2:-}" = "--build" ] && do_build=1 + +fail=0 +pass() { printf 'ok %s\n' "$1"; } +bad() { printf 'FAIL %s\n' "$1"; fail=$((fail + 1)); } +warn() { printf 'warn %s\n' "$1"; } + +shopt -s nullglob +archives=( "$dir"/*.tar.gz "$dir"/*.zip ) + +if [ "${#archives[@]}" -eq 0 ]; then + bad "no .tar.gz or .zip archives found in $dir" + exit $fail +fi + +# --- signatures ------------------------------------------------------------- + +for f in "${archives[@]}"; do + base=$(basename "$f") + + if [ ! -f "$f.asc" ]; then + bad "$base has no detached signature" + continue + fi + + if ! gpg --verify "$f.asc" "$f" >/dev/null 2>&1; then + bad "$base signature does not verify (is the key imported?)" + continue + fi + pass "$base signature verifies" + + # Which key, and does it meet policy? algo 1 = RSA, 17 = DSA. + keyid=$(gpg --list-packets "$f.asc" 2>/dev/null \ + | awk -F'keyid ' '/keyid/ {print $2; exit}') + digest=$(gpg --list-packets "$f.asc" 2>/dev/null \ + | awk -F'digest algo ' '/digest algo/ {split($2,a,","); print a[1]; exit}') + + if [ -n "$keyid" ]; then + read -r algo bits <<<"$(gpg --with-colons --list-keys "$keyid" 2>/dev/null \ + | awk -F: '/^pub:/ {print $4, $3; exit}')" + case "$algo" in + 1) if [ "${bits:-0}" -ge 4096 ]; then + pass "$base signed with RSA-$bits key $keyid" + elif [ "${bits:-0}" -ge 2048 ]; then + warn "$base signed with RSA-$bits key $keyid (policy prefers 4096)" + else + bad "$base signed with RSA-$bits key $keyid (policy requires >= 2048)" + fi ;; + 17) bad "$base signed with DSA key $keyid — policy says do not use DSA" ;; + "") warn "$base signing key $keyid not in local keyring; cannot check strength" ;; + *) warn "$base signed with key $keyid of unexpected algorithm $algo" ;; + esac + fi + + # digest algo 2 = SHA-1. + if [ "${digest:-}" = "2" ]; then + bad "$base signature uses SHA-1 — policy says avoid further use of SHA-1" + elif [ -n "${digest:-}" ]; then + pass "$base signature digest algo $digest (not SHA-1)" + fi +done + +# --- checksums -------------------------------------------------------------- + +for f in "${archives[@]}"; do + base=$(basename "$f") + found="" + for alg in 512 256; do + [ -f "$f.sha$alg" ] || continue + found=1 + have=$(shasum -a "$alg" "$f" | awk '{print $1}') + + # shasum(1) format is what the policy asks for, and "shasum -c" is + # the authoritative check, so try it first. + if ( cd "$dir" && shasum -a "$alg" -c "$base.sha$alg" >/dev/null 2>&1 ); then + pass "$base sha$alg matches" + else + # Legacy "gpg --print-md" checksum formatting writes + # ":" then uppercase hex in space-separated groups across + # following lines; "shasum -c" cannot read that. Normalise to bare + # lowercase hex so a correct digest in the wrong format is reported + # as a format problem rather than a mismatch. + # + # Truncate to the digest's own length. Filenames contain hex + # letters ("apache" starts with one), so an unbounded [0-9a-f]+ + # runs past the digest and into the name. + width=$(( alg / 4 )) + want=$(tr -d ' \n' < "$f.sha$alg" \ + | sed 's/^.*://' \ + | tr 'A-Z' 'a-z' \ + | grep -oE '[0-9a-f]+' \ + | head -1 | cut -c1-"$width") + + if [ "$want" = "$have" ]; then + warn "$base sha$alg digest is correct, but the file is not in" + warn " shasum(1) format or names a different path, so" + warn " 'shasum -c' fails for downloaders" + else + bad "$base sha$alg does NOT match" + fi + fi + done + [ -n "$found" ] || bad "$base has no .sha512 or .sha256" + [ -f "$f.md5" ] && bad "$base has a .md5 — MD5 files must not be published" + [ -f "$f.sha1" ] && bad "$base has a .sha1 — SHA-1 files must not be published" + [ -f "$f.sig" ] && bad "$base has a .sig — binary signatures must not be published" +done + +# --- archive contents ------------------------------------------------------- + +list_archive() { + case "$1" in + *.tar.gz) tar tzf "$1" 2>/dev/null ;; + *.zip) unzip -Z1 "$1" 2>/dev/null ;; + esac +} + +for f in "${archives[@]}"; do + base=$(basename "$f") + names=$(list_archive "$f") + + for required in LICENSE NOTICE; do + if printf '%s\n' "$names" | grep -qE "(^|/)$required(\.txt)?$"; then + pass "$base contains $required" + else + bad "$base is missing $required" + fi + done + + case "$base" in + *source*) + junk=$(printf '%s\n' "$names" | grep -E '\.(class|jar|war)$' || true) + if [ -n "$junk" ]; then + n=$(printf '%s\n' "$junk" | wc -l | tr -d ' ') + bad "$base (source) contains $n compiled file(s):" + printf ' %s\n' $junk + echo " A source release should not ship third-party binaries." + echo " Review packaging and licensing before the vote." + else + pass "$base (source) has no compiled artifacts" + fi ;; + esac +done + +# --- optional build --------------------------------------------------------- + +if [ -n "$do_build" ]; then + src="" + for candidate in "${archives[@]}"; do + case "$candidate" in *source*.tar.gz) src="$candidate"; break ;; esac + done + if [ -z "$src" ]; then + warn "no source tar.gz found; skipping build check" + else + tmp=$(mktemp -d) + if tar xzf "$src" -C "$tmp" \ + && ( cd "$tmp"/*/ && mvn -q -ntp -DskipTests=true install >/dev/null 2>&1 ); then + pass "$(basename "$src") builds from source" + else + bad "$(basename "$src") does NOT build from source" + fi + rm -rf "$tmp" + fi +fi + +echo +if [ "$fail" -eq 0 ]; then + echo "All checks passed. Still download it yourself and look at it." +else + echo "$fail check(s) failed — do not call the vote." +fi +exit "$fail" diff --git a/skills/roller-security/SKILL.md b/skills/roller-security/SKILL.md new file mode 100644 index 000000000..6f29f7ac7 --- /dev/null +++ b/skills/roller-security/SKILL.md @@ -0,0 +1,120 @@ +--- +name: roller-security +description: Triage Apache Roller security reports, maintain private case tracking, prepare CVE records, coordinate fixes and reporter review, and prepare disclosure with a release. +--- + + + +# Apache Roller security response + +Use this workflow for vulnerability response, not ordinary bug fixes. The PMC +owns acceptance, severity, release scope, and disclosure decisions. This skill +supports those decisions; loading it does not authorize messages or publication. +Parts are designed to work with Obsidian (the triage- directory is a vault), +but Obsidian is optional. + +## Confidentiality and project practice + +Keep reports, reproductions, CVE reservations, reporter identities, investigation +notes, and disclosure schedules in a private workspace. If using `triage-/` +in a checkout, verify its local Git exclusion before adding case material. +Exclusion prevents accidental additions; it is not access control and does not +remove material from Git history. Never force-add private records. + +Roller uses public code review for fixes. Follow the PMC's agreed workflow and +keep vulnerability characterization out of branch names, commits, PR text, tests, +and comments until disclosure. Neutral wording alone does not prove that a diff +is safe: review what the entire change reveals, including related cases. Resolve +uncertainty with the PMC and ASF Security. Do not assume that derivability from +public source makes an unannounced finding appropriate for publication. + +Keep only generic procedures and synthetic templates in this skill. Do not add +live portal screenshots, case-derived examples, or release execution notes. + +## Read the case before acting + +1. Locate the private workspace and read its summary and the item's `TRACKING.md`. +2. Read the original report, reproduction evidence and `IMPLEMENTATION.md`. +3. Check the implementation branch, review state and release target against the + actual repository. Do not infer that a message was sent from a draft file. +4. Reconcile `CVE_FORM.md` before using the portal or drafting an advisory. + +For new cases, copy `assets/item-template/` into a private item directory. +`TRACKING.md` frontmatter stores facts; checkboxes store workflow state. The +terminal board reads those files directly: + +```sh +python3 skills/roller-security/scripts/triage-status.py "$TRIAGE_DIR" +``` + +Set `TRIAGE_DIR` to the actual private workspace. Commands above run from the +Roller checkout root; when installed elsewhere, resolve scripts relative to this +skill. Plain Markdown editing works; Obsidian Tasks is optional. Read +[tracking conventions](references/obsidian-tasks-tracking.md) when creating or +migrating records. Keep completion dates and mark irrelevant tasks explicitly. + +## Workflow + +Read [ASF process and routing](references/asf-process.md) for policy sources and +project-specific decisions. Verify current policy when performing the workflow. + +1. Acknowledge receipt without inventing a verdict or fix commitment. +2. Investigate reachability, actor privileges, configuration, affected versions, + impact and duplicates. Record evidence separately from inference. Use the + [codebase orientation](references/roller-codebase-map.md) to find entry points. +3. Record the PMC's acceptance, rejection or duplicate decision. Explain a + rejection with verified reasons; re-evaluate each report independently. +4. Tell the reporter the accepted remedy and any agreed schedule. Distinguish + planned work from completed work. Check existing credit preferences first; + ask when unclear. Use [communication templates](references/comms-templates.md). +5. Assess [severity and CVSS](references/severity-and-cvss.md), reconcile the CVE + worksheet, and request an ID from ASF Security through the + [portal workflow](references/cve-portal.md). +6. Implement and review the fix under the agreed project workflow. Demonstrate + that regression tests detect the reported behavior and pass with the fix; + keep sensitive reproduction evidence private. Use the target branch's + supported JDK and documented test commands, not a machine-specific SDK path. +7. Give the reporter the fix and draft advisory for comment with a reasonable + deadline. Coordinate merge timing with the PMC; the ASF default places + reporter review before commit. Record any agreed project variation. +8. Release the approved fix. The companion `roller-release` skill covers the + mechanics when available; otherwise use the project's release documentation. +9. Coordinate disclosure with release availability, verify announcement recipients, + update public security information, and add announcement references to the CVE. + Do not rewrite pushed Git commits to add CVE IDs. + +## Multiple reports + +Give each case an owner and next task. Record shared code and duplicate +relationships in frontmatter. Coordinate merge order when fixes overlap, and +consider whether publishing one change reveals another case. Set release targets +explicitly; the board's release gates are reminders, not release authorization. + +## Helpers and limits + +- `scripts/triage-status.py [--mine ]` derives stages, next + tasks, blockers and consistency warnings without writing case data. +- `scripts/migrate-status.py [--write]` previews migration from + legacy `status.yml`; writes only when requested and retains legacy originals. +- `scripts/check-private.sh [--range ] [--pr ]` checks + common private paths and wording. Use explicit commit endpoints such as + `master..HEAD` (or `master...HEAD`); a lone revision such as `HEAD` is rejected. + Without `--range`, it checks the last 20 commit messages. + It is a heuristic, not publication approval: + manually inspect the full diff, filenames, screenshots, archives and PR text. + Public security documentation can legitimately trigger its vocabulary checks. diff --git a/skills/roller-security/assets/SUMMARY.md b/skills/roller-security/assets/SUMMARY.md new file mode 100644 index 000000000..15c46eb05 --- /dev/null +++ b/skills/roller-security/assets/SUMMARY.md @@ -0,0 +1,92 @@ + + +# Apache Roller — [year] security triage (batch summary) + +> **PRIVATE — do not commit, do not discuss publicly.** `triage-[year]/` is +> to be verified Git-excluded before use. These are undisclosed reports under the ASF +> security process. + +The queries below are optional views for Obsidian with the Tasks plugin. +Each item's `TRACKING.md` is authoritative: YAML frontmatter stores structured case +facts, and Tasks checkboxes store all workflow state. Do not copy stage or next +actions into prose here; they will drift. + +## Open work + +```tasks +not done +tags include #roller-security +group by folder +sort by priority +sort by due +``` + +## Blocked work + +```tasks +not done +tags include #roller-security/blocked +group by folder +``` + +For a terminal-friendly board and consistency checks, run: + +```sh +python3 skills/roller-security/scripts/triage-status.py triage-[year]/ +``` + +This file is for what the board cannot hold: provenance, the reasoning behind +decisions, and the shape of the batch. + +## Provenance + +- Source: [who forwarded, from where, on what date] +- Reporter(s): [name, contact, any affiliation] +- Submitted: [date] Verified against: [version @ commit] +- Dashboard: +- Notes on the forward itself: [what was supplied and what remains to be independently verified] + +## Batch shape + +Which items cluster, and why it matters for sequencing: + +- **[cluster name]** — items NN, NN. Same underlying pattern; one fix branch or + coordinated branches. [Merge order.] +- **[cluster name]** — items NN, NN. Shared configuration prerequisite; assess each item independently. +- **Duplicates / merges** — [item NN partly duplicates the closed report of + DATE; CVE merge or split discussed with security@apache.org on DATE]. + +## Release plan + +Target: [version]. Rationale for bundling or splitting. Anything whose severity +or reporter deadline argues for its own release, and the date that decision gets +revisited. + +## Decision log + +Reasoning that would otherwise be lost. Append, don't rewrite — when a reporter +or ASF Security asks in six months why something was closed, this is the answer. + +| Date | Item | Decision | Reasoning | +|---|---|---|---| + +## Open questions for the PMC + +Things needing a `private@roller.apache.org` discussion rather than one person's +call — scope of the security model, whether a behaviour is intended, whether to +publish a Roller security page. diff --git a/skills/roller-security/assets/item-template/CVE_FORM.md b/skills/roller-security/assets/item-template/CVE_FORM.md new file mode 100644 index 000000000..532cffa5c --- /dev/null +++ b/skills/roller-security/assets/item-template/CVE_FORM.md @@ -0,0 +1,92 @@ + + +# CVE form — item NN: short title + +> **PRIVATE:** unpublished CVE entry worksheet; do not commit or disclose. + +**CVE disposition:** separate CVE +**Portal record URL:** not yet created + +## Record + +| Portal field | Exact value | +|---|---| +| CVE ID | not yet allocated | +| Apache PMC | `roller` | +| Project list(s) | `dev@roller.apache.org, user@roller.apache.org` | +| State | `DRAFT` | + +## Identity + +| Portal field | Exact value | +|---|---| +| Title | … | +| Problem type | `CWE-NNN: Canonical Title` | +| Source of vulnerability discovery | [verified discovery source] | + +## Affected product + +| Portal field | Exact value | +|---|---| +| Product name | `Apache Roller` | +| Source repository | `https://github.com/apache/roller` | +| Default status | `unknown` | + +| Status | Version | Less than | Less than or equal | Version type | +|---|---|---|---|---| +| affected | … | | | `semver` | + +## CVE Description + +```text +Exact public text to paste. +``` + +## References + +| URL | Tag(s) | +|---|---| +| https://… | `patch` | + +## Rating Metric + +| Portal field | Exact value | +|---|---| +| Editor choice | `ASF severity rating` (or `Other` when the direct choice is absent) | +| Generated metric | `Other` / `Textual description of severity` | +| Text | [agreed ASF severity] | + +## CVSS (optional) + +| Version | Scenario | Vector | Score | +|---|---|---|---| +| `CVSS:…` | `GENERAL` | [vector for the selected CVSS version] | `…` | + +## Credits + +| Language | Public name | Role | +|---|---|---| +| `en` | … | `finder` | + +Add `Internal references`, `Defects`, `Advisory ID`, platforms, modules, source +files/routines, Timeline, Tags, or internal Comments only when there is an actual +value to enter. For a duplicate/merged report, use a short no-record form; for a +pending report, record the pending disposition and unresolved decisions rather +than a speculative form. `TRACKING.md` owns workflow state; `IMPLEMENTATION.md` +owns the settled remedy; `REPRODUCTION.md` owns evidence — this worksheet holds +only portal-entry values. diff --git a/skills/roller-security/assets/item-template/IMPLEMENTATION.md b/skills/roller-security/assets/item-template/IMPLEMENTATION.md new file mode 100644 index 000000000..9ceae6cb8 --- /dev/null +++ b/skills/roller-security/assets/item-template/IMPLEMENTATION.md @@ -0,0 +1,70 @@ + + +# Item NN — [title] + +> **PRIVATE.** Undisclosed vulnerability under the ASF security process. This +> directory must be verified Git-excluded before recording case details. Do not commit, do not +> paste into a public issue, do not quote in a commit message. + +Case metadata and workflow: [[TRACKING]] + +Branch base: `master` @ `[sha]` + +## Verification + +What was checked, and how. The four things that decide severity: + +- **Reachability** — entry point, and the filter/interceptor path that reaches + it. Name files and lines. +- **Actor** — anonymous / any authenticated user / `EDIT_DRAFT` / weblog admin / + global admin. Justify it; this is the biggest severity lever. +- **Preconditions** — default config or not. Quote the property and its default + from `roller.properties`. +- **Duplicate check** — which other items and previously closed reports were + checked, and the outcome. + +State plainly what was demonstrated versus inferred. Independently verify +forwarded summaries and record unresolved assumptions. + +## Approach + +The fix pattern, and why this one rather than the alternatives. If the codebase +already contains a safe version of this pattern elsewhere, name it — matching an +existing idiom is easier to review and less likely to regress. + +## Changes + +| File | Change | +|---|---| + +## Verified safe, deliberately NOT changed + +Call sites that look like they belong in the fix but don't, with the reason for +each. This section is worth as much as the fix itself: it is what stops the next +person re-opening settled ground, and it is what a reviewer checks first. + +## Tests + +Test classes, behavior covered, and evidence that the tests detect the reported +problem and pass with the fix. Record any limits to the verification. + +Full suite result and JDK used: [N run, N failures, N errors, N skipped] + +## Open questions + +Anything left for the PMC or the reporter to decide. diff --git a/skills/roller-security/assets/item-template/TRACKING.md b/skills/roller-security/assets/item-template/TRACKING.md new file mode 100644 index 000000000..043dae757 --- /dev/null +++ b/skills/roller-security/assets/item-template/TRACKING.md @@ -0,0 +1,109 @@ +--- +id: "NN" +slug: short-kebab-slug +title: >- + One-line private description of the report. + +# Structured case metadata belongs in this frontmatter. Workflow state does not: +# task checkboxes below are the sole source for stage, next action, blockers, +# action dates, and individual task assignments. +owner: # case coordinator; blank = unclaimed + +# Provenance +reporter: "name " +reported_on: # YYYY-MM-DD +forwarded_on: # YYYY-MM-DD +thread: # private mail-thread link +verified_against: # version and full commit ID + +# Assessment facts +decision: pending # pending | accept | reject | duplicate | wont-fix +actor: # anonymous | authenticated | edit_draft | weblog_admin | global_admin +preconditions: >- + Unknown. +affected: +severity: # critical | important | moderate | low +cvss: # CVSS:3.1/AV:.../... +cvss_score: + +# External identifiers and implementation artifacts +cve: # CVE-YYYY-NNNNN +branch: # name the change, never the flaw +fix_commit: +tests: [] +target_release: +credit: >- + Use the reporter's stated preference; ask if unclear. +announcement_url: + +# Cross-item relationships +related: [] +duplicates: [] +shares_code_with: [] +blocks: [] +--- + + + +# Item NN — title + +> **PRIVATE.** Undisclosed vulnerability under the ASF security process. Verify this +> workspace is excluded from Git before recording case details. Do not commit, +> paste into a public issue, or quote in a commit message. + +## Workflow + +These checkboxes are the authoritative status record. Complete them in Markdown +or Obsidian Tasks and preserve the `✅ YYYY-MM-DD` completion date. Keep the +`#roller-security/item-NN` and milestone tags intact. + +When one path does not apply, complete its tasks and add `#not-applicable`; +never delete checklist rows. Add `#roller-security/blocked` to an open task and +put the blocker in that task's text. Add `#owner/name` to assign an individual +task when its owner differs from the case coordinator above. + +### Triage + +- [ ] Acknowledge receipt to the reporter #roller-security/item-NN #roller-security/stage/acknowledged +- [ ] Investigate: reachability, minimum actor, preconditions, duplicates — record in `IMPLEMENTATION.md` #roller-security/item-NN #roller-security/stage/investigating + +Complete exactly one of the next two normally; complete the other with +`#not-applicable`. For a rejection, mark every later task `#not-applicable` too. + +- [ ] Accept the report #roller-security/item-NN #roller-security/stage/accepted +- [ ] Reject, merge, or close the report and explain why to the reporter #roller-security/item-NN #roller-security/stage/rejected + +### Fix + +- [ ] Tell the reporter we accept it, intend to fix it, and how #roller-security/item-NN +- [ ] Rate ASF severity (and CVSS if needed), prepare `CVE_FORM.md`, request the CVE, and record its ID #roller-security/item-NN #roller-security/stage/cve-requested +- [ ] Prepare the fix and regression tests under the agreed review workflow #roller-security/item-NN +- [ ] Review the fix: supported-JDK tests and disclosure review #roller-security/item-NN #roller-security/stage/fix-ready +- [ ] Send the fix and draft advisory to the reporter for comment; agree a deadline #roller-security/item-NN +- [ ] Merge the reviewed fix under the agreed workflow #roller-security/item-NN #roller-security/stage/fix-committed + +### Release and disclosure + +- [ ] Release the fixed Roller version #roller-security/item-NN #roller-security/stage/released +- [ ] At the agreed disclosure point, advance the portal record and verify announcements #roller-security/item-NN #roller-security/stage/announced +- [ ] Add the announcement permalink to the CVE record, update the batch summary, and close #roller-security/item-NN #roller-security/stage/complete + +## Activity notes + +Use dated bullets for context that is not an actionable task. Do not restate +stage, next action, or blockers here. diff --git a/skills/roller-security/references/asf-process.md b/skills/roller-security/references/asf-process.md new file mode 100644 index 000000000..d6be6a610 --- /dev/null +++ b/skills/roller-security/references/asf-process.md @@ -0,0 +1,47 @@ + + +# ASF process and Roller coordination + +Use the [ASF committer security process](https://www.apache.org/security/committers.html) +as the authoritative workflow. Check the +[project security list directory](https://security.apache.org/projects/) for +current routing; do not infer a list from a project name. + +If Roller has no dedicated security list, use its private PMC list for case +coordination and explicitly copy `security@apache.org`. Existing forwarded +threads help establish recipients, but inspect them before replying. Public +`dev@roller.apache.org` and `user@roller.apache.org` lists are for announcement +at the agreed disclosure point, not live case tracking. + +The default process covers acknowledgement, investigation, acceptance or rejection, +CVE allocation, agreement on a fix, reporter review, commit, release and disclosure. +The default places reporter review before commit. Agree and document any project +variation with the PMC and ASF Security; do not turn one past release's sequence +into a permanent rule. + +After or alongside release announcement, coordinate the advisory through the +portal. Check that it reaches the release-announcement audiences, reporter, +security contact and `oss-security@lists.openwall.com`. Then update project +security information and add the public announcement reference to the CVE. +Do not rewrite pushed Git commits afterward. + +Keep case decisions and evidence private. A code change can reveal a vulnerability +even without security terminology. Review public changes in context; there is no +list of words that makes publication safe. If accidental disclosure occurs, +coordinate the response and timeline with ASF Security rather than assuming +editing a public message removes copies or notifications. diff --git a/skills/roller-security/references/comms-templates.md b/skills/roller-security/references/comms-templates.md new file mode 100644 index 000000000..9023892c2 --- /dev/null +++ b/skills/roller-security/references/comms-templates.md @@ -0,0 +1,88 @@ + + +# Communication templates + +These are synthetic templates. Replace bracketed fields from the private case; +never guess recipients, findings, release commitments, IDs or credit. Drafts +stay in the case directory. Preparing a draft does not authorize sending it. +Use the routing in [asf-process.md](asf-process.md). + +## Acknowledge + +Subject: Re: [original report subject] + +Thank you for reporting this. We have received your report and are investigating. +We will follow up when we have assessed it. Please keep the details private while +we coordinate the response. + +[Name, on behalf of the Apache Roller project] + +## Accept + +Subject: Re: [original report subject] + +We have accepted your report. We plan to address it by [concrete agreed remedy]. +[Agreed target release or next update date, if known.] We will share the fix and +draft advisory for your review. Please let us know if this approach misses any +part of the behavior you reported. + +[Allocated CVE ID, if already available.] +[Confirm existing credit preference, or ask if none was supplied.] + +[Name, on behalf of the Apache Roller project] + +## Reject or close as duplicate + +Subject: Re: [original report subject] + +We investigated [reported behavior] against [revision and configuration]. Our +conclusion is [decision], because [verified evidence and relevant preconditions]. +[For a duplicate, explain the relationship through the private thread.] +Please let us know if we have misunderstood a prerequisite or missed evidence +that changes this assessment. + +[Name, on behalf of the Apache Roller project] + +## Request reporter review + +Subject: Re: [original report subject] + +The proposed fix is available at [agreed review location]. The draft advisory +is [attached or included through the agreed private channel]. Could you review +both by [date and timezone], particularly [specific validation question]? +Our proposed release/disclosure schedule is [agreed schedule]. Please tell us +if you need more time or identify a problem with the fix or advisory. + +Credit is currently [agreed wording]. Please confirm any corrections. + +[Name, on behalf of the Apache Roller project] + +## Public advisory + +Use the portal-generated advisory and review its fields before sending: + +- Subject: [CVE identifier]: Apache Roller: [public title] +- Affected versions: [verified range] +- Severity: [PMC-approved rating] +- Description: [impact and prerequisites sufficient for operator assessment] +- Recommended action: [available fixed release and official download link] +- Credit: [approved wording] +- References: [public advisory and other appropriate public links] + +Do not include private correspondence, internal record links, or reproduction +steps by default. Coordinate recipients and timing with the ASF process. diff --git a/skills/roller-security/references/cve-portal.md b/skills/roller-security/references/cve-portal.md new file mode 100644 index 000000000..bbd078279 --- /dev/null +++ b/skills/roller-security/references/cve-portal.md @@ -0,0 +1,52 @@ + + +# CVE portal workflow + +The ASF's authenticated portal is . Its UI and +schema can change. Use the current record and portal help as the source of truth; +this skill intentionally contains no live screenshots or copied CVE records. + +1. Reconcile the private `CVE_FORM.md` worksheet from tracking, reproduction and + implementation evidence. Ask ASF Security about merged or split records. +2. Allocate through the portal or ASF Security. Record the allocated ID and + private record URL in the case workspace, never in this skill. +3. Enter the product, affected versions, problem type, description, public + references, agreed severity and permitted credit. Use canonical identifiers + and only version ranges supported by the investigation. +4. Inspect generated CVE JSON and announcement previews as well as editor fields. + Unassessed versions must not silently become 'unaffected'. Do not use zero as + an invented lower bound. Verify range inclusivity and the product's default + status after every version edit. +5. If using a Source/JSON editor, start with this record's generated document. + Preserve generated identifiers and private workflow metadata; change only + understood fields. Save and inspect the generated result. Do not paste an + unrelated record or assume that private envelope field names are stable. +6. Keep draft preparation separate from sending announcements or advancing + publication state. Verify the current lifecycle controls and the agreed + release/disclosure timing before either action. + +A generated JSON state is not proof that a CVE has been publicly announced. +Private comments and references must remain outside public record fields and +email bodies. Inspect actual To/Cc/Bcc recipients; do not rely on remembered +portal defaults. Add the public announcement link after disclosure and verify +that it appears in the intended record. + +For duplicate or rejected cases, record the no-separate-CVE disposition rather +than creating a speculative record. Consult the +[ASF process](https://www.apache.org/security/committers.html) for allocation, +review and announcement requirements. diff --git a/skills/roller-security/references/obsidian-tasks-tracking.md b/skills/roller-security/references/obsidian-tasks-tracking.md new file mode 100644 index 000000000..ddc77ccbf --- /dev/null +++ b/skills/roller-security/references/obsidian-tasks-tracking.md @@ -0,0 +1,140 @@ + + +# Obsidian Tasks tracking for security reports + +Keep case files in a private `triage-/` workspace. Obsidian with the Tasks +plugin is optional; plain Markdown editing and the terminal board also work. +Every vulnerability directory contains one `TRACKING.md` and one +`CVE_FORM.md`; copy both from `assets/item-template/` when creating a report. +`CVE_FORM.md` is a portal-entry worksheet and does not own stage, next action or +other workflow state; see `references/cve-portal.md`. + +## One file, two responsibilities + +Do not keep `status.yml` beside `TRACKING.md`. They would contain overlapping +mutable state and eventually disagree. + +Use the YAML frontmatter in `TRACKING.md` only for structured facts that Tasks is +not designed to model: report identity and provenance, the assessment outcome, +CVE identifier, implementation artifacts, release target, credit, and +cross-item relationships. + +Use Tasks checkboxes as the sole source for: + +- current stage (derived from completed `#roller-security/stage/...` tasks); +- next action (the first incomplete workflow task); +- blockers (`#roller-security/blocked` on an incomplete task); +- action dates (`✅ YYYY-MM-DD` and `📅 YYYY-MM-DD`); +- individual assignments (`#owner/name` when different from frontmatter + `owner`, which names the overall case coordinator). + +Never add `stage`, `next_action`, or `blocked_on` to frontmatter. Do not write a +parallel status sentence in `SUMMARY.md` or `IMPLEMENTATION.md`. + +## Keep the checklist short + +The checklist is a workflow tracker, not a procedure manual. One row per action +somebody actually takes and can tick. Detail about +*how* to do a step belongs in `SKILL.md` or the reference files; detail about +what was found belongs in `IMPLEMENTATION.md` and the activity notes. + +A checklist that enumerates every sub-step (rate severity, build the CVSS +vector, write the test, watch it fail, watch it pass, re-fail it, run the +suite…) stops being read, and half-ticked sub-steps make the derived stage +meaningless. Fold them into the one row whose completion they define: "Review +the fix: supported-JDK tests and disclosure review" is +one tickable action, and if it isn't done, it isn't ticked. + +Add a row only when it represents a decision or a hand-off that can be +independently late — sending a message, merging, releasing. Resist adding one +because a step is important; importance is what the skill text is for. + +## Task conventions + +- Preserve `#roller-security/item-NN` on every workflow task. It makes vault-wide + Tasks queries reliable even if a note moves. +- Preserve exactly one `#roller-security/stage/` tag on each milestone + task. `new` means no milestone has been completed. +- Record `✅ YYYY-MM-DD` when completing tasks, manually or through the + Tasks plugin. +- Put a real response deadline on the reporter-review task with + `📅 YYYY-MM-DD`; do not keep a separate deadline field in YAML. +- To block work, add `#roller-security/blocked` and describe what is required in + the task text. Remove the tag when unblocked. +- For alternative or irrelevant tasks, check the task and add + `#not-applicable`. Do not delete it. The board ignores such tasks when deriving + stage, but retaining them proves the step was considered. +- Complete exactly one of the `accepted` and `rejected` milestones normally. + Mark the other `#not-applicable`. For a rejected item, mark all downstream + tasks `#not-applicable` so they disappear from open-work queries. +- Keep reasoning and red/green evidence in `IMPLEMENTATION.md`; a task should + say what must be done, not reproduce sensitive technical detail. + +## Stage derivation + +The terminal board derives the stage from the furthest completed milestone +that is not tagged `#not-applicable`: + +`new → acknowledged → investigating → accepted → cve-requested → fix-ready → fix-committed → released → announced → complete` + +`rejected` is an alternative terminal milestone after `investigating`. +The current action is the first incomplete, applicable task in file order. This +makes the checklist order operational, not decorative. + +## Obsidian dashboard queries + +Put vault-wide queries in `SUMMARY.md`: + + ```tasks + not done + tags include #roller-security + group by folder + sort by priority + sort by due + ``` + +For blocked work: + + ```tasks + not done + tags include #roller-security/blocked + group by folder + ``` + +The terminal script is a validator and compact report, not a second tracker: + + python3 skills/roller-security/scripts/triage-status.py "$TRIAGE_DIR" + +It reads frontmatter and tasks directly and never writes status. + +## Migrating legacy `status.yml` + +Set `TRIAGE_DIR` to the private workspace. Run from the Roller checkout root +(or resolve script paths relative to the installed skill). Run the migration +tool once for an existing workspace: + + python3 skills/roller-security/scripts/migrate-status.py "$TRIAGE_DIR" + python3 skills/roller-security/scripts/migrate-status.py "$TRIAGE_DIR" --write + +The first command is a dry run. `--write` creates each `TRACKING.md` and renames +the source to `status.yml.legacy`; it does not delete the source data. Review the +generated tasks and frontmatter, then delete the legacy backups when satisfied. +The migration completes the imported current-stage milestone and any milestones +with explicit legacy dates; it deliberately leaves other checklist rows open +rather than inventing evidence that they were done. Reconcile those open rows in +Obsidian. Never edit a `.legacy` file or use it to answer status questions. diff --git a/skills/roller-security/references/roller-codebase-map.md b/skills/roller-security/references/roller-codebase-map.md new file mode 100644 index 000000000..13c3441eb --- /dev/null +++ b/skills/roller-security/references/roller-codebase-map.md @@ -0,0 +1,41 @@ + + +# Roller codebase orientation + +This is an architectural starting point, not a list of findings. Confirm paths, +framework versions and behavior on the branch being investigated. Keep case +characterization and reproduction instructions in the private case workspace. + +- `app/src/main/java/org/apache/roller/weblogger/` contains application code. +- `app/src/main/resources/` contains application and framework configuration. +- `app/src/main/webapp/` contains web resources and deployment descriptors. +- `app/src/test/` contains application tests. +- `db-utils/` contains database utilities. +- `assembly-release/` defines release packaging outside the default reactor. +- `it-selenium/` contains browser integration tests. + +Start from the reported entry point and follow the configured request processing +chain into the action or servlet, business service, and persistence layer. Find +configuration defaults in the branch's `roller.properties`; distinguish them +from deployment overrides. Check permissions and ownership in the actual call +path rather than inferring them from class or method names. + +Record the tested revision, runtime, actor, input, observed outcome and expected +invariant in `IMPLEMENTATION.md`. Search related call sites to assess scope, but +avoid expanding a public patch into still-undecided cases without coordination. +Use a supported JDK from the branch's build documentation and CI configuration. diff --git a/skills/roller-security/references/severity-and-cvss.md b/skills/roller-security/references/severity-and-cvss.md new file mode 100644 index 000000000..b297fe057 --- /dev/null +++ b/skills/roller-security/references/severity-and-cvss.md @@ -0,0 +1,39 @@ + + +# Severity and CVSS + +Use the [ASF severity guidance](https://www.apache.org/security/impact_levels.html) +for the project's qualitative rating. Use the relevant +[FIRST CVSS specification](https://www.first.org/cvss/) when a numerical score is +needed. State the CVSS version explicitly; metrics from different versions are +not interchangeable. + +Assess the demonstrated scenario: reachable interfaces, required privileges, +user interaction, configuration prerequisites and confidentiality, integrity and +availability impact. A non-default configuration is a condition to describe, +not automatic grounds for rejection or an arbitrary score reduction. + +Keep the rationale with the private case. Separate verified facts from plausible +but untested outcomes. Do not copy a score from a similar report or infer severity +from a vulnerability class alone. Have the PMC review the rating and vector, and +check that the advisory's prerequisites agree with them. + +Enter the agreed ASF rating in the portal's supported textual severity field. +If also entering CVSS, verify its generated version, vector and score against +the selected specification/calculator. Leave unresolved worksheet fields blank +rather than providing a severity or vector that looks like a settled assessment. diff --git a/skills/roller-security/scripts/check-private.sh b/skills/roller-security/scripts/check-private.sh new file mode 100755 index 000000000..5415ff1e2 --- /dev/null +++ b/skills/roller-security/scripts/check-private.sh @@ -0,0 +1,249 @@ +#!/usr/bin/env bash +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Disclosure lint for Apache Roller security work. +# +# Usage: +# scripts/check-private.sh # check staged changes + recent commits +# scripts/check-private.sh --range master..HEAD +# scripts/check-private.sh --pr # also lint a PR's title and body +# --range requires an explicit BASE..TIP or BASE...TIP commit range. +# +# This checks selected wording and common private paths. Follow the agreed +# project workflow and review what the whole change reveals before publication. +# +# Checks four things that are cheap to get wrong and impossible to take back: +# 1. triage-/ and security notes are excluded from git +# 2. no triage or security note is staged or tracked +# 3. no branch name, commit message, or added line announces the flaw +# 4. with --pr, no PR title or description announces the flaw +# +# This is a lint, not a guarantee. Read the diff, and read the PR body — the +# grep cannot infer case context and can flag innocent words. It does not +# inspect image contents, archives, all unstaged files or full committed diffs. + +set -uo pipefail + +RED=$'\033[31m'; YEL=$'\033[33m'; GRN=$'\033[32m'; OFF=$'\033[0m' +[[ -t 1 ]] || { RED=""; YEL=""; GRN=""; OFF=""; } + +fail=0 +warn=0 +range="" +pr="" + +while [[ $# -gt 0 ]]; do + case "$1" in + --range) + if [[ $# -lt 2 || -z "$2" || "$2" == --* ]]; then + echo "--range requires BASE..TIP (for example, master..HEAD)" >&2 + exit 2 + fi + range="$2"; shift 2 ;; + --pr) pr="$2"; shift 2 ;; + -h|--help) sed -n '2,26p' "$0"; exit 0 ;; + *) echo "unknown argument: $1" >&2; exit 2 ;; + esac +done + +cd "$(git rev-parse --show-toplevel 2>/dev/null)" || { echo "not a git repo" >&2; exit 2; } + +if [[ -n "$range" ]]; then + base=${range%%..*} + tip=${range#*..} + tip=${tip#.} + if [[ "$range" != *..* || -z "$base" || -z "$tip" || "$base" == -* || "$tip" == -* || "$tip" == .* || "$tip" == *..* ]]; then + echo "--range requires explicit BASE..TIP or BASE...TIP endpoints; got '$range'" >&2 + exit 2 + fi + if ! git rev-parse --verify --quiet "${base}^{commit}" >/dev/null || + ! git rev-parse --verify --quiet "${tip}^{commit}" >/dev/null; then + echo "--range endpoints must resolve to commits: '$range'" >&2 + exit 2 + fi +fi + +say_fail() { echo "${RED}FAIL${OFF} $*"; fail=1; } +say_warn() { echo "${YEL}WARN${OFF} $*"; warn=1; } +say_ok() { echo "${GRN}ok${OFF} $*"; } + +# --------------------------------------------------------------------------- # +# 1. private directories are excluded +# --------------------------------------------------------------------------- # +shopt -s nullglob +private_paths=( triage-*/ SECURITY-REPORTS-*.md ) +shopt -u nullglob + +if [[ ${#private_paths[@]} -eq 0 ]]; then + say_ok "no triage directories present" +else + for p in "${private_paths[@]}"; do + if git check-ignore -q "$p"; then + src=$(git check-ignore -v "$p" | cut -f1) + say_ok "$p excluded (${src})" + else + say_fail "$p is NOT excluded from git — add it to .git/info/exclude: + echo '/${p%/}/' >> .git/info/exclude + (use .git/info/exclude, not .gitignore: .gitignore is a committed file + and this exclusion is a local working preference, not project policy)" + fi + done +fi + +# --------------------------------------------------------------------------- # +# 2. nothing private is tracked or staged +# --------------------------------------------------------------------------- # +tracked=$(git ls-files -- 'triage-*' 'SECURITY-REPORTS-*' 2>/dev/null) +if [[ -n "$tracked" ]]; then + say_fail "private files are TRACKED by git:" + echo "$tracked" | sed 's/^/ /' + echo " git rm --cached # then verify with: git log --all -- " +else + say_ok "no private files tracked" +fi + +staged=$(git diff --cached --name-only 2>/dev/null | grep -E '^(triage-|SECURITY-REPORTS-)' || true) +if [[ -n "$staged" ]]; then + say_fail "private files are STAGED:" + echo "$staged" | sed 's/^/ /' +else + say_ok "no private files staged" +fi + +# --------------------------------------------------------------------------- # +# 3. disclosure language +# --------------------------------------------------------------------------- # +# Terms that name a flaw rather than a change. CVE is here because a commit +# carrying a CVE id before announcement is itself the disclosure. +TERMS='CVE-[0-9]{4}|\b(vulnerab|exploit|IDOR|SSRF|SSTI|XXE|CSRF|XSS|RCE)|'\ +'\b(security (fix|issue|flaw|hole|bug))|\b(attack(er)?|malicious|injection)|'\ +'\b(privilege escalation|auth(oriz|entic)ation bypass|arbitrary (code|file))' + +# Retrospective wording: describing what the code USED TO do. TERMS catches the +# words for a flaw; this catches the sentence shape that states the released +# version had one, which is the same disclosure by a different route and is easy +# to write by accident when explaining why a test exists. Warn rather than fail — +# "leak" is ordinary in a resource-handling comment, and past tense is ordinary +# in a javadoc about behaviour that never changed. +RETRO='\b(before|prior to) (this|the) (fix|change|commit|patch)|'\ +'\b(previously|formerly|used to) (allow|permit|expos|resolv|accept|admit|be)|'\ +'\bthe (hole|flaw|defect|weakness|escape|bypass)\b|'\ +'\bleak(s|ed|ing)?\b|'\ +'\b(sandbox escape|path traversal|directory traversal|open redirect)|'\ +'\b(proof.of.concept|PoC)\b|'\ +'\b(unauthenticated|anonymous) [a-z]* ?(can|could|is able to)\b|'\ +'\bwould have (allowed|permitted|exposed|resolved)\b' + +branch=$(git rev-parse --abbrev-ref HEAD 2>/dev/null || echo "") +if [[ -n "$branch" ]] && echo "$branch" | grep -qiE "$TERMS"; then + say_fail "branch name '$branch' names the flaw — rename it after what the change does: + git branch -m " +else + say_ok "branch name '$branch' is clean" +fi + +if [[ -n "$range" ]]; then + subjects=$(git log --format='%h %s%n%b' "$range" -- 2>/dev/null) || { + echo "could not read commits in range '$range'" >&2 + exit 2 + } + label="commits in $range" +else + subjects=$(git log --format='%h %s%n%b' -20 2>/dev/null || true) + label="last 20 commit messages" +fi + +hits=$(echo "$subjects" | grep -inE "$TERMS" || true) +if [[ -n "$hits" ]]; then + say_fail "$label contain disclosure language:" + echo "$hits" | sed 's/^/ /' + echo " Unpushed? amend or rebase. Already pushed? do NOT rewrite —" + echo " tell security@apache.org; it may mean accelerating the release." +else + say_ok "$label are clean" +fi + +# Both greps use -E: in BRE, GNU grep treats \+ as the repetition operator, so +# '^\+\+\+' would silently match (and filter) every added line. +added=$(git diff --cached -U0 2>/dev/null | grep -E '^\+' | grep -vE '^\+\+\+' || true) +if [[ -n "$added" ]]; then + code_hits=$(echo "$added" | grep -inE "$TERMS" || true) + if [[ -n "$code_hits" ]]; then + say_warn "staged additions mention flaw terms — check comments and test names:" + echo "$code_hits" | head -20 | sed 's/^/ /' + echo " Some of these are legitimate (a class genuinely named" + echo " CsrfFilter, an existing javadoc). Judge each one." + else + say_ok "staged additions are clean" + fi +fi + +# Retrospective wording, across both commit messages and staged additions. +retro_scan=$(printf '%s\n%s\n' "$subjects" "$added") +retro_hits=$(echo "$retro_scan" | grep -inE "$RETRO" || true) +if [[ -n "$retro_hits" ]]; then + say_warn "wording describes what the code used to do — read these:" + echo "$retro_hits" | head -20 | sed 's/^/ /' + echo " Saying the old behaviour was wrong states that the released" + echo " version has the flaw. Describe what the code does now instead." +else + say_ok "no retrospective wording in commits or staged additions" +fi + +# --------------------------------------------------------------------------- # +# 4. PR title and body (opt-in, needs gh) +# --------------------------------------------------------------------------- # +# The PR description is the most common place a public fix turns into a public +# disclosure, because a good PR description explains why — and for an +# undisclosed vulnerability the why is the exploit. +if [[ -n "$pr" ]]; then + if ! command -v gh >/dev/null 2>&1; then + say_warn "--pr given but 'gh' is not installed; skipping PR check" + else + pr_text=$(gh pr view "$pr" --json title,body \ + --template '{{.title}}{{"\n"}}{{.body}}' 2>/dev/null || true) + if [[ -z "$pr_text" ]]; then + say_warn "could not read PR #$pr (wrong number, or not authenticated?)" + else + pr_hits=$(echo "$pr_text" | grep -inE "$TERMS" || true) + if [[ -n "$pr_hits" ]]; then + say_fail "PR #$pr title/body contains disclosure language:" + echo "$pr_hits" | sed 's/^/ /' + echo " gh pr edit $pr --body-file " + echo " Note: GitHub keeps the original body under 'edited', and" + echo " the list notification already carried it. Rewriting" + echo " reduces the signal; it does not unpublish it." + else + say_ok "PR #$pr title/body has no flaw terms" + fi + say_warn "read PR #$pr yourself — a body can disclose with no flagged word, + e.g. stating the defect plainly, or saying the tests were verified + failing before the fix (which confirms exploitability in the release)" + fi + fi +fi + +echo +if [[ $fail -ne 0 ]]; then + echo "${RED}Potential disclosure or exclusion problems found.${OFF} Investigate the FAIL lines before proceeding." + exit 1 +elif [[ $warn -ne 0 ]]; then + echo "${YEL}Review the warnings, then proceed if they are false positives.${OFF}" + exit 0 +else + echo "${GRN}Clean.${OFF} Still read the diff — this lint does not understand context." + exit 0 +fi diff --git a/skills/roller-security/scripts/migrate-status.py b/skills/roller-security/scripts/migrate-status.py new file mode 100644 index 000000000..e1637ad65 --- /dev/null +++ b/skills/roller-security/scripts/migrate-status.py @@ -0,0 +1,263 @@ +#!/usr/bin/env python3 +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Migrate legacy status.yml files into Tasks-backed TRACKING.md notes. + +The default is a dry run. With --write, each source becomes status.yml.legacy +after its TRACKING.md has been generated and validated. No source is deleted. +""" + +import argparse +import datetime +import importlib.util +import json +import pathlib +import re +import sys + +SCRIPT_DIR = pathlib.Path(__file__).resolve().parent +TEMPLATE = SCRIPT_DIR.parent / "assets" / "item-template" / "TRACKING.md" +BOARD_PATH = SCRIPT_DIR / "triage-status.py" + +spec = importlib.util.spec_from_file_location("roller_triage_status", BOARD_PATH) +if spec is None or spec.loader is None: + sys.exit(f"cannot load {BOARD_PATH}") +board = importlib.util.module_from_spec(spec) +spec.loader.exec_module(board) + +FIELD_ORDER = [ + "id", "slug", "title", "owner", + "reporter", "reported_on", "forwarded_on", "thread", "verified_against", + "decision", "actor", "preconditions", "affected", "severity", "cvss", + "cvss_score", "cve", "branch", "fix_commit", "tests", "target_release", + "credit", "announcement_url", "related", "duplicates", "shares_code_with", + "blocks", +] +WORKFLOW_FIELDS = { + "stage", "next_action", "blocked_on", "acknowledged_on", + "reporter_last_contact", "reporter_deadline", "accepted_on", "rejected_on", + "cve_requested_on", "cve_state", "fix_ready_on", "fix_committed_on", + "released_on", "announced_on", "completed_on", +} +MILESTONE_DATES = { + "acknowledged": "acknowledged_on", + "accepted": "accepted_on", + "rejected": "rejected_on", + "cve-requested": "cve_requested_on", + "fix-ready": "fix_ready_on", + "fix-committed": "fix_committed_on", + "released": "released_on", + "announced": "announced_on", + "complete": "completed_on", +} + + +def scalar(value): + if value is None or value == "": + return "" + if isinstance(value, list): + return "[" + ", ".join(json.dumps(str(part)) for part in value) + "]" + if isinstance(value, bool): + return "true" if value else "false" + if isinstance(value, (int, float)): + return str(value) + return json.dumps(str(value), ensure_ascii=False) + + +def render_frontmatter(data): + keys = [key for key in FIELD_ORDER if key in data and key not in WORKFLOW_FIELDS] + keys.extend(sorted(key for key in data if key not in keys and key not in WORKFLOW_FIELDS)) + lines = ["---"] + for key in keys: + rendered = scalar(data.get(key)) + lines.append(f"{key}: {rendered}" if rendered else f"{key}:") + lines.append("---") + return "\n".join(lines) + + +def one_line(value): + return re.sub(r"\s+", " ", str(value or "")).strip() + + +def add_done_date(text, date_value): + if not date_value or "✅" in text: + return text + parsed = board.parse_date(date_value) + return f"{text} ✅ {parsed}" if parsed else text + + +def set_checkbox(line, done, not_applicable=False, done_date=None): + match = board.TASK_RE.match(line) + if not match: + return line + text = match.group(2) + if not_applicable and "#not-applicable" not in text: + text += " #not-applicable" + if done: + text = add_done_date(text, done_date) + prefix = line[:line.index("[")] + return f"{prefix}[{'x' if done else ' '}] {text}" + + +def mark_legacy_stage(body, data): + stage = one_line(data.get("stage") or "new") + allowed = set(board.PROGRESS_STAGES) | {"rejected"} + if stage not in allowed: + raise ValueError(f"unknown legacy stage {stage!r}") + + lines = body.splitlines() + task_rows = [] + stage_rows = {} + for index, line in enumerate(lines): + match = board.TASK_RE.match(line) + if not match: + continue + task_rows.append(index) + milestone = board.MILESTONE_TAG.search(match.group(2)) + if milestone: + stage_rows[milestone.group(1)] = index + + target = stage_rows.get(stage) if stage != "new" else None + for index in task_rows: + milestone_match = board.MILESTONE_TAG.search(lines[index]) + milestone = milestone_match.group(1) if milestone_match else "" + date_field = MILESTONE_DATES.get(milestone) + done_date = data.get(date_field) if date_field else None + # A legacy stage proves that milestone was reached, but not that every + # earlier checklist row was actually completed. Preserve explicit dates + # and the current milestone; leave everything else open for review. + done = bool(milestone and (milestone == stage or done_date)) + not_applicable = False + + if stage == "rejected": + if milestone == "accepted" or index > target: + done = True + not_applicable = True + elif stage in board.PROGRESS_STAGES[3:]: + if milestone == "rejected": + done = True + not_applicable = True + + lines[index] = set_checkbox(lines[index], done, not_applicable, done_date) + return "\n".join(lines) + + +def add_carried_forward_task(body, data): + action = one_line(data.get("next_action")) + blocker = one_line(data.get("blocked_on")) + deadline = board.parse_date(data.get("reporter_deadline")) + if not action and blocker: + action = "Resolve legacy blocker" + if not action and deadline: + action = "Reach the reporter-response deadline" + if not action: + return body + + item_id = one_line(data.get("id")) or "NN" + task = f"- [ ] {action} #roller-security/item-{item_id}" + if blocker: + task += f" #roller-security/blocked — blocked on: {blocker}" + if deadline: + task += f" 📅 {deadline}" + carried = "### Carried-forward work\n\n" + task + "\n\n" + marker = "### Triage" + if marker not in body: + raise ValueError("TRACKING.md template has no intake heading") + return body.replace(marker, carried + marker, 1) + + +def add_migration_notes(body, data): + notes = [ + f"- {datetime.date.today()} — Migrated from `status.yml`; legacy stage was " + f"`{one_line(data.get('stage')) or 'new'}`." + ] + for field, label in ( + ("reporter_last_contact", "Legacy reporter last contact"), + ("cve_state", "Legacy CVE portal state"), + ): + value = one_line(data.get(field)) + if value: + notes.append(f"- {label}: {value}.") + return body.rstrip() + "\n\n" + "\n".join(notes) + "\n" + + +def generate(status_path): + data = board.parse_yaml(status_path.read_text(encoding="utf-8")) + item_id = one_line(data.get("id")) or status_path.parent.name.split("-", 1)[0] + slug = one_line(data.get("slug")) or status_path.parent.name.split("-", 1)[-1] + title = one_line(data.get("title")) or slug.replace("-", " ") + data["id"], data["slug"], data["title"] = item_id, slug, title + + template_text = TEMPLATE.read_text(encoding="utf-8") + _, template_body = board.split_frontmatter(template_text, TEMPLATE) + body = template_body.replace("#roller-security/item-NN", f"#roller-security/item-{item_id}") + body = body.replace("# Item NN — title", f"# Item {item_id} — {title}") + body = mark_legacy_stage(body, data) + body = add_carried_forward_task(body, data) + body = add_migration_notes(body, data) + return render_frontmatter(data) + "\n" + body.lstrip() + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("triage_dir", help="e.g. triage-/") + parser.add_argument("--write", action="store_true", help="write notes and rename sources") + args = parser.parse_args() + + root = pathlib.Path(args.triage_dir) + if not root.is_dir(): + sys.exit(f"not a directory: {root}") + statuses = sorted(root.glob("*/status.yml")) + if not statuses: + sys.exit(f"no */status.yml found under {root}") + + generated = [] + for status in statuses: + tracking = status.with_name("TRACKING.md") + legacy = status.with_name("status.yml.legacy") + if tracking.exists() or legacy.exists(): + sys.exit(f"refusing to overwrite migration output in {status.parent}") + try: + text = generate(status) + except (OSError, ValueError) as error: + sys.exit(f"{status}: {error}") + generated.append((status, tracking, legacy, text)) + print(f"would migrate {status} -> {tracking}") + + if not args.write: + print(f"dry run: {len(generated)} item(s); re-run with --write after review") + return + + for status, tracking, legacy, text in generated: + status.rename(legacy) + try: + tracking.write_text(text, encoding="utf-8") + loaded = board.load_tracking(tracking) + expected = one_line(board.parse_yaml(legacy.read_text(encoding="utf-8")).get("stage")) or "new" + if board.val(loaded, "stage") != expected: + raise ValueError( + f"derived stage {board.val(loaded, 'stage')!r} != legacy stage {expected!r}" + ) + except Exception: + if tracking.exists(): + tracking.unlink() + legacy.rename(status) + raise + print(f"migrated {status.parent.name}; retained {legacy.name}") + + +if __name__ == "__main__": + main() diff --git a/skills/roller-security/scripts/triage-status.py b/skills/roller-security/scripts/triage-status.py new file mode 100644 index 000000000..2068d87f6 --- /dev/null +++ b/skills/roller-security/scripts/triage-status.py @@ -0,0 +1,418 @@ +#!/usr/bin/env python3 +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Print a Tasks-backed board for in-flight vulnerability reports. + +Usage: + python3 scripts/triage-status.py triage-/ + python3 scripts/triage-status.py triage-/ --mine "" + +Reads each triage-//TRACKING.md. YAML frontmatter supplies stable +case metadata; Obsidian Tasks checkboxes supply stage, next action, blockers, +and dates. This script validates that source directly and never writes status. + +No dependencies. PyYAML is used when present; otherwise a small parser covers +the frontmatter subset used by the template. +""" + +import argparse +import datetime +import pathlib +import re +import sys + +PROGRESS_STAGES = [ + "new", + "acknowledged", + "investigating", + "accepted", + "cve-requested", + "fix-ready", + "fix-committed", + "released", + "announced", + "complete", +] +DISPLAY_STAGES = PROGRESS_STAGES[:-1] + ["rejected", "complete"] +TERMINAL = {"rejected", "complete"} +MILESTONE_TAG = re.compile(r"#roller-security/stage/([a-z0-9-]+)") +TASK_RE = re.compile(r"^\s*[-*]\s+\[([^\]])\]\s+(.*?)\s*$") +DATE_PATTERNS = { + "due": re.compile(r"📅\s*(\d{4}-\d{2}-\d{2})"), + "done_on": re.compile(r"✅\s*(\d{4}-\d{2}-\d{2})"), +} + + +# --------------------------------------------------------------------------- # +# parsing +# --------------------------------------------------------------------------- # + +def _fallback_parse(text): + """Parse the flat key/value YAML subset used by TRACKING.md frontmatter.""" + data = {} + lines = text.splitlines() + i = 0 + while i < len(lines): + raw = lines[i] + i += 1 + line = raw.split("#", 1)[0].rstrip() if not raw.lstrip().startswith("#") else "" + if not line.strip(): + continue + match = re.match(r"^([A-Za-z_][A-Za-z0-9_]*):\s*(.*)$", line) + if not match: + continue + key, value = match.group(1), match.group(2).strip() + if value in (">-", "|", ">", "|-"): + block = [] + while i < len(lines): + nxt = lines[i] + if nxt.strip() and not nxt.startswith((" ", "\t")): + break + block.append(nxt.strip()) + i += 1 + data[key] = " ".join(part for part in block if part).strip() + continue + if value.startswith("[") and value.endswith("]"): + inner = value[1:-1].strip() + data[key] = [ + part.strip().strip("\"'") for part in inner.split(",") if part.strip() + ] + continue + data[key] = value.strip().strip("\"'") + return data + + +def parse_yaml(text): + try: + import yaml # type: ignore + + data = yaml.safe_load(text) or {} + if isinstance(data, dict): + return dict(data) + except Exception: + pass + return _fallback_parse(text) + + +def split_frontmatter(text, path): + lines = text.splitlines() + if not lines or lines[0].strip() != "---": + raise ValueError(f"{path}: missing YAML frontmatter") + for index in range(1, len(lines)): + if lines[index].strip() == "---": + return "\n".join(lines[1:index]), "\n".join(lines[index + 1:]) + raise ValueError(f"{path}: unterminated YAML frontmatter") + + +def parse_date(value): + if not value: + return None + try: + return datetime.date.fromisoformat(str(value)[:10]) + except (TypeError, ValueError): + return None + + +def clean_task_text(text): + text = MILESTONE_TAG.sub("", text) + text = re.sub(r"\s+#roller-security/item-[A-Za-z0-9_-]+", "", text) + text = re.sub(r"\s+#roller-security/blocked\b", "", text) + text = re.sub(r"\s+#not-applicable\b", "", text) + text = re.sub(r"\s+#owner/[A-Za-z0-9_-]+", "", text) + for pattern in DATE_PATTERNS.values(): + text = pattern.sub("", text) + return re.sub(r"\s{2,}", " ", text).strip() + + +def parse_tasks(body): + tasks = [] + for line_number, line in enumerate(body.splitlines(), 1): + match = TASK_RE.match(line) + if not match: + continue + marker, text = match.groups() + milestone = MILESTONE_TAG.search(text) + due = DATE_PATTERNS["due"].search(text) + done_on = DATE_PATTERNS["done_on"].search(text) + owner = re.search(r"#owner/([A-Za-z0-9_-]+)", text) + tasks.append({ + "line": line_number, + "marker": marker, + "done": marker.lower() == "x", + "text": text, + "display": clean_task_text(text), + "stage": milestone.group(1) if milestone else "", + "applicable": "#not-applicable" not in text, + "blocked": "#roller-security/blocked" in text, + "item_tagged": "#roller-security/item-" in text, + "due": parse_date(due.group(1)) if due else None, + "done_on": parse_date(done_on.group(1)) if done_on else None, + "owner": owner.group(1) if owner else "", + }) + return tasks + + +def val(item, key, default=""): + value = item.get(key, default) + if value is None: + return default + if isinstance(value, list): + return [str(part) for part in value] + return str(value).strip() + + +def as_list(item, key): + value = item.get(key) or [] + if isinstance(value, str): + value = [part.strip() for part in value.strip("[]").split(",") if part.strip()] + return [str(part).strip().strip("\"'") for part in value if str(part).strip()] + + +def derive_stage(tasks): + completed = { + task["stage"] for task in tasks + if task["done"] and task["applicable"] and task["stage"] + } + if "rejected" in completed: + return "rejected" + reached = [stage for stage in PROGRESS_STAGES[1:] if stage in completed] + return reached[-1] if reached else "new" + + +def next_task(item): + if val(item, "stage") in TERMINAL: + return None + for task in item["_tasks"]: + if not task["done"] and task["applicable"]: + return task + return None + + +def load_tracking(path): + frontmatter, body = split_frontmatter(path.read_text(encoding="utf-8"), path) + data = parse_yaml(frontmatter) + data["_dir"] = path.parent.name + data["_path"] = path + data["_tasks"] = parse_tasks(body) + data["stage"] = derive_stage(data["_tasks"]) + data["_next_task"] = next_task(data) + return data + + +def stage_rank(stage): + if stage == "rejected": + return len(PROGRESS_STAGES) + try: + return PROGRESS_STAGES.index(stage) + except ValueError: + return len(PROGRESS_STAGES) + 1 + + +def cell(value, width): + text = str(value or "—") + if len(text) > width: + text = text[:width - 1] + "…" + return text.ljust(width) + + +# --------------------------------------------------------------------------- # +# checks +# --------------------------------------------------------------------------- # + +def check(items, today): + warnings = [] + by_slug = {val(item, "slug") or item["_dir"]: item for item in items} + + for item in items: + name = item["_dir"] + stage = val(item, "stage", "new") + tasks = item["_tasks"] + normal_milestones = { + task["stage"] for task in tasks + if task["done"] and task["applicable"] and task["stage"] + } + + if not tasks: + warnings.append((name, "no Tasks checklist found")) + continue + untagged = [task for task in tasks if not task["item_tagged"]] + if untagged: + warnings.append((name, f"{len(untagged)} workflow task(s) lack an item tag")) + unknown = sorted({ + task["stage"] for task in tasks + if task["stage"] and task["stage"] not in set(PROGRESS_STAGES) | {"rejected"} + }) + for unknown_stage in unknown: + warnings.append((name, f"unknown milestone stage {unknown_stage!r}")) + + missing = [ + milestone for milestone in PROGRESS_STAGES[1:] + ["rejected"] + if not any(task["stage"] == milestone for task in tasks) + ] + if missing: + warnings.append((name, "missing milestone task(s): " + ", ".join(missing))) + + if "accepted" in normal_milestones and "rejected" in normal_milestones: + warnings.append((name, "accepted and rejected are both complete and applicable")) + if stage in PROGRESS_STAGES[2:]: + reached = PROGRESS_STAGES.index(stage) + for prior in PROGRESS_STAGES[1:reached]: + prior_tasks = [task for task in tasks if task["stage"] == prior] + if prior_tasks and not any(task["done"] or not task["applicable"] for task in prior_tasks): + warnings.append((name, f"stage is {stage}, but prior {prior} milestone is open")) + for task in tasks: + if task["done"] and task["stage"] and task["applicable"] and not task["done_on"]: + warnings.append((name, f"completed {task['stage']} milestone has no ✅ date")) + if not task["done"] and not task["applicable"]: + warnings.append((name, f"line {task['line']} is open but tagged #not-applicable")) + if not task["done"] and task["due"] and task["due"] < today: + warnings.append((name, f"overdue task ({task['due']}): {task['display']}")) + + if stage not in TERMINAL and not item["_next_task"]: + warnings.append((name, "no open next task, but item is not terminal")) + if stage not in TERMINAL and stage != "new" and not val(item, "owner"): + warnings.append((name, "unclaimed — no case coordinator while in flight")) + + if stage not in {"new", "acknowledged", "investigating", "rejected"} and not val(item, "severity"): + warnings.append((name, "accepted without a severity rating")) + if stage_rank(stage) >= stage_rank("cve-requested") and stage != "rejected" and not val(item, "cve"): + warnings.append((name, "past CVE request but no CVE id recorded")) + if stage_rank(stage) >= stage_rank("fix-committed") and stage != "rejected" and not val(item, "branch"): + warnings.append((name, "fix committed but no branch recorded")) + if stage_rank(stage) >= stage_rank("announced") and stage != "rejected" and not val(item, "announcement_url"): + warnings.append((name, "announced but announcement_url is empty")) + if stage == "rejected" and val(item, "decision") in ("", "pending", "accept"): + warnings.append((name, "rejected milestone complete but decision does not say so")) + + for other_slug in as_list(item, "shares_code_with"): + other = by_slug.get(other_slug) + if not other: + warnings.append((name, f"shares_code_with unknown item {other_slug!r}")) + continue + branch, other_branch = val(item, "branch"), val(other, "branch") + if branch and other_branch and branch != other_branch: + warnings.append((name, f"shares code with {other_slug} on a different branch " + f"({branch} vs {other_branch}) — coordinate merge order")) + for blocked_slug in as_list(item, "blocks"): + other = by_slug.get(blocked_slug) + if (other and val(other, "stage") not in TERMINAL + and stage_rank(val(other, "stage")) > stage_rank(stage)): + warnings.append((name, f"blocks {blocked_slug}, which is already further along")) + + return warnings + + +# --------------------------------------------------------------------------- # +# output +# --------------------------------------------------------------------------- # + +def render(items, today, mine=None): + rows = sorted(items, key=lambda item: (stage_rank(val(item, "stage")), item["_dir"])) + width = max([len(item["_dir"]) for item in rows] + [4]) + out = ["", f"{'ITEM'.ljust(width)} {'STAGE'.ljust(14)}{'OWNER'.ljust(12)} " + f"{'SEV'.ljust(10)} {'CVE'.ljust(16)} BRANCH", + "-" * (width + 68)] + + current = None + for item in rows: + stage = val(item, "stage", "new") + if stage != current: + current = stage + out.append("") + owner = val(item, "owner") + marker = "* " if mine and owner and mine.lower() in owner.lower() else " " + out.append( + f"{item['_dir'].ljust(width)}{marker}{stage.ljust(14)}" + f"{cell(owner, 12)} " + f"{cell(val(item, 'severity'), 10)} " + f"{cell(val(item, 'cve'), 16)} " + f"{val(item, 'branch') or '—'}" + ) + + out.extend(["", "NEXT TASKS", "-" * (width + 68)]) + for item in rows: + task = item["_next_task"] + if not task: + continue + line = f"{item['_dir'].ljust(width)} {task['display']}" + if task["owner"]: + line += f" [owner: {task['owner']}]" + if task["due"]: + line += f" [due: {task['due']}]" + if task["blocked"]: + line += " [blocked]" + out.append(line) + + counts = {} + for item in items: + stage = val(item, "stage", "new") + counts[stage] = counts.get(stage, 0) + 1 + summary = " ".join(f"{stage}={counts[stage]}" for stage in DISPLAY_STAGES if stage in counts) + out.extend(["", f"{len(items)} items: {summary}"]) + + releases = sorted({val(item, "target_release") for item in items if val(item, "target_release")}) + for release in releases: + pending = [ + item["_dir"] for item in items + if val(item, "target_release") == release + and val(item, "stage") not in TERMINAL + and stage_rank(val(item, "stage")) < stage_rank("fix-ready") + ] + if pending: + out.append(f"Release {release} is gated on {len(pending)} item(s) not fix-ready: " + f"{', '.join(pending)}") + + warnings = check(items, today) + if warnings: + out.extend(["", "CHECKS", "-" * (width + 68)]) + for name, message in warnings: + out.append(f"{name.ljust(width)} {message}") + return "\n".join(out) + "\n" + + +def main(): + parser = argparse.ArgumentParser( + description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter + ) + parser.add_argument("triage_dir", help="e.g. triage-/") + parser.add_argument("--mine", help="mark items coordinated by this name with *") + args = parser.parse_args() + + root = pathlib.Path(args.triage_dir) + if not root.is_dir(): + sys.exit(f"not a directory: {root}") + + paths = sorted(root.glob("*/TRACKING.md")) + legacy = sorted(root.glob("*/status.yml")) + if paths and legacy: + sys.exit("mixed TRACKING.md and status.yml sources found; finish the one-time " + "migration before using the board") + if not paths: + if legacy: + sys.exit("legacy status.yml files found but no TRACKING.md files; run " + "scripts/migrate-status.py first") + sys.exit(f"no */TRACKING.md found under {root} — copy assets/item-template/ " + "into each item directory") + + try: + items = [load_tracking(path) for path in paths] + except (OSError, ValueError) as error: + sys.exit(str(error)) + sys.stdout.write(render(items, datetime.date.today(), args.mine)) + + +if __name__ == "__main__": + main()