docs: Fix the AI migration prompt against real consumer builds - #227
Open
tablackburn wants to merge 1 commit into
Open
docs: Fix the AI migration prompt against real consumer builds#227tablackburn wants to merge 1 commit into
tablackburn wants to merge 1 commit into
Conversation
The migration guide's AI prompt had never been exercised. Running it
verbatim against four real public consumer builds (three psake, one
Invoke-Build, PowerShellBuild pins from 0.6.1 to 0.8.2) surfaced five
defects. None produced a build file that would not parse, but two could
silently change build behaviour.
- Step 5 told the agent to change "the -Version on `task ... -FromModule
PowerShellBuild -Version`". Most real consumers use -MinimumVersion;
-Version is an alias for -RequiredVersion, so following the step
literally converts a floor into an exact-version requirement. Every
psake run flagged this as wrong and had to improvise. The step now says
to keep whichever parameter the consumer already uses and change only
its value.
- Step 5 also drove agents to replace floating pins ('latest') with a
hard pin on the unpublished 1.0.0, breaking the consumer's bootstrap
immediately where 'latest' would have kept working. It now says to
leave floating pins floating and report them.
- The baseline step opened with "Before editing anything" but was
numbered after the step that applies edits. It is now step 2.
- The conventions section said $PSBPreference is "populated in
build.ps1". It is set in the psake file's properties block or in
.build.ps1; build.ps1 is only a bootstrap wrapper. The inputs section
now points at the task file and allows for names like
.mymodule.build.ps1.
- The completeness self-check only verified the document's tail. One run
computed a self-consistent but wrong entry count and still passed it.
The check now pairs every Quick Start bullet with its heading, and
step 8 uses that list as the reporting checklist.
Also adds a non-interactive fallback for the prompt's three "ask me"
branches, and corrects the claim about what the prompt was tested
against.
Re-running the revised prompt against the same consumer confirms each
fix: the -MinimumVersion parameter is preserved, the floating pin is left
floating with a review marker, and all 18 entries are paired and
reported.
Refs #159
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011GYJrhbrDzqufaeMqD9QjT
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The migration guide's AI prompt is a v1.0.0 deliverable that had never been run. I ran it verbatim against four real public consumer builds and fixed the five defects it surfaced. This is the "test the AI prompt" half of #159 only — the guide coherence pass and the changelog entry are untouched.
Consumers tested, unmodified, each migrated by a fresh agent given only the prompt and the repository:
jimbrig/PSUtils— psake; has the bogusTest.ScriptAnalysisEnabled, floatinglatestpinsflycastpartnersinc/FreshservicePS— psake; pins PowerShellBuild 0.6.1, psake 4.9.0, Pester 5.3.3pauby/PSTodoWarrior— Invoke-Build; non-default.pstodowarrior.build.ps1, inline PSDepend hashtablemilestonesys/MilestonePSTools— psake; many$PSBPreferencevalues,AlphabeticParamsOrder, bootstrap manifestWhat the prompt got right
The fixes are narrow, so this is worth recording:
WebFetch, citing its summarizing behaviour, and fetched raw text instead.SetOnlineHelpUrlstask matches^online version:in docs front matter, which PlatyPS 1.x renames toHelpUri:; unflagged, every command doc silently loses its online-help URL.Defects fixed
-Versionontask ... -FromModule PowerShellBuild. Three of four fixtures use-MinimumVersion, and-Versionis an alias for-RequiredVersion— following the step literally turns a floor into an exact-version requirement. Every psake run flagged it and improvised. Now: keep whichever parameter the consumer already uses, change only its value.PowerShellBuild = 'latest'with a hard'1.0.0', breaking the bootstrap immediately wherelatestwould have kept working and picked up 1.0.0 on release. Now: leave floating pins floating and report them.$PSBPreferencewas said to be "populated in build.ps1". It is set in the psake file'spropertiesblock or in.build.ps1;build.ps1is a bootstrap wrapper. The inputs section now points at the task file and allows for names like.mymodule.build.ps1.Plus a one-line non-interactive fallback for the prompt's three "ask me" branches, and a correction to the claim about what the prompt was tested against.
Test plan
[Parser]::ParseFile,Import-PowerShellDataFile)-MinimumVersionpreserved, floating pin left floating with a marker, all 18 entries paired and reportedBreaking changes
None. Documentation only.
Refs #159