feat(wrapper-generator): follow @odata.nextLink with -All on every list cmdlet - #3743
Open
Joywambui-maina wants to merge 28 commits into
Open
feat(wrapper-generator): follow @odata.nextLink with -All on every list cmdlet #3743Joywambui-maina wants to merge 28 commits into
Joywambui-maina wants to merge 28 commits into
Conversation
Graph declares Edm.Int32/Int64 as "number" with the real type in the format; mapping by type alone emitted double? against Kiota's int? and did not compile. An explicit format now decides the CLR type, mirroring Kiota's own mapping.
…racle audit Auditing every v1.0 GET command in MgCommandMetadata.json against the singularizer surfaced four words where the rules disagree with shipped cmdlet names: Cookies -> "Cooky" (ships as ...HostCookie), Skus kept as-is (ships as Get-MgSubscribedSku), Dns -> "Dn" (ships as Get-MgDomainVerificationDnsRecord), Ios -> "Io" (ships as Get-MgDeviceAppManagementIosManagedAppProtection). Adds two irregulars and two invariants, each with a pinned test, and refreshes the README test count. 82 tests passing. Full-inventory match after fix: 796 of 870 noun segments; the remaining 74 are action/function segments and AutoRest hand renames, tracked separately.
…nt, start edge-case catalog Only 2 of 30 shipped whois-family commands truncate "Whois" to "Whoi"; per review decision the generator emits the corrected ...HostWhois (no alias for the old name), and the parity gate reports it as [CORRECTED] instead of failing. "Statistics" joins the invariants, found via the DEVX Humanizer exception list. edge-cases/naming-edge-cases.md starts the per-class catalog of naming defects. 88 tests passing.
Build-WrapperModule.ps1 turns one OpenAPI doc into an importable module (kiota client + wrappers + csproj + dll + PSD1 manifest), reading the Kiota-compatible docs by default with a hard kiota timeout and per-module doc fallback. Test-WrapperModule.ps1 imports each build in a fresh pwsh and verifies exports, worker pairing, and the sessionless NoGraphSession path. All 35 cmdlet-producing v1.0 modules build and pass.
Compiling all v1.0 modules against freshly generated kiota clients surfaced eight alignment defects, each fixed and pinned by a test: dispatchers re-wrapped worker errors (NoGraphSession was lost); body properties colliding with path ids (published convention: -DeviceId1); bare model types colliding with namespaces and BCL types (now fully qualified, mirroring kiota's move-inside and reserved-name renames at root and in sub-namespaces); collection responses resolved from their own $ref; underscore members (riskEventTypes_v2 -> RiskEventTypesV2); $select/$expand emitted only where declared; re-fetch only where a GET exists; media/content endpoints skipped like $value.
…ant, catalog kiota edge cases
The full-module parity sweep found two shipped-name issues: AutoRest
truncated /places/{id}/checkIns at the preposition (8 commands ship as
*-MgPlaceCheck while Get-MgPlaceCheckInCount keeps "In") - corrected per
policy with gate rows and pinned tests; and "Rights" needs to be an
inflection invariant (Get-MgPrivacySubjectRightsRequest, 42 cmdlets) -
Compliance now matches 23 of 23. New edge-cases/kiota-alignment file
catalogs the compile-found defect classes; README refreshed. 103 tests.
A second operation resolving to an already-written cmdlet file now fails generation with the full collision list instead of silently overwriting it, which is the silent-drop failure mode AutoRest had. OData cast list/item pairs (owners/graph.user) now merge like plain pairs, and the sweep's collisions land as cited NamingOverrides entries: termStore and agreement-file stitches, default-singleton renames (SubSite, DefaultDrive, DefaultCalendarEvent), and nested navs the SDK never shipped. Remaining families are tracked on #3704.
… data Derive-CollisionResolutions.ps1 replays the checked-in collision inventory (212 lines, 365 contested routes) against MgCommandMetadata and emits exact-match resolution data: 191 suppressions (routes the published SDK prunes) and 64 renames (published nouns), each entry carrying its oracle evidence. The files embed into the generator and apply only when UseCollisionData is set; -Validate fails on drift, and a new xunit test runs it on every `dotnet test` so staleness fails the suite instead of depending on someone remembering to run the script by hand. Derivation itself fails on any unclassified or ambiguous route. Only 2 cross-path variant merges exist in all of v1.0 (GroupPhoto, ShareListItem) - deferred with the singleton side kept, cataloged in crosspath-merge-edge-cases.md. Full 39-module v1.0 generation now produces zero collisions; 20 published commands that lost filename races are recovered; exact-name matches rise 5,042 -> 5,098. Also: cmdlets emit into a per-module namespace derived from the client namespace instead of the leftover MgPoC placeholder; Build-WrapperModule's generated csproj references Authentication by a relative path instead of an absolute one; its -Configuration parameter now actually reaches the wrapper generator's own build, not just the final module build; and a pre-existing nullable warning in the list/item pairing check is fixed. 121 tests pass.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
fix(wrapper-generator): fail loudly on cmdlet file collisions
Request bodies bound only top-level primitives, so 4,466 property occurrences across the v1.0 specs had no parameter. Every shape the classifier reaches now binds: referenced models and enums, formatted strings, schema-less UntypedNode values (converted on assignment, nulls dropped to match the published SDK's AddIf), and the numeric INF/NaN union. The invented -Password pair is replaced by the published -PasswordProfile. New gates verify it - omission oracle, coverage sweep, inventory diff, runtime conversions: 0 unbound across all 38 specs, 35 modules build and import, 148 tests. The pre-existing naming-parity gap is tracked separately.
Both lines carried the same singularizer-era fixes as different commits, so the five conflicts resolve to identical rules plus each side's real additions: powershell-v3's docs/edge-cases layout and lean comments win; the packaging line's Rights invariant, ResolveParameterNameCollisions, and PlaceCheckIn corrections survive. README test count set to the measured 121; its no-compile-test claim was already false here and is corrected.
…t-body-properties
- Emit media/content downloads (78 routes), completing the shape list in #3709 - Stamp each cmdlet with a [GraphRoute] attribute so the parity gate reads the operation's route from the compiled assembly rather than reconstructing it from generated C#, removing the cast and parameterized-function exclusions that left 1,669 cmdlets unverified - Order Count before the cast suffix on /$count routes (126 cmdlets) - Drop the unusable -OutFile parameter from content writes returning an entity - Collapse the three parallel OData segment tables into one - Refresh generator docs against measured figures 38/38 modules generate and build, 184/184 tests.
…s' into feat/wrapper-operation-shapes
…e}/wrapper
All 38 modules' Kiota clients and generated cmdlets are committed - 11,719
cmdlet files, 9,051 public names - so a clean checkout builds with only the
.NET SDK. The API version nests under wrapper/ because AutoRest clears
src/{Module}/{version} on regeneration; both projects target netstandard2.0
so one dll path serves Core and Desktop. DirectoryObjects no longer
double-declares publicKeyInfrastructure. Gates and docs re-measured against
this corpus: parity 9,548 of 10,385 joinable, omission oracle 0 failures.
…cation binding Manifests now declare RequiredModules on Microsoft.Graph.Authentication (minimum read from its csproj) and packages no longer embed the Authentication assemblies: a module-local copy at a different version splits the GraphSession static under Windows PowerShell's loader, reporting NoGraphSession while connected. ModuleVersion now equals the package version - the hard-coded placeholder made every versioned-folder install refuse to import. Proven by marker-client probes; package gate passes.
…e base class Microsoft.Graph.Wrapper.Runtime hosts what every generated cmdlet duplicated: GraphClientCmdlet owns -AccessToken/-Headers and transport acquisition - the session path reuses one request adapter keyed to the session HttpClient's identity (rebuilt on reconnect), the token path shares one HttpClient with per-request auth headers instead of leaking a connection pool per call. GraphRouteAttribute, UntypedValue and the bearer provider move here from the per-module Shared.g.cs. 22 pinned tests; emitted code unchanged until the emitter adopts the base class in the follow-up.
Every generated cmdlet now derives from GraphClientCmdlet: transport, the -AccessToken/-Headers surface and Graph error translation come from Microsoft.Graph.Wrapper.Runtime instead of being repeated per file, and the per-module Shared.g.cs is gone. The corpus shrinks ~417k lines (30%) with zero behavioral change: operation inventory unchanged (11,719 = 11,719), parity identical (9,548 of 10,385), omission oracle 0 failures, 38/38 build from a clean index snapshot. Also fixes the parity gate picking the runtime dll for modules sorting after R, and repo-root discovery in git worktrees.
…d Authentication module The first live-tenant run of a wrapper cmdlet crashed casting kiota's ObservabilityOptions: module bins carried the whole Authentication dependency closure (~60 dlls), loading a second identity of each assembly into the default load context beside the one the installed Microsoft.Graph.Authentication serves from its isolated context. Bins now carry only the three wrapper assemblies plus the two kiota dlls Authentication does not ship - Std.UriTemplate (preloaded via RequiredAssemblies: its requester lives in the isolated context and cannot probe the module folder) and Serialization.Multipart. PruneModuleBin removes package assets by package id so kept versions stay derived from the restore graph; DisableTransitiveProjectReferences and Private="false" stop the Authentication project copies. Cmdlet discovery and the parity gate now import Authentication before loading wrapper assemblies - nine cmdlets typed with kiota's Date/Time structs force those loads at class load, and both probes silently degraded without the resolver. Proven live against Graph: session, dispatcher, adapter reuse and 16-way parallel all pass; tools/Test-WrapperLive.ps1 is that gate, kept in the repo because offline gates are structurally blind to resolution faults that only manifest on a real request. Inventory unchanged (11,719 = 11,719), parity identical (9,548 of 10,385), oracle 0 failures, package gate PASS.
The #3712 evidence sweep found four defects between "packages exist" and "testers can install them": the nuspec declared no dependencies, so a clean machine got the wrapper without Authentication and import failed (Install-Module resolves from NuGet metadata, not the manifest); packages versioned as stable 2.39.0, colliding number-for-number with the real SDK's release train; the module GUID was random every build, breaking Update-Module identity across handout iterations; and both checkouts held stale pre-fix nupkgs under shipping ids. Packages now declare Microsoft.Graph.Authentication as an open-floor dependency (matching the manifest minimum and the use-latest ruling), always carry a prerelease label (-Prerelease, default wrapperpreview01), and derive a stable RFC 4122 name-based GUID from the module name - no lookup table, identical across builds. Stale artifacts deleted. Proven three rounds each: 38 of 38 packages carry the dependency, label and distinct stable GUIDs; Save-Module from a folder repository auto-resolves Authentication, and the saved layout imports and completes a live Graph call in a host that can see nothing else; package gate PASS. Also corrects the Std.UriTemplate comment - the AssemblyRef lives in Microsoft.Kiota.Abstractions, not the HTTP library - and the live gate's committed name (tools/Test-WrapperLive.ps1).
Joywambui-maina
requested review from
Peter Ombwa (peombwa) and
Ramses Sanchez-Hernandez (ramsessanchez)
and
a lite review from Copilot
August 24, 2026 21:49
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the generated Microsoft Graph PowerShell wrapper cmdlets to support consistent, pipeline-friendly pagination: list-shaped cmdlets can now follow @odata.nextLink when -All is provided, stream each page to the pipeline as it arrives, and emit a one-time warning when results are truncated due to missing -All. It also adjusts exception handling so PipelineStoppedException (e.g., Select-Object -First, Ctrl+C) is not caught and rethrown as a Graph failure.
Changes:
- Add
-Allto list cmdlets and implement a@odata.nextLinkloop (withWithUrl) that streams pages and re-applies headers on continuation requests. - Cap paging when
-Topis bound (at whole-page granularity) and warn once when a first response includes@odata.nextLinkbut-Allis not supplied. - Update cmdlet
try/catchblocks to excludePipelineStoppedExceptionfrom Graph error translation.
Reviewed changes
Copilot reviewed 300 out of 11723 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| src/Users/wrapper/v1.0/Cmdlets/RemoveMgUserSettingStorage.g.cs | Updates exception handling to allow PipelineStoppedException to propagate rather than being translated into a Graph failure. |
| src/Identity.Partner/wrapper/v1.0/Cmdlets/GetMgTenantRelationshipDelegatedAdminCustomer_List.g.cs | Implements -All pagination via @odata.nextLink and emits a truncation warning when -All is not supplied. |
| src/Applications/wrapper/v1.0/Cmdlets/GetMgApplication_List.g.cs | Adds -All pagination and re-applies ConsistencyLevel + custom headers across continuation requests. |
| src/CloudCommunications/v1.0/custom/ListCmdlet.cs | Introduces/extends the shared runtime paging surface (including an All switch) used by generated list cmdlets. |
Suppressed comments (1)
src/Applications/wrapper/v1.0/Cmdlets/GetMgApplication_List.g.cs:60
- The ConsistencyLevel parameter HelpMessage appears to be copied from the If-Match/ETag description, which is misleading for users. Since this string is repeated across many generated list cmdlets, it should be corrected in the wrapper generator/template (and regenerated) to describe ConsistencyLevel usage (e.g., "eventual" for advanced queries such as $count/$search).
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Joywambui-maina
force-pushed
the
feat/wrapper-pagination
branch
from
August 24, 2026 22:24
bf2e605 to
bc7f1e6
Compare
…dlets List cmdlets emitted only the first page and dropped @odata.nextLink, so scripts silently received partial data. EmitListGet - the single template behind every list-shaped cmdlet - now emits an -All switch that follows each non-empty nextLink to exhaustion, streaming every page to the pipeline before the next request is issued, and the public dispatcher declares the switch so it reaches the worker. A bound -Top caps the total under -All at whole-page granularity, matching the published ListCmdlet's shipped semantics. Without -All a surviving nextLink writes one short warning and costs no extra request - deliberately stronger than the published SDK, which truncates silently; approved in the design spec. Continuations go through kiota's WithUrl and re-apply headers only: the link already carries the query state, and a raw-URL builder ignores query bindings. A pipeline stop passes through the shared catch instead of being re-branded a Graph failure. Decisions and evidence in tools/WrapperGenerator/docs/pagination.md. Proven: 189 generator tests including five pagination pins, and tools/Test-WrapperPaging.ps1 drives a real compiled cmdlet and public dispatcher across a stubbed two-page collection - ten assertions, three rounds, all pass. The regenerated corpus follows in a separate pull request.
Output of the pagination emitter change, with no hand edits: 1,479 list-shaped cmdlets gain -All and the nextLink loop, 1,334 dispatchers declare the switch, and every cmdlet's shared catch gains the pipeline-stop filter. Proven against the sealed pre-change baseline: operation inventory unchanged (11,719 = 11,719), name parity identical (9,548 of 10,385), body-binding oracle 0 failures, package and live gates PASS, and -All walks a real Graph page boundary - 182 items across two server pages, matching a separately calibrated total.
Joywambui-maina
force-pushed
the
feat/wrapper-pagination
branch
from
August 25, 2026 19:38
bc7f1e6 to
c8a833a
Compare
Joywambui-maina
changed the base branch from
feat/wrapper-packaging
to
feat/wrapper-pagination-generator
August 25, 2026 19:38
Peter Ombwa (peombwa)
previously approved these changes
Aug 27, 2026
Joywambui-maina
force-pushed
the
feat/wrapper-pagination-generator
branch
2 times, most recently
from
August 27, 2026 18:55
ca3f753 to
40f7b22
Compare
Base automatically changed from
feat/wrapper-pagination-generator
to
powershell-v3
August 27, 2026 21:40
Joywambui-maina
dismissed
Peter Ombwa (peombwa)’s stale review
August 27, 2026 21:40
The base branch was changed.
Post-squash update merge; content equals the patch-identity-verified rebase.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes proposed:
Add -All to every list-shaped cmdlet (1,479) and its public dispatcher (1,334) — follows @odata.nextLink until the collection is exhausted, streaming each page to the pipeline before requesting the next
Cap the total with a bound -Top at whole-page granularity, matching the published SDK's shipped semantics
Warn once when a first response still carries a nextLink and -All wasn't supplied — no extra request; deliberately stronger than the published SDK, which truncates silently (this was approved in the design spec)
Continue through kiota's WithUrl, re-applying headers only, so ConsistencyLevel and -Headers survive page 2+
Let a pipeline stop (Select-Object -First N, Ctrl+C) pass through both catch layers instead of being re-branded a Graph failure — the change touching all 11,719 generated files
Record the decisions and their evidence in tools/WrapperGenerator/docs/pagination.md