Add native AI code-review job (praktika review) - #140
Merged
Conversation
Contributor
|
Workflow [Praktika CI Advanced], commit [95d339a] Summary: ✅ Code ReviewResult: What changed: Adds a native AI-powered PR review command with structured model output, read-only repository investigation tools, bounded GitHub review actions, and a Bedrock OpenAI provider. It also adds dedicated runner infrastructure and permissions, extends GitHub review APIs, and renames the existing Anthropic Bedrock provider.
|
maxknv
force-pushed
the
ai-code-review-job
branch
from
August 21, 2026 08:13
fd2ab94 to
7ff8e63
Compare
Adds an in-package code-review job invoked as `praktika review`. The model is consulted for a structured result only (read-only investigation tools, no shell, no GitHub writes); trusted job code applies the bounded action surface: a top-level summary comment, a batched inline review, and resolve/unresolve/reply on the reviewing bot's own threads (ownership enforced in code). - gh.py: port post_pr_review / post_pr_line_comment / resolve / unresolve review-thread write methods. - ai/provider.py: add provider-agnostic complete() seam; anthropic.py reuses it in on_job_failure. - ai/bedrock_openai.py: new bedrock-openai provider (OpenAI models on Bedrock via the Converse API, boto3 auth). - Rename the Anthropic-on-Bedrock provider "bedrock" -> "bedrock-anthropic" for symmetry with "bedrock-openai". - ai_review.py + `praktika review` CLI; wire a Code Review job into the advanced PR workflow with a repo-local prompt file. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
maxknv
force-pushed
the
ai-code-review-job
branch
from
August 21, 2026 08:16
7ff8e63 to
e403872
Compare
The Code Review job needs bedrock:InvokeModel, which general job runners must not have. Add a dedicated arm-2xsmall-bedrock runner pool whose instance role carries a scoped bedrock:InvokeModel grant, and route the Code Review job to it. - runner_pool.py: RunnerPool now honors ext["iam_statements"], appending them to the RunnerAccess inline policy (mirrors OrchestratorPool). - projects.py: new arm-2xsmall-bedrock pool + scoped Bedrock statement (foundation-model / inference-profile resources only). - settings.py: RunnerLabels.SMALL_ARM_BEDROCK. - praktika_pr_advanced.py: Code Review runs_on the new pool. - test_infra_projects.py: expect the new launch template on the arm64 builder. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
maxknv
force-pushed
the
ai-code-review-job
branch
from
August 21, 2026 08:34
5822e11 to
a484fe5
Compare
When a job was kicked to a pool whose SQS queue does not exist (e.g. the pool is not deployed yet), _dispatch failed and kick() called finish(success=False) with no output, so the check went red but kept the stale "QUEUED: job dispatched to runner pool X" summary -- misleading. _dispatch now returns (ok, reason); kick() completes the check with a clear "Failed to dispatch ... (queue X)" message, and when the failure is a missing queue it adds a hint that the pool is likely not deployed and to run `praktika infrastructure --deploy`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The job check summary/body reported only the runner instance id. Add the runner pool (the job's runs_on) alongside it, so a reader can tell which pool and instance role executed the job -- useful when a pool-specific permission (e.g. the Bedrock code-review pool) is involved. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
gpt-oss is a reasoning model that intermixes analysis with its answer, so
parsing free-text JSON failed ("model reply did not parse into a JSON object").
Switch the bedrock-openai provider to a two-phase flow:
1. investigate with the read-only tools and produce free-text findings
(reasoning_effort=medium; reasoningContent stripped when echoing tool
turns back, which Converse otherwise rejects for gpt-oss);
2. a separate small-context call that FORCES a submit_result tool whose
input schema is response_schema, returning validated arguments as JSON.
Add response_schema to the complete() seam; ai_review passes _REVIEW_SCHEMA
and logs the raw reply on parse failure. Also pass toolConfig on the
exhausted-path summary call (Converse requires it once the history contains
tool blocks). Add network-free provider tests + a schema-aware review stub.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![praktika-gh[bot]](https://avatars.githubusercontent.com/in/3494315?s=60&v=4){kind=small}
Prepend a fixed "---\n### Code Review" header to the posted review body so the comment always has a consistent title and a horizontal rule above it, instead of whatever heading the model wrote (e.g. "Summary"). Instruct the model (prompt + schema) not to add its own top-level title to avoid a duplicate heading. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The push-webhook Lambda only triggers Main CI for branches in ALLOWED_PUSH_BRANCHES. This was hardcoded to "main", so projects whose default branch is "master" silently dropped merge-commit push events and never ran Main CI. - OrchestratorPool now defaults ext["allowed_push_branches"] to [Settings.MAIN_BRANCH] instead of ["main"]. - praktika init scaffolds the branch explicitly into the generated projects.py, so the config is self-documenting rather than relying on the default. Merged into the existing ext dict alongside iam_statements and fixed the pre-existing under-indentation of that line. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two coupled problems seen on live runs:
1. gh api user returns nothing for a GitHub App installation token, so the bot
login was unknown. The model then could not recognize its own review threads
(so it re-posted the same inline finding every run -> duplicate comments) and
all thread actions were skipped ("bot login could not be determined").
Fix: _authenticated_login now falls back to the author of a CI-automatic
comment the bot already posted on the PR (the report/review comment), which
yields e.g. praktika-gh[bot] for App tokens.
2. Even with the login known, guard against re-posting: _post_inline_findings
now drops any finding whose (path, line) already has a bot-authored thread,
and the prompt tells the model not to re-raise an issue that already has one.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mirror the ClickHouse code-review job's gpt-5.x + xhigh reasoning. Bedrock has
the gpt-5.6 family (via global inference profiles), closer than gpt-oss:
- bedrock-openai defaults to global.openai.gpt-5.6-sol at xhigh reasoning; the
advanced PR workflow sets both explicitly on the Code Review job.
- The two OpenAI families take different reasoning fields, so the provider is
now family-aware: gpt-5.x uses nested reasoning:{effort} (supports xhigh),
gpt-oss uses flat reasoning_effort (xhigh clamped to high).
- Add a --reasoning-effort CLI arg plumbed into the provider.
- Drop the misleading cost=$... field from the review logs (no gpt-5.6 pricing
table; token counts are kept).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
maxknv
force-pushed
the
ai-code-review-job
branch
from
August 21, 2026 11:07
08611e9 to
4d56b31
Compare
The GH write helpers (post_updateable_comment, post_pr_review, post_pr_line_comment, resolve/unresolve_pr_review_thread) return False on API failure rather than raising, but the review job ignored their results and always reported OK — so a permissions/API failure produced a falsely-successful (and possibly partial) review. Each apply step now returns its write-failure strings; review() aggregates them and fails the job (with the failures in the Result info) when any write failed. Addresses PR #140 review comment. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The bot login was inferred from the author of a comment containing a CI marker, i.e. from user-controllable text: a PR participant posting that marker could have their login returned and treated as the bot, breaking ownership detection. Drop the login entirely. list_pr_review_threads now requests GitHub's viewerDidAuthor (evaluated server-side against the authenticated token, not spoofable). Thread ownership (resolve/unresolve/reply) and inline-finding dedup now key on viewerDidAuthor; the model is given an authored_by_me flag per thread instead of a BOT_LOGIN. Removes _authenticated_login / _infer_bot_login / _norm_login and the --bot-login CLI arg. Addresses PR #140 review comment. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Review comment now always shows an overall result and a plain change summary, even when there are no findings (previously terse gpt-5.6 output left the comment with neither). Adds change_summary + verdict (no_issues / nits_only / issues_found / blocking_issues) to the structured schema and prompt; the comment renders "Result: <verdict>" and "What changed: <summary>" above the findings. verdict is also surfaced in the job info. - Set reasoning effort to high (was xhigh) for the bedrock-openai default and the Code Review job. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
maxknv
force-pushed
the
ai-code-review-job
branch
from
August 21, 2026 12:01
8916899 to
95d339a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A native, in-package AI code-review job invoked as
praktika review.The model is consulted for a structured result only: it may investigate the checked-out PR with read-only tools (
read_file/grep_repo), but it has no shell and no GitHub write access. Trusted Praktika code then applies a bounded action surface: a top-level summary comment, a single batched inline review, and resolve/unresolve/reply on the reviewing bot's own review threads (thread ownership enforced in code, not trusted from the model).Configuration is entirely via CLI args, so a consuming repo wires it as a plain job command with no new config classes or Settings:
Changes
praktika/gh.py: port the missing PR review-write methods:post_pr_review(batched COMMENT review),post_pr_line_comment,resolve_pr_review_thread,unresolve_pr_review_thread.praktika/orchestrator/ai/provider.py: provider-agnosticcomplete()seam;anthropic.pyextracts its tool-use loop intocomplete()and reuses it inon_job_failure(behavior unchanged).praktika/orchestrator/ai/bedrock_openai.py: newbedrock-openaiprovider: OpenAI models on Bedrock via the Converse API (boto3 auth, no extra dep).bedrocktobedrock-anthropic, for symmetry withbedrock-openai(breaking for externalprovider="bedrock"configs).praktika/ai_review.pyand thepraktika reviewCLI subcommand (--provider/--model/--prompt/--bot-login/--dry-run).Code Reviewjob into the advanced PR workflow with a repo-localci/prompts/code_review.md.