[DRAFT]: Add manual scores - #2589
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 097eaf76-ae49-44f8-b44d-9841fa2cc1cb
|
|
||
| model_config = ConfigDict(extra="forbid") | ||
|
|
||
| attack_result_id: uuid.UUID = Field(..., description="ID of the attack whose objective is being scored") |
There was a problem hiding this comment.
I think we can/should only rely on message_id ? the attack_id can be found by looking up the message's conversation_id in the attack result table.
Otherwise, there's a flaw where one can send a request with an attack_id and a message_id that are not coherent (i.e. score a message from another attack that, for example, does not have an objective, thereby bypasing this check)
| scorable=MessageScorable(message_piece_ids=(request.message_id,)), | ||
| ) | ||
| score = scores[0] | ||
| if attack.last_score is None: |
There was a problem hiding this comment.
[GHCP generated but Behnam agrees]: Could updating the attack result be an explicit option, such as “Update attack score and outcome”, rather than happening only when last_score is unset?
When selected, the operation should atomically replace last_score with the new manual score, derive and update outcome, and update outcome_reason, regardless of the current last_score. Without it, the score should only be added to the message’s score history.
Assuming manual scores are restricted to true/false, this operation becomes cleaner and unambiguous because the outcome maps directly from the selected score.
|
|
||
| attack_result_id: uuid.UUID = Field(..., description="ID of the attack whose objective is being scored") | ||
| message_id: uuid.UUID = Field(..., description="ID of the message piece to score") | ||
| score_type: Literal["true_false", "float_scale"] = Field( |
There was a problem hiding this comment.
[GHCP generated but Behnam agrees]: Could we restrict manual objective scores to true/false for this initial implementation? Attack outcomes are verdicts, and PyRIT already requires objective scorers to produce true/false scores.
Supporting raw float scores introduces additional semantics around thresholds and whether the float represents auxiliary evidence or the canonical attack verdict. Keeping manual scores boolean avoids that ambiguity and leaves float scoring for a follow-up where its contract can be designed explicitly.
| if not 0 <= success_threshold <= 1: | ||
| raise ValueError("Manual score success threshold must be between 0 and 1.") | ||
|
|
||
| self._value = value |
There was a problem hiding this comment.
I was thinking, it might be a good idea to also have a "user_identifier" field, that we attach to the produced Score's metadata. its value can come from the authenticated user's email or oid which should be available in the FastAPI middleware. We can use this data to know "who" added this manual score.
2 participants
Description
Adds manual scoring for assistant responses. Through the GUI, users can assign true/false or float scores, provide an optional rationale, and configure a per-score success threshold using a synchronized slider and numeric input. Manual scoring requires an attack objective, preserves all scores in the existing conversation history, and promotes the first score to the attack result when no previous last_score exists. The backend validates score values, objectives, thresholds, and attack/message ownership while preserving existing attack scores.
Supports:
• True/false verdicts.
• Float scores between 0 and 1 .
• A per-score success threshold, defaulting to 0.5 .
• Optional rationale.
• Provides synchronized slider and numeric-input controls for float thresholds.
• Persists the selected threshold in score metadata.
• Requires an attack objective before manual scoring:
• Missing objectives are indicated on the score action.
• Clicking the action opens the objective editor with an explanatory warning.
• The backend also rejects manual scores without an objective.
• Allows setting an attack-wide objective from an empty conversation.
• Hides Add objective until a target is configured.
• Allows manual scoring on historical and operator-locked conversations while leaving other mutation restrictions intact.
• Keeps all manual scores in the existing immutable conversation score history.
• When an attack has no last_score , promotes its first manual score to last_score and derives the attack outcome:
• True/false values map directly to success or failure.
• Float values are compared with the submitted success threshold.
• Preserves an existing last_score when additional manual scores are added.
• Validates that the message belongs to the supplied attack before scoring it.
• Extends attack PATCH support to update the shared objective and its hash.
Screenshots:
![Uploading image.png…]()
Tests and Documentation
• Added focused backend route, service, validation, and score-persistence coverage.
• Added frontend coverage for both score types, threshold controls, objective requirements, historical conversations, existing scores, and response-only scoring.
• Ran Ruff, ty , backend unit tests, and focused frontend Jest suites.