docs(evaluation): use GOOGLE_API_KEY in the script-based evaluation - #410
Merged
vvbandeira merged 1 commit intoSep 24, 2026
Merged
Conversation
The script-based evaluation reads GOOGLE_API_KEY and calls the Gemini API through google-genai. It has no --creds-path option and does not read a service-account file. [vvbandeira: kept only the script_based_evaluation changes. The backend still authenticates Vertex AI with GOOGLE_APPLICATION_CREDENTIALS.] Signed-off-by: Harsh Kumar Patwa <harshkumarpatwa@gmail.com> Signed-off-by: Harsh Kumar <harshkumar3446@gmail.com> Signed-off-by: Vitor Bandeira <vvbandeira@precisioninno.com>
vvbandeira
added a commit
to vvbandeira/ORAssistant
that referenced
this pull request
Sep 24, 2026
* upstream/master: Add Vertex AI Ranking API as reranker option (The-OpenROAD-Project#411) docs(evaluation): use GOOGLE_API_KEY in the script-based evaluation (The-OpenROAD-Project#410) Build(deps): Bump types-requests from 2.32.0.20240622 to 2.33.0.20260906 in /frontend (The-OpenROAD-Project#404) test: add unit tests for evaluation metrics module (The-OpenROAD-Project#409) Signed-off-by: Vitor Bandeira <vvbandeira@precisioninno.com> # Conflicts: # backend/src/chains/hybrid_retriever_chain.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes #232 by @harsh-kumar-patwa, narrowed to the script-based evaluation. The commit keeps the contributor's authorship and sign-offs.
Change
evaluation/script_based_evaluation/.env.sampleandREADME.mdnow ask forGOOGLE_API_KEYinstead of a service-account file.--creds-pathoption and thesecret.jsonrequirement.This matches the code:
config/config.pyreadsGOOGLE_API_KEY, andmodels/gemini_model.pycalls the Gemini API throughgoogle-genai.main.pyhas no--creds-pathargument.Changes from #232 that this PR leaves out
#232 also removed
GOOGLE_APPLICATION_CREDENTIALS,GOOGLE_PROJECT_ID, andGOOGLE_CLOUD_LOCATIONfrom the backend, theseed-credentialstarget, and the Secret CI credential steps. The backend still needs them.ChatVertexAIandVertexAIEmbeddingsauthenticate through Application Default Credentials, which readGOOGLE_APPLICATION_CREDENTIALSinside the Google auth library, not in project code. Without that credential, the Secret CI evaluation cannot start the backend.Relates to #155. A full deprecation needs the backend to move from Vertex AI to the Gemini API.
🤖 Generated with Claude Code