Add experimental imports to prodDenylist in import automation workflow - #2242
hareesh-ms wants to merge 1 commit into
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds "Brazil_RuralDevelopmentProgram", "FireWFIGS", and "INPE_Fire_Event_Count" to the production denylist in the Cloud Workflows YAML configuration. Feedback indicates that these same entries must also be added to the duplicate denylist in the Airflow DAG file (import_automation_workflow.py) to maintain consistency and prevent accidental production ingestion.
| - dryRunIngestion: ${string(default(map.get(args, "dryRunIngestion"), false)) == "true"} | ||
| - forceIngestion: ${string(default(map.get(args, "forceIngestion"), false)) == "true"} | ||
| - prodDenylist: | ||
| "Brazil_RuralDevelopmentProgram": true |
There was a problem hiding this comment.
The production denylist is defined in two places: here in the Cloud Workflows YAML configuration (import-automation-workflow.yaml) and in the Airflow DAG file (import_automation_workflow.py under PROD_DENYLIST). To ensure consistent behavior across both execution environments and prevent these experimental imports from being ingested into production via Airflow, please also add "Brazil_RuralDevelopmentProgram", "FireWFIGS", and "INPE_Fire_Event_Count" to PROD_DENYLIST in import-automation/workflow/import_automation_workflow.py.
hareesh-ms
changed the title
1 participant
Summary
Adds the following auto-refresh imports to
prodDenylistinimport-automation/workflow/import-automation-workflow.yaml(maintained in alphabetical order) so they are skipped during production ingestion:Brazil_RuralDevelopmentProgramFireWFIGSINPE_Fire_Event_CountBackground
These imports have been moved to the
experimentalimport group due to licensing issues and should be excluded from production ingestion via the scheduled auto-refresh workflow.