Skip to content

Snt25 659 incidence - #126

Open
EstebanMontandon wants to merge 4 commits into
mainfrom
SNT25-659_incidence
Open

EstebanMontandon wants to merge 4 commits into
mainfrom
SNT25-659_incidence

Conversation

@EstebanMontandon

Copy link
Copy Markdown
Collaborator

Summary

Migrates snt_dhis2_incidence's computation script (snt_dhis2_incidence.ipynb + snt_dhis2_incidence.r) off its bespoke setup helpers onto the shared init_snt_workspace() / load_snt_config() / load_dataset_file() pattern, and fixes two latent bugs surfaced along the way.

  • Setup consolidation: replaced load_utils() / setup_paths() / create_intermediate_data_dir() / install_and_load() / set_env_openhexa() / import_config_json() / config_generic() / config_incidence() / set_fixed_cols() with init_snt_workspace() + load_snt_config() (from code/snt_utils.r) and equivalent inline config assignments. Output paths (INTERMEDIATE_DATA_PATH, DATA_PATH) are unchanged in practice.
  • Fix: careseeking file loading could crash the whole run. read_csv(CARESEEKING_FILE_PATH, ...) is now wrapped in tryCatch — a corrupted/malformed user-provided file now falls back to DHS data instead of stopping the pipeline. Also simplified the two-flag IMPORTED_CARESEEKING_FROM_FILE / IMPORTED_CARESEEKING_FROM_DHS tracking down to one flag, since "not from file" now unambiguously means "from DHS."
  • Fix: join_careseeking_data() silently continued on bad input. When neither ADM1_ID nor ADM2_ID is present, it now stop()s with a clear message instead of logging an "error" severity and falling through to a confusing downstream crash.
  • Reduced implicit global-variable coupling between functions: check_fixed_cols_in_routine(), check_dhis2_indicators_cols_in_routine(), check_pres_col() (renamed from check_PRES_col), load_population_data(), select_routine_dataset_and_filename(), and prepare_disaggregated_indicators()'s call site now take/return explicit values instead of reading or writing globals implicitly.
  • Removed dead code: build_monthly_cases() / build_yearly_incidence() were unused (the notebook has always used inline equivalent logic) and had already drifted from it.
  • Typo fix: coherence_checkes_yearly_incidence()coherence_check_yearly_incidence().

Follow-up (not fixed in this PR)

snt_dhis2_incidence.r still has ~18 functions that set globals via <<- instead of returning a value (prepare_disaggregated_indicators(), select_population_column(), load_dhs_careseeking_data(), load_reporting_rate_data(), check_reporting_rate_data(), enforce_numeric_cols(), handle_zeros_in_reporting_rate(), and the coherence_check_*() family), which snt_dhis2_incidence.ipynb then reads back out of the global environment.

We should try to avoid functions that set global variables the notebook reads back later in an obscure way , at the end this tends to hide side effects behind what looks like a simple call, and makes debugging harder since the cause of a bug can be several cells away from where it surfaces (missing references to variables secretly created). We should also aim for generic, reusable functions instead of wrapping notebook cell computations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant