Skip to content

feat: collapse regional producers into one PDL-derived archetype (#33) #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Hackatro wants to merge 7 commits into
base: main
Choose a base branch
from
Open

feat: collapse regional producers into one PDL-derived archetype (#33) #39

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
3672625
Collapse producers onto PDL entity ids (#33 1/2).
Hackatro Aug 16, 2026
5f9f267
Update README.md (#33 1/2)
Hackatro Aug 16, 2026
ba07dce
cleanup: docstring + topology snapshot (#33 1/2)
Hackatro Aug 17, 2026
c3bda86
feat: model regional originators and Rotterdam sourcing (#33 2/2)
Hackatro Aug 25, 2026
5e06feb
feat: export roster-derived map placements (#33 2/2)
Hackatro Aug 25, 2026
b188824
feat: bind regional producer inputs and guard routing (#33 2/2)
Hackatro Aug 25, 2026
a6c152b
fix: lane utilisation calculation (#33 2/2)
Hackatro Aug 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
129 changes: 76 additions & 53 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,13 @@

Agent-based supply chain simulation for the **PROVIDER** research project (BMBF-funded, OFFIS e.V.).

Models the global soya supply chain from Brazilian/US farms through wholesalers, sea transport,
EU processors, and feed manufacturers to EU livestock farms. Scenarios apply KG-derived shock
parameters (drought, port capacity, input price shocks) and observe emergent price and sourcing
behaviour across the chain.
Models the global soya supply chain from Brazil, Argentina, and US farms through wholesalers,
sea transport, EU processors, and feed manufacturers to EU livestock farms. A PDL YAML
describes entities, stages, and shock events; a roster sidecar next to it declares
archetypes and which PDL entities are not modelled. Added trading entities and
edges are listed in the sidecar but not yet wired into the roster builder. The
model applies those shocks (drought,
port capacity, input prices) and records price and sourcing behaviour across the chain.

Built on [Melodie](https://github.com/ABM4ALL/Melodie) (Python ABM framework).

Expand All @@ -16,30 +19,33 @@ Built on [Melodie](https://github.com/ABM4ALL/Melodie) (Python ABM framework).
```
simenv/
├── pyproject.toml
├── web/ ← globe frontend (see web/README.md)
└── src/
└── provider_simenv/
├── main.py ← entry point
├── model.py ← simulation orchestrator
├── scenario.py ← all scenario parameters
├── scenario.py ← engine parameters (counts, costs, sigmas)
├── topology.py ← PDL + roster → agent lists and flow graph
├── environment.py ← global state + price aggregation
├── pdl_loader.py ← PDL YAML → events / entities
├── data_collector.py ← Melodie output registration
├── tick_writer.py ← per-tick PostgreSQL writer
├── db_config.py ← PostgreSQL connection config
├── visualize_sql.py ← plots from SQLite (post-run)
├── visualize_csv.py ← plots from CSV (fallback)
├── export_bundle.py ← CSV run → web/public/bundle.json
├── scenarios/
│ ├── s1-soja.pdl.yaml
│ └── s1-soja.roster.yaml
├── agents/
│ ├── farmer.py ← BRA / USA / EU farmers
│ ├── farmer.py ← producers + EU livestock farms
│ ├── trader.py ← wholesalers + feed traders
│ ├── transport.py ← land + sea transport operators
│ └── process.py ← processors + feed manufacturers
└── data/
├── input/
│ └── SimulatorScenarios.csv ← scenario definitions
└── output/ ← generated at runtime
├── Result_Simulator_*.csv
├── provider-simenv.sqlite
├── price_curves.png
└── volume_flow.png
│ ├── SimulatorScenarios.csv ← working copy Melodie reads
│ └── SimulatorScenarios_template.csv ← edit this; every run copies it
└── output/ ← generated at runtime
└── Result_Simulator_*.csv
```

---
Expand All @@ -54,7 +60,6 @@ simenv/
Melodie>=0.6.0
pandas
numpy
matplotlib
pyyaml
```

Expand All @@ -66,7 +71,7 @@ psycopg2-binary
```

If `sqlalchemy` / `psycopg2-binary` are not installed, the tick writer disables itself
silently and the simulation continues normally. CSV and SQLite outputs are unaffected.
silently and the simulation continues normally. CSV output is unaffected.

### Install

Expand Down Expand Up @@ -97,55 +102,69 @@ pip install '.[db]'

## Running the Simulation

After `pip install -e .`, from the **repository root**:

```bash
# Navigate to the simulation package — Melodie resolves data/ paths from here
cd src/provider_simenv
# Windows: the scenario summary prints characters that fail on cp1252
# $env:PYTHONIOENCODING="utf-8"

# Run
python main.py
# All rows in SimulatorScenarios.csv (no PDL events)
python -m provider_simenv.main

# Shocks and roster from the shipped soja PDL
python -m provider_simenv.main --pdl src/provider_simenv/scenarios/s1-soja.pdl.yaml
```

Alternatively, from the package directory (what the Docker image does):

```bash
cd src/provider_simenv
python main.py --pdl scenarios/s1-soja.pdl.yaml
```

The simulation reads scenarios from `data/input/SimulatorScenarios.csv`, runs all scenarios
in sequence, and writes output to `data/output/`.
Every run copies `SimulatorScenarios_template.csv` onto `SimulatorScenarios.csv` first.
With `--pdl`, the live CSV is then rewritten to two rows: baseline (`id=0`) and one PDL
scenario (`id=1`). Shock magnitudes and timing come from PDL events at runtime, not from
CSV columns.

**What runs automatically:**

1. Simulation loop — all scenarios, 52 steps each, per-tick stdout
1. Simulation loop — each CSV row, `period_num` steps (default 365)
2. Per-tick PostgreSQL writes via `tick_writer.py` (if Postgres is reachable; silent skip otherwise)
3. Post-run: CSV → SQLite merge → `provider-simenv.sqlite`

### Generate Plots (after a run)
### Update the globe frontend after a run

```bash
# From src/provider_simenv/

# Recommended: plots from SQLite — price curves + BRA/USA volume flow
python visualize_sql.py

# Fallback: plots from CSV
python visualize_csv.py
python -m provider_simenv.export_bundle --scenario 1
```

Output PNGs are saved to `data/output/`.
The exporter writes `web/public/bundle.json`. See `web/README.md` for the frontend workflow.

---

## Scenarios

Scenarios are defined in `data/input/SimulatorScenarios.csv`. Each row is one scenario run.
`data/input/SimulatorScenarios.csv` is the working table Melodie reads. Change the
**template**, not the live file — every run overwrites the live copy from the template.

Each row is one run. Columns are engine parameters (agent counts, routing, size sigmas,
storage, length). Producer counts use PDL entity ids (`n_brazil_farms`, `n_argentina_farms`,
`n_us_farms`).

Shocks are not CSV columns. Pass `--pdl`; `EventTracker` applies drought, capacity, and
input-price events from the YAML. The shipped files are `scenarios/s1-soja.pdl.yaml` and
`scenarios/s1-soja.roster.yaml` beside it (archetype declarations and not-modelled
exclusions; trading entities and edges are declared but not yet wired into the roster
builder).

| Parameter | Effect |
|---|---|
| `farm_capacity_bra` | BRA farm output multiplier — `1.0` = normal, `0.7` = 30% drought loss |
| `port_capacity_sa` | SA export port throughput multiplier |
| `fertilizer_price_factor` | Multiplier on BRA farmer fixed costs (GTA Red interventions) |
| `energy_price_factor` | Multiplier on all transport fixed costs |
| `oil_mill_capacity` | EU processor output multiplier |
| `feed_mill_capacity` | Feed manufacturer output multiplier |
| `shock_onset_step` | Step at which shock starts ramping in |
| `shock_ramp_steps` | Steps to ramp from baseline to full shock value |
| `wholesaler_storage_capacity` | Max tonnes a wholesaler can hold per step (default: 20 000 t) |
| `period_num` | Number of simulation steps (default: 52) |
| `n_brazil_farms` / `n_argentina_farms` / `n_us_farms` | Producer agent counts |
| `share_santos_port` | Explicit weight for the Santos route; unspecified route weights are resolved per origin |
| `shock_ramp_steps` | Ramp length when a PDL shock is active |
| `size_sigma_brazil_farms` | Log-normal farm-size spread (`0` = identical farms) |
| `wholesaler_storage_capacity` | Max tonnes a wholesaler can hold per step (default 2857 t/day) |
| `period_num` | Number of simulation steps (default 365) |

---

Expand All @@ -170,6 +189,9 @@ You can also configure a PostgreSQL interface for data storage (see next chapter
docker run --rm -v <path to pdl file directory>:/scenarios provider-simenv --pdl /scenarios/<pdl filename> --postgres-url <PostgreSQL URL string>
```

If the PDL has a roster sidecar, mount the directory that contains both files
(`s1-soja.pdl.yaml` and `s1-soja.roster.yaml`).

---

## PostgreSQL Setup (Optional)
Expand Down Expand Up @@ -206,7 +228,7 @@ Override by constructing `PostgresDBConfig` with different values or passing env
### Verify data after a run

```sql
-- Should return 3 rows, each COUNT = 52 (for a 3-scenario, 52-step run)
-- PDL run: 2 scenarios, each COUNT = period_num (default 365)
SELECT id_scenario, COUNT(*)
FROM "Result_Simulator_Environment"
GROUP BY id_scenario
Expand All @@ -218,6 +240,7 @@ ORDER BY id_scenario;
```
Result_Simulator_Environment
Result_Simulator_BraFarmers
Result_Simulator_ArgFarmers
Result_Simulator_UsaFarmers
Result_Simulator_Wholesalers
Result_Simulator_Processors
Expand All @@ -226,6 +249,9 @@ Result_Simulator_FeedTraders
Result_Simulator_EuFarmers
```

Melodie CSV names follow the agent list (`Result_Simulator_BrazilFarms.csv`,
`ArgentinaFarms`, `UsFarms`). Postgres keeps the keys above.

Tables are dropped and recreated at the start of each full simulation run (first scenario only).
Subsequent scenarios within the same run append to the existing tables.

Expand All @@ -234,7 +260,7 @@ Subsequent scenarios within the same run append to the existing tables.
## docker-compose

You can also run a docker compose that sets up a PostgreSQL database and links it to the simulation container.
Using our helper script, everything is configured automatically.
Using our helper script, everything is configured automatically.
Just run

```bash
Expand All @@ -249,22 +275,19 @@ from the repository root. This builds the database container, if it's not alread
| File | Description |
|---|---|
| `data/output/Result_Simulator_*.csv` | Raw per-agent per-step output written by Melodie |
| `data/output/provider-simenv.sqlite` | All CSVs merged into one SQLite database (post-run) |
| `data/output/price_curves.png` | Soja + feed price development across all scenarios |
| `data/output/volume_flow.png` | BRA vs USA sourcing volumes per scenario |
| `web/public/bundle.json` | Exported run for the globe (`python -m provider_simenv.export_bundle`) |

---

## Known Issues / Notes

- **Run from `src/provider_simenv/`** — Melodie resolves `data/` paths relative to the working directory. Running from the repo root will fail to find input files.
- **Melodie SQLite mode is disabled** — `data_output_type="sqlite"` silently drops all rows due to a missing `conn.commit()` in SQLAlchemy 2.0. The `csv_to_sqlite()` function in `main.py` is used instead and called automatically.
- **`python main.py` needs the package directory** (`src/provider_simenv/`). After `pip install -e .`, `python -m provider_simenv.main` from the repo root works — `Config` resolves `data/` from `main.py`'s location, not the cwd.
- **`run_stepwise()`** in `model.py` is the designated integration hook for external control (e.g. palaestrAI). It yields a state dict `{step, shock_scale, soja_price, feed_price, ...}` after every simulation step.

---

## Project

PROVIDER — AI-based supply chain resilience simulation
OFFIS e.V. – Institut für Informatik, Oldenburg
PROVIDER — AI-based supply chain resilience simulation
OFFIS e.V. – Institut für Informatik, Oldenburg
BMBF-funded
7 changes: 4 additions & 3 deletions src/provider_simenv/agents/farmer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
role="producer" Regional soja producers. All run one region-agnostic step:
output = base_yield * effective("capacity")
price = effective fixed_costs / delivered quantity * (1 + margin)
Producers differ only in their cost/margin parameters and which PDL shocks their archetype binds
(BRA binds fertilizer. Each binds its own entity's supply/capacity shock)
Producers differ only in their cost/margin parameters and declared PDL bindings.
Each binds declared inputs and its own entity's supply/capacity shock.
No producer is hardwired shock-immune or as a fixed surplus supplier.
(supply surges come from the PDL)

Expand Down Expand Up @@ -118,7 +118,8 @@ def _step_producer(self):
Output = base_yield * effective("capacity")
effective("capacity") is this producer's own supply-shock multiplier (1.0 when unbound)
so a drought or a supply event scales output directly. Lower output raises the per unit price.
effective("fertilizer") (1.0 when unbound) raises effective fixed costs this step for producers that bind it (BRA)
effective("fertilizer") (1.0 when unbound) raises effective fixed costs
for producers with that declared input.
"""
capacity = self.effective("capacity")
self.quantity_available = self.base_yield * capacity
Expand Down
Loading