Follow-up on #664
With a very limited allowlist, the assistant may try to search for external information several times, trying to find relevant urls it is allowed to open. For example, with just OpenFn documentation in the allowlist, and the user question "What is the latest format for tracked entity attributes according to the DHIS2 API? I need the exact field names", the assistant is likely to spend all of its allowed tool uses trying to find the information within OpenFn urls.
The url list will require ongoing updates, but to test and optimise assistant behaviour around navigating allowed/banned urls, to e.g. avoid stalling at search attempts, we need to add more urls in the list.
We need to curate an allowlist of URLs, and eventually decide whether users should control it in the future. We need to:
- Add urls: Ask services team for top priorities + ask Claude for suggestions based on our full adaptor list.
- The prompt pins FHIR to R4 currently. Without that, the model picks R5. This presumably depends on the adaptor, and so this prompt should probably be removed.
Things to bear in mind:
- We are allowed_domains for safety reasons, and will stick with that for as long as anyone makes a feature request that allows other domains.
- The allowed_domains parameter costs no context tokens at all, confirmed by measurement, so the list can be long for free. But the prompt currently recites the whole list into the system prompt at roughly eight tokens per domain. In the future we'll probably have a separate research sub-agent that handles searches/fetches so I wouldn't worry too much about the token cost now.
- Note also that a bare domain covers its subdomains, so hl7.org also grants foundry.hl7.org and terminology.hl7.org.
Follow-up on #664
With a very limited allowlist, the assistant may try to search for external information several times, trying to find relevant urls it is allowed to open. For example, with just OpenFn documentation in the allowlist, and the user question "What is the latest format for tracked entity attributes according to the DHIS2 API? I need the exact field names", the assistant is likely to spend all of its allowed tool uses trying to find the information within OpenFn urls.
The url list will require ongoing updates, but to test and optimise assistant behaviour around navigating allowed/banned urls, to e.g. avoid stalling at search attempts, we need to add more urls in the list.
We need to curate an allowlist of URLs, and eventually decide whether users should control it in the future. We need to:
Things to bear in mind: