Proposal: special toolchain names referring to particular override levels

[rami3l]

Part of #4059.

Since #4932 it has been decided that only characters satisfying GeneralSecurityProfile::identifier_allowed() should be allowed in rustup toolchain names.

This has opened up new possibilities for us to assemble our special/reserved toolchain names as needed with special characters such as @ or =.

Following up the above efforts, noting the five levels of rustup toolchain overrides:

1. The +<toolchain> override.
2. The RUSTUP_TOOLCHAIN environment variable.
3. The rustup override set override.
4. The rust-toolchain.toml override.
5. The default toolchain.

... aiming to address #4391, I propose the following changes:

• Make env and default new reserved words in addition to the existing none.
• Introduce the following toolchain syntax:
  • env: Refers to the toolchain that should be activated by the env override.
    • This will not be useful per se but sometime people want to be explicit, e.g. rustup run env cargo instead of cargo.
  • dir=<dir>: Refers to the toolchain that should be activated by the dir override on <dir> or its fallback dirs.
    • e.g. cargo +dir=./crates/download
  • file=<file>: Refers to the toolchain that should be activated by the TOML override in <file> which follows the rust-toolchain.toml schema.
    • e.g. cargo +file=windows.toml, for a Windows-specific rust-toolchain.toml setup.
  • default: Refers to the default toolchain. Bails out if none is set.
    • e.g. cargo +default

Concerns

• How should we handle the error for dir= and file= when the dir/TOML override doesn't exist? Should we bail out or warn and fall back to the next level?
  • My guess is that we should warn and fall back.
• Consider the relation between this proposal and Should settings.toml allow unqualified toolchain names? #4945. Should we resolve the toolchain name when the user issues rustup default file=windows.toml? If so, should we fully resolve or partially resolve it?
  • My guess is that we should partially resolve to an official or a custom toolchain name (that isn't a reserved word and doesn't have special characters like =).
• Can we use this in RUSTUP_TOOLCHAIN (is RUSTUP_TOOLCHAIN not guaranteed to be a parsable toolchain name by downstreams other than rustup)? If we can, we will also resolve Environment variable to disable rust-toolchain.toml #2793 at the same time by setting RUST_TOOLCHAIN=default.
  • I think we probably can.

Note

This also simplifies rust-lang/cargo#10098 because this involves changing the override lookup logic to work with arbitrary base paths. In that case, we'd just need to alter the path for our dir/TOML override source by prefixing it with the path passed via cargo -C.

[ChrisDenton]

I'll think about this some more later but my initial impressions is good. I do like the general idea.

is RUST_TOOLCHAIN not guaranteed to be a parsable toolchain name by downstreams other than rustup

Hm, I do somewhat suspect others do parse this (and the toolchain file). On the other hand we already support things like stable (iirc) so I don't think adding default to the mix really makes things any less rustup specific. However, the more complex syntax (e.g. dir=) might possibly be more of a problem. Then again it's unlikely to match any real-world toolchain if it's not parsed.

[rami3l]

@ChrisDenton In that case we can probably partially resolve and replace it in place until no reserved words or special characters remain, upon the first encounter.

IIRC we already need to convert any toolchain override to a RUSTUP_TOOLCHAIN value anyway.

[djc]

It seems to me like we have a clear use case for file= but I don't think it's as obvious for the other ones? I don't think we should try to overgeneralize/implement a more complex solution than what we have a need for.

[rami3l]

It seems to me like we have a clear use case for file= but I don't think it's as obvious for the other ones? I don't think we should try to overgeneralize/implement a more complex solution than what we have a need for.

@djc Of course I can do this in separate patches.

As for the concrete use case, as previously mentioned:

• Adding the file= level will resolve rust-toolchain.toml: support multiple channels, or rather support nightly AND stable version. #4391
• Adding the default level will resolve Environment variable to disable rust-toolchain.toml #2793

I can remember others being mentioned in previous discussions, but it might be just me. I'll try to post the link here next time I encounter that thread/post.

[ChrisDenton]

I do think default and file= are useful.

The justification for env does seem quite weak though. And it would indeed be useful to find some discussion of where dir= would be useful.

[rami3l]

The justification for env does seem quite weak though.

@ChrisDenton FYI I am reminded of why I have the impression that env is mentioned somewhere:

Our old guidelines of running rustup with cargo run used to be RUSTUP_FORCE_ARG0=rustup rustup run <active-toolchain> cargo run -- (where <active-toolchain> will be env here), it became cargo run --config env.RUSTUP_FORCE_ARG0='rustup' -- only a bit later.

The reason why rustup run <active-toolchain> was used instead of cargo is that the former isn't a proxy mode call so RUSTUP_FORCE_ARG0=rustup is a no-op for this call and this call only.

I do agree that this was a very particular use case for which we now have a pretty good alternative, and I'd be glad to leave this unimplemented unless there are some other real use cases. Same thing for dir= although I'm still searching about the references of that in my memory... I believe it has something to do with workspaces though. 🤔