Patch original-stack auxv and restore segment protections - #2
Conversation
greetings from Port Edwards. This supersedes pkgforge-dev#1, keeping the two changes that survived review and dropping the rest. This loader is load-bearing for sharun, so anything that might regress it is worse than the leak it fixes. Kept: - stack: patch the kernel auxv on the original [stack] so tools that read it there, frida-gum in particular, see the loaded binary's AT_BASE, AT_PHDR, AT_PHNUM, AT_PHENT and AT_ENTRY rather than the loader's. The original patch walked from `environ`, which glibc moves to the heap on setenv, so it could read past a heap block and then silently stop patching. This takes the auxv from `start_stack` in /proc/self/stat, the kernel's own record of the initial stack pointer, which setenv does not move. Checked end to end after 64 setenv() calls. - loader: restore each segment's ELF-declared protection with mprotect once relocations are done, instead of leaving every segment writable and executable for the life of the process. Running it after the relocation loop keeps writes into not-yet-protected segments working. The result matches a real execve's maps, including ld.so's RELRO handling. Dropped from pkgforge-dev#1, all of which the review flagged: - close_fds() closed every descriptor instead of only FD_CLOEXEC ones, which breaks inherited descriptors (shell redirections, systemd socket activation). - reset_signals() changes dispositions before the loader has finished its own allocation and file IO, which can break glibc internals. - PR_SET_NAME is a no-op under sharun's hardlinked layout and leaks an internal filename into comm otherwise. - the loader self-unmap drops the executable from /proc/self/maps while /proc/self/exe still names it, breaking the correspondence tools rely on, and its ppc64 trampoline used the wrong syscall number. Built for x86_64, aarch64, riscv64, powerpc64 (both endians) and loongarch64, and exercised against dynamic glibc binaries and after setenv. We aim to provide the software that shapes the world of tomorrow.
|
Correction to my earlier claim: this does not let a sharun-built AppImage drop sharun calls
The patch is correct for callers that pass the program as the executable; there the |
make_stack enumerates the auxv entries the loaded program needs, and the list never included AT_SYSINFO_EHDR. The kernel maps the vDSO in every process and userland-execve leaves it mapped, but without the entry the dynamic linker cannot find it, so glibc falls back to real syscalls for time(), clock_gettime() and gettimeofday(). A seccomp policy that only allows the vDSO path (e.g. Ladybird's) then fails where a normal execve would not. All five supported architectures emit AT_SYSINFO_EHDR from ARCH_DLINFO, including big-endian powerpc64, where VDSO_AUX_ENT is a plain NEW_AUX_ENT. The address is unchanged by userland-execve, so passing getauxval(AT_SYSINFO_EHDR) reproduces execve's own value; it goes through push_usize, so big-endian keeps the native byte order.
This comment was marked as outdated.
This comment was marked as outdated.
|
Reviewed 228a8b0 (both commits) on x86_64, measuring main and this branch against real execve. Verdict: both changes do what they claim, and I could not construct a regression on any path that already works. Numbers below. Setup. Probe program walks the original [stack] auxv the way frida-gum's gum_read_auxv_from_stack does (parse start_stack from /proc/self/stat field 28, hop argc/argv/envp, read the AT_ pairs from there) and compares it against getauxval and dl_iterate_phdr inside the loaded program. Targets: glibc dynamic PIE, glibc static-pie, C++ dynamic (libstdc++), musl dynamic, each with and without 64 setenv() calls in the target. 1. Original-stack auxv patch
2. mprotect segment restore
3. AT_SYSINFO_EHDR carry-over (commit 2)
Regression statement. Every case that runs on main runs on this branch with normalized-identical stdout and exit 0: glibc dyn, C++, musl dyn, each with and without setenv. The one failing case, glibc static-pie, fails identically before and after: SIGSEGV with no output under both main and this branch (rc=139 both) while real execve works. Pre-existing, untouched by this PR, but worth its own issue since it reproduces on main. Non-blocking notes
|
|
Follow-up to the review: ran the same three-way comparison on the other five architectures. Same verdict everywhere: this branch matches real execve, main does not, and nothing that worked on main regressed. Setup. Cross-built the loader from main and from this branch for aarch64, riscv64, powerpc64 (BE), powerpc64le and loongarch64 (bootlin glibc toolchains; zig cc as linker for loongarch64), built the probe per arch, and ran each under qemu-user: qemu-direct as the execve baseline, then main, then this branch. qemu's /proc/self/stat emulation proved faithful on all five (the baseline stack walk works, which is the precondition for the whole original-stack fix). Sysroot glibc is recent (bootlin 2026.08, glibc 2.4x) on every arch. Probe results, per arch, each with and without 64 setenv() calls:
powerpc64 big-endian does not work on either branch, and it is not this PR. Pre-existing bug, identical SIGSEGV on main and this branch (rc=139 both): on ELFv1 (BE), e_entry is a function descriptor [addr, toc, env] holding link-time addresses; the kernel (and qemu) add the load bias to both words and enter at descriptor[0] with r2 = descriptor[1] + bias (see qemu linux-user elfload.c init_thread, which mirrors the kernel). src/run.rs jumps at the raw entry instead, so BE never starts, with or without this PR. I verified the PR's changes on BE by applying a local test-only kernel-mimicking fix to run.rs/exec.rs on top of each branch (not committed, not pushed): with it, this branch measures perfect on BE (rc=0, stack auxv == getauxval == dl_iterate_phdr, AT_SYSINFO_EHDR carried, maps restored to r-xp r--p rw-p), while main still shows the auxv mismatch and rwx text. So the fixes themselves are sound on BE too, but BE needs that entry-point fix as a separate change, same bucket as the pre-existing static-pie segfault I noted on x86_64. Build claim: independently reproduced: the branch cross-builds clean for all six targets (aarch64, riscv64gc, powerpc64, powerpc64le, loongarch64 gnu + x86_64 native). Caveat as before: everything here is under qemu-user emulation, not real hardware. |
|
Correction to my ppc64 BE note above: "BE does not work on either branch" was wrong for real-world AppImages, as the G5 result shows. What I measured was a flavor artifact, now pinned down properly. What the ppc64 BE AppImages actually ship. The Galculator build runs in ghcr.io/pkgforge-dev/archlinux on linux/ppc64. That container's glibc is ELFv2 on big-endian: /usr/lib/ld64.so.2, e_flags 0x2 (abiv2), zero .opd sections, entry point in .text. For ELFv2 the kernel does a raw entry jump with r12 = entry (arch/powerpc/kernel/process.c start_thread, "Look ma, no function descriptors!"), which is byte-for-byte what src/run.rs does. So the shipped configuration is exactly the case run() handles, and the G5 result is consistent. Where my earlier crash came from. I had tested against an ELFv1 glibc (Debian ports / bootlin flavor). There ld64.so.1's e_entry is a .opd function descriptor (verified: entry lands inside .opd, contents [entry_addr, toc, 0]), and for ELFv1 the kernel instead loads nip and r2 FROM the descriptor plus the load bias. The loader's raw jump can't start that flavor, identically on main and this branch. It is a real incompatibility with ELFv1-BE glibc distributions, but not one any of these AppImages exercise, and not something this PR touches. Re-measurement in the real configuration (qemu 10, ArchPPC container rootfs, sharun-style invocation: loader -> ld64.so.2 -> program, both branches as shipped, no local patches):
One honest caveat for the future: if a ppc64 BE ELFv1 glibc (Debian-ports flavor) is ever the target, run() will need the ELFv1 descriptor entry (load nip/toc from e_entry plus load bias, mirroring kernel start_thread) as a pre-existing follow-up. I verified this branch's changes behave correctly on BE under that hypothetical too (locally patched run, full probe pass: rc=0, stack auxv == getauxval == dl_iterate_phdr, AT_SYSINFO_EHDR carried, maps restored). The rest of the cross-arch table (aarch64, riscv64, powerpc64le, loongarch64) stands unchanged: this branch matches the execve baseline everywhere and main does not. |
* Patch original-stack auxv and restore segment protections greetings from Port Edwards. This supersedes #1, keeping the two changes that survived review and dropping the rest. This loader is load-bearing for sharun, so anything that might regress it is worse than the leak it fixes. Kept: - stack: patch the kernel auxv on the original [stack] so tools that read it there, frida-gum in particular, see the loaded binary's AT_BASE, AT_PHDR, AT_PHNUM, AT_PHENT and AT_ENTRY rather than the loader's. The original patch walked from `environ`, which glibc moves to the heap on setenv, so it could read past a heap block and then silently stop patching. This takes the auxv from `start_stack` in /proc/self/stat, the kernel's own record of the initial stack pointer, which setenv does not move. Checked end to end after 64 setenv() calls. - loader: restore each segment's ELF-declared protection with mprotect once relocations are done, instead of leaving every segment writable and executable for the life of the process. Running it after the relocation loop keeps writes into not-yet-protected segments working. The result matches a real execve's maps, including ld.so's RELRO handling. Dropped from #1, all of which the review flagged: - close_fds() closed every descriptor instead of only FD_CLOEXEC ones, which breaks inherited descriptors (shell redirections, systemd socket activation). - reset_signals() changes dispositions before the loader has finished its own allocation and file IO, which can break glibc internals. - PR_SET_NAME is a no-op under sharun's hardlinked layout and leaks an internal filename into comm otherwise. - the loader self-unmap drops the executable from /proc/self/maps while /proc/self/exe still names it, breaking the correspondence tools rely on, and its ppc64 trampoline used the wrong syscall number. Built for x86_64, aarch64, riscv64, powerpc64 (both endians) and loongarch64, and exercised against dynamic glibc binaries and after setenv. We aim to provide the software that shapes the world of tomorrow. * stack: carry the vDSO over as AT_SYSINFO_EHDR make_stack enumerates the auxv entries the loaded program needs, and the list never included AT_SYSINFO_EHDR. The kernel maps the vDSO in every process and userland-execve leaves it mapped, but without the entry the dynamic linker cannot find it, so glibc falls back to real syscalls for time(), clock_gettime() and gettimeofday(). A seccomp policy that only allows the vDSO path (e.g. Ladybird's) then fails where a normal execve would not. All five supported architectures emit AT_SYSINFO_EHDR from ARCH_DLINFO, including big-endian powerpc64, where VDSO_AUX_ENT is a plain NEW_AUX_ENT. The address is unchanged by userland-execve, so passing getauxval(AT_SYSINFO_EHDR) reproduces execve's own value; it goes through push_usize, so big-endian keeps the native byte order. --------- Co-authored-by: Nemo <328747105+Nemo-010@users.noreply.github.com> Co-authored-by: Muhtasham Nawr al-Mahmud <muhtaseem2005@gmail.com>
greetings from Port Edwards.
These are the two changes that survived review; the rest were dropped. This loader is load-bearing for sharun, so anything that might regress it is worse than the leak it fixes.
Kept:
environ, which glibc moves to the heap on setenv, so it could read past a heap block and then silently stop patching. This takes the auxv fromstart_stackin /proc/self/stat, the kernel's own record of the initial stack pointer, which setenv does not move. Checked end to end after 64 setenv() calls.Dropped, all of which the review flagged:
Built for x86_64, aarch64, riscv64, powerpc64 (both endians) and loongarch64, and exercised against dynamic glibc binaries, after setenv, and under a seccomp filter that traps the time syscall.
We aim to provide the software that shapes the world of tomorrow.
Requested by Samueru via errand.
Conversation: https://discord.com/channels/1313385177703256064/1549286930515107880