Fix process state leaks and harden memory protections

[mnalmahmud]

Fix process state leaks, harden memory protections, and self-unmap the loader

This PR addresses several issues where userland-execve leaves behind process state that a real kernel execve would clean up. These leaks cause real-world crashes in applications that inspect their own process (e.g. Frida, Chromium's sandbox, debuggers) and leave unnecessary attack surface.

Changes

1. Enforce W^X memory protections (loader.rs)

What: After mapping each PT_LOAD segment with PROT_WRITE (needed to zero .bss padding), we now call mprotect to restore the segment's intended permissions (e.g. read-only for .rodata, read+exec for .text).

Why: The original code left every mapped segment writable and executable for the entire lifetime of the process. This is a W^X violation — any code injection vulnerability in the loaded application could trivially overwrite executable code in-place.

Edge case: The mprotect pass runs after the relocation loop, not inside the segment loop. Relocations (needed for static-PIE musl binaries) may write into read-only segments. If we locked permissions before relocations, the relocation writes would SEGFAULT.

2. Patch the original kernel auxv (stack.rs)

What: At the end of make_stack, we walk backward from libc::environ to find the original auxiliary vector on the kernel-provided [stack] and overwrite AT_BASE, AT_PHDR, AT_PHNUM, AT_PHENT, and AT_ENTRY with the loaded binary's values.

Why: userland-execve builds a new stack with correct auxv and pivots to it. But the original kernel stack remains mapped and is listed as [stack] in /proc/self/maps. Some tools — notably frida-gum — don't use getauxval() or the new stack. Instead, they parse /proc/self/maps to find the [stack] region, then read the auxv directly from that memory. Since the original auxv still describes the loader (where AT_BASE is 0 because the loader has no interpreter), Frida sees AT_BASE=0, concludes there is no dynamic linker, fails to resolve module paths, and crashes with a null assertion.

Edge case: environ can theoretically be NULL (though practically never is on Linux). We guard against that. The patching also uses as usize casts instead of .try_into().unwrap() since AT_* constants are c_ulong which is always the same width as usize on the architectures this crate supports.

3. Close leaked file descriptors (exec.rs)

What: Before jumping to the loaded binary, we iterate /proc/self/fd and close every fd > 2 (preserving stdin/stdout/stderr).

Why: A real execve automatically closes all file descriptors marked O_CLOEXEC. Since no real execve happens here, any fds opened by the Rust runtime, the loader, or the caller leak into the target application. This can cause resource exhaustion or, in security-sensitive applications, leak handles to files the target shouldn't have access to.

Edge case: We collect the fd list into a Vec before closing, so the directory fd used by read_dir itself doesn't interfere with iteration. The ReadDir handle's drop will attempt to close an already-closed fd, which harmlessly returns EBADF.

4. Reset signal handlers and mask (exec.rs)

What: We reset all signal handlers (1–63) to SIG_DFL and clear the signal mask via sigprocmask(SIG_SETMASK, empty).

Why: A real execve resets all caught signal handlers to their defaults and unblocks all signals. If the caller or the Rust runtime installed any custom handlers (e.g. for SIGSEGV backtraces) or masked any signals, the loaded application inherits that dirty state. This can cause the application to silently ignore signals it expects to receive, or fail to generate core dumps on crashes.

Edge case: SIGKILL and SIGSTOP cannot have their handlers changed; sigaction returns an error for these, which we intentionally ignore.

5. Set process name (exec.rs)

What: We call prctl(PR_SET_NAME) with the loaded binary's filename (truncated to 15 chars per kernel limit).

Why: A real execve updates /proc/self/comm to the new binary's name. Without this, the process shows up as userland-execve (or sharun) in ps, top, htop, and system monitors, which is confusing for users and breaks process management scripts that match by name.

6. Self-unmap the loader from memory (run.rs)

What: Before jumping to the entry point, we parse /proc/self/maps to find all file-backed memory regions belonging to the loader binary, allocate a single isolated executable page, copy a position-independent assembly trampoline to it, mprotect it to r-x (removing the temporary write permission), and jump to the trampoline. The trampoline loops through the regions issuing raw munmap syscalls, then pivots the stack pointer and jumps to the loaded binary's entry point.

Why: A real execve completely replaces the address space. Without self-unmapping, the loader's .text, .data, and .rodata segments remain permanently mapped in the target process. This wastes memory and, more importantly, pollutes /proc/self/maps — tools that walk the memory map (debuggers, profilers, crash reporters) will see unexpected regions belonging to a binary that isn't supposed to be running.

Why a trampoline: You cannot munmap code you're currently executing — the CPU would immediately page-fault on the next instruction fetch. The trampoline lives on a separate page that is not part of the loader, so it can safely unmap everything underneath the loader and then jump away.

Architecture support: Trampolines are provided for all five architectures the crate supports:

Arch	munmap syscall nr	Calling convention
x86_64	11	rax=nr, rdi=addr, rsi=len
aarch64	215	x8=nr, x0=addr, x1=len
riscv64	215	a7=nr, a0=addr, a1=len
loongarch64	215	a7=nr, a0=addr, a1=len
powerpc64	73	r0=nr, r3=addr, r4=len

Edge case: The trampoline page itself (4KB) remains mapped after the jump — this is unavoidable since the CPU is executing from it at the moment of the final jmp/br/jr. Anonymous heap allocations from the Rust runtime also remain, since we cannot distinguish them from the target binary's own anonymous mappings in /proc/self/maps. Both are benign.

Testing

Built a C test payload that verifies all six fixes from inside the loaded process:

--- RUNNING PAYLOAD TESTS ---
[PASS] Memory protections intact (no rwxp)
[PASS] Self-unmapping successful (loader not in memory)
[PASS] No leaked file descriptors
[PASS] Signal handlers properly reset
[PASS] Process name (comm) correctly set
[PASS] AT_BASE correctly set in auxv
-----------------------------
ALL TESTS PASSED!

[mnalmahmud]

keep whatever you deem necessary and drop others.
the auxv patching is the main purpose of the PR and others just fell into my view so fixed them as well.

The details about the PR were generated with the help of AI.

[Samueru-sama]

thanks @mnalmahmud

will review later as I just noticed this by accident, I didn't get the PR email because I wasn't watching the repo

[talaria0101]

Thanks for the thorough write-up. I read the full patch, applied it to a scratch tree, cross-built it for every supported architecture, and ran the loader against real binaries to check each claim. The mprotect work is a genuine improvement and the reasoning about relocation ordering is exactly right.

That said, I found four confirmed correctness bugs, and two of the changes conflict with the reason this fork exists. Details and reproductions below.

Context that shapes this review

This fork is used by Anylinux-sharun. sharun runs dynamic binaries through a bundled dynamic linker. The reason it uses userland-execve instead of execve-ing ld-linux.so directly is that execve-ing the linker makes /proc/self/exe point at the linker, which breaks many applications. Because userland-execve performs no execve, /proc/self/exe keeps pointing at the original process binary.

sharun is deployed hardlinked under each application's name, so /proc/self/exe and /proc/self/comm already carry the application's name for free. Changes 5 and 6 in this PR both touch that machinery, which is why they get their own section below.

What I verified works

Claim	Result
W^X on loaded segments	Confirmed. Baseline maps /usr/bin/cat text as rwxp; patched maps it r-xp.
mprotect-after-relocations ordering	Confirmed. A static-pie musl target still runs correctly.
Self-unmap (mechanically)	Confirmed. No loader-backed regions remain in the target's maps. See the caveat in item 6.
Cross-assembly	cargo build --lib succeeds for aarch64, powerpc64 and riscv64. The x86_64 trampoline is 50 bytes and position independent.

---

Blocking: correctness

1. powerpc64 munmap syscall number is wrong (src/run.rs:134)

li 0, 73
sc

On powerpc, __NR_munmap is 91. Syscall 73 is sigpending, verified against powerpc-linux-any/asm/unistd_64.h:

#define __NR_sigpending 73
#define __NR_mmap       90
#define __NR_munmap     91

The trampoline will call sigpending(addr, len) on each region, writing the pending signal set into the loader's read-only text (EFAULT) and unmapping nothing. The arch table in the PR description repeats the same number, so this path was never executed. The other four are correct (x86_64 = 11, asm-generic = 215).

2. close_fds() closes every descriptor, not the CLOEXEC ones (src/exec.rs:20-33)

The PR describes this as O_CLOEXEC emulation, but it never checks FD_CLOEXEC. A real execve preserves non-CLOEXEC descriptors. Measured with 3</etc/hostname 4</etc/hostname:

# real execve
3 -> /etc/hostname
4 -> /etc/hostname

# via patched loader
(fds 3 and 4 gone)

This breaks shell redirections, systemd socket activation (LISTEN_FDS), and any caller deliberately passing descriptors. Fix: fcntl(fd, F_GETFD) and close only when FD_CLOEXEC is set.

3. auxv patch walks off a heap allocation once anything calls setenv (src/stack.rs:207-240)

The walk starts at environ and steps past its NULL terminator to reach auxv. That is only valid while environ still points at the kernel-supplied stack array. glibc's setenv/putenv reallocates environ onto the heap. I instrumented the walk:

# clean process
environ=0x7fffe54f3268  auxv_scan_start=0x7fffe54f35c8  -> patches AT_PHDR/PHNUM/PHENT/BASE/ENTRY

# after 64 setenv() calls
environ=0x55fb70144a50  auxv_scan_start=0x55fb70144fb0  -> heap; patches nothing

Two problems. The frida fix silently stops working, and the loop performs out-of-bounds reads past a heap block, writing 8 bytes at any 16-byte-aligned slot whose first word happens to be 3, 4, 5, 7 or 9, until it finds a zero. That is arbitrary heap corruption in a library any consumer may call after touching the environment.

Capture the original stack environ at startup, or locate auxv via env_end from /proc/self/stat.

4. String::truncate(15) panics on a non-UTF-8 boundary (src/exec.rs:67)

$ cp /bin/echo /tmp/aaaaaaaaaaaaaaéééééé-bin
$ ./userland-execve /tmp/aaaaaaaaaaaaaaéééééé-bin hi
panicked at src/exec.rs:67:18: assertion failed: self.is_char_boundary(new_len)

Any basename with a multi-byte character crossing byte 15 kills the process. Truncate the raw bytes from OsStr::as_bytes() instead. That also fixes to_str() on line 65 silently skipping non-UTF-8 paths.

---

Blocking: conflicts with this fork's purpose

5. PR_SET_NAME should be dropped (src/exec.rs:64-80)

Real execve does set comm from the basename of the executed file rather than from argv[0]:

$ bash -c 'exec -a totally-fake-name /bin/sleep 2'
argv[0]=totally-fake-name, file=/bin/sleep  ->  comm=sleep

So the prctl matches kernel semantics in the abstract. But it is the wrong target here. Because sharun is hardlinked under the application's name, comm is already correct before this call. Measured against a simulated sharun layout:

Normal deployment, hardlink name matches the real binary name:

	baseline	patched
/proc/self/comm	mycoolapp	mycoolapp

A pure no-op.

Real binary named differently, for example shared/bin/.mycoolapp-real:

	baseline	patched
/proc/self/comm	mycoolapp	.mycoolapp-real

This is a regression, and specifically against the motivation given in the PR description. An internal implementation filename now leaks into ps, htop and pkill in place of the name the user invoked. Any layout with a versioned or dot-prefixed real binary hits this.

No upside for the primary consumer, two downsides, and it is the source of the panic in item 4. I would remove it rather than gate it.

6. Self-unmap breaks the /proc/self/exe to /proc/self/maps correspondence (src/run.rs:155-218)

Running a probe binary that reports its own identity, through a loader hardlinked as bin/mycoolapp:

real execve (control)   exe=/tmp/probe               mappings matching exe = 5
baseline loader         exe=/tmp/sh2/b/mycoolapp     mappings matching exe = 4
patched loader          exe=/tmp/sh2/bin/mycoolapp   mappings matching exe = 0   <-- none

To be precise about what does and does not survive:

• /proc/self/exe still resolves. mm->exe_file is a file reference set at execve time and is not cleared by munmap, so the readlink still returns the hardlink path. $ORIGIN expansion, which glibc derives from /proc/self/exe, is unaffected. The core trick this fork exists for survives.
• What breaks is the correspondence. After the self-unmap, /proc/self/exe names a file with zero regions in /proc/self/maps. Real execve never produces that state. The common pattern of "readlink /proc/self/exe, then locate that path in /proc/self/maps to get the main module's load base" now returns nothing. That pattern is used by crash reporters, profilers and module enumerators, which is the same class of tooling this PR aims to help.

There is a tension worth calling out: change 2 adds the auxv patch so that frida-gum can find things by walking /proc/self/maps, and change 6 then removes the executable from /proc/self/maps. I have not run frida against this, so treat that as something to test rather than a confirmed break, but the two changes pull against each other for the same consumer.

I am not claiming the baseline is correct either. It reports 4 matching mappings, but those are the loader's own text rather than the application's, so a consumer doing that lookup gets a wrong-but-present answer today and an empty answer after this patch. The reviewable point is that this PR changes that invariant by default, with no opt-out, and without validation against the bundled applications that motivated the fork.

Please put this behind an ExecOptions flag that defaults to off.

---

Should fix

7. The trampoline leaves the rtld_fini register dirty

On x86_64 the ABI specifies that %rdx at process entry is the rtld_fini pointer and must be NULL. The trampoline copies rdx into r14 but never clears it, so rdx is now deterministically the new stack pointer, a guaranteed-nonzero garbage function pointer. glibc's _start forwards it to __libc_start_main, which registers it with atexit.

This is pre-existing rather than introduced here, but the new trampoline is the natural place to fix it and instead makes the bad value deterministic. Confirmed by adding xor edx, edx at label 2:, using a static-pie glibc target:

# before
atexit-ran
Segmentation fault (exit=139)

# after
atexit-ran
static-pie main ok argc=1  (exit=0)

The same applies to x0 on aarch64 and a0 on riscv64/loongarch64. Those happen to hold 0 from the last munmap return, but hold the regions pointer when the region list is empty.

8. Loses the rax = 0 guarantee from commit 0ddcb88

The old x86_64 path had inout("rax") 0 => _ specifically to fix a segfault. The trampoline only ends with rax == 0 incidentally, via the last munmap return value. On the jz 2f path (empty region list) rax is whatever run() left behind. Add an explicit xor eax, eax.

9. Trampoline page is mapped RWX (src/run.rs:194)

PROT_READ | PROT_WRITE | PROT_EXEC, then immediately mprotected down to R|X. In a PR about W^X hardening, map it RW, copy, then mprotect to R|X. Costs nothing and avoids SELinux execmem and hardened-allocator denials.

Related: I tested the loader under prctl(PR_SET_MDWE, PR_MDWE_REFUSE_EXEC_GAIN) and it fails at loader.rs:117 with EACCES both before and after this PR, because segments are still mapped prot | PROT_WRITE with PROT_EXEC set. The hardening claim does not extend to MDWE.

10. for sig in 1..64 misses signal 64 (src/exec.rs:44)

SIGRTMAX is 64 on Linux. Use 1..=64.

11. Unmangled global symbols exported from a library crate

$ nm libuserland_execve.rlib | grep tramp
0000000000000032 T tramp_end
0000000000000000 T tramp_start

userland-execve is a library. Two T symbols named tramp_start/tramp_end in the global namespace will collide with any downstream object using those names. Mark them .hidden and prefix them, or use sym operands with local labels.

12. No bounds checks on the trampoline page layout (src/run.rs:202-206)

The trampoline is copied to offset 0 and the regions array to offset 2048, with no assertion that tramp_size <= 2048 or that regions.len() * 16 <= 2048. Both hold today (50 bytes, roughly 5 regions) but corrupt silently if either grows. Two assert!s would cover it.

13. Hardcoded 4096 (src/run.rs:193,210)

The rest of the crate uses sysconf(PAGE_SIZE). It is safe here because the kernel rounds up, but aarch64 with 64K pages makes the constant meaningless. Inconsistent with loader.rs:76.

14. line.ends_with(exe_str) is a fragile match (src/run.rs:162)

Matching the whole maps line by suffix can match an unrelated file whose path ends with the exe path. I checked the deleted-binary case and it does work, since readlink /proc/self/exe also returns the " (deleted)" suffix, so that concern does not apply. Splitting off the pathname field and comparing exactly is still clearer.

15. Gate the remaining behaviour changes behind ExecOptions

ExecOptions already exists as the extension point. Closing descriptors, resetting signal dispositions and unmapping the loader are policy decisions a consumer might want individually. As written, exec() silently changes semantics for every existing caller.

Ordering note: close_fds() and reset_signals() run at exec.rs:82-83, but run() afterwards does file IO and allocation. Resetting SIGCANCEL (32) and SIGSETXID (33) to SIG_DFL breaks glibc internals for the remainder of loader execution. Move these as late as possible.

---

Minor

• loader.rs:67: the // TODO: read only fix comment was removed, but the base reservation still stays PROT_READ|PROT_WRITE and inter-segment gaps remain writable. Nothing was fixed there.
• The PR description has the ReadDir drop order backwards. It claims the dirfd is closed after the loop and returns EBADF harmlessly. The iterator chain is consumed by collect(), so ReadDir drops at the semicolon on exec.rs:26, before the loop. Harmless either way, but the stated reasoning is inverted.
• std::primitive::usize (exec.rs:72) is an odd spelling; plain usize works.
• Segment page sharing. mprotect at loader.rs:141 rounds size up to a page. If two PT_LOAD segments share a page (older linkers, no -z separate-code), the later segment's protection wins because the loop follows ascending vaddr order. Probably fine, but worth an explicit comment.

---

Verdict

Request changes.

Four confirmed defects: a wrong syscall number that makes the ppc64 trampoline a no-op, execve semantics broken for inherited descriptors, out-of-bounds heap access in the auxv walk, and a reproducible panic on non-ASCII binary names. Separately, two changes work against what this fork is for: PR_SET_NAME is a no-op at best and leaks internal filenames at worst, and the self-unmap removes the executable from /proc/self/maps while /proc/self/exe still names it.

[mnalmahmud]

@Samueru-sama hey! sorry had a very busy schedule last 2 weeks, what i missed?

[Samueru-sama]

@Samueru-sama hey! sorry had a very busy schedule last 2 weeks, what i missed?

Apparently there is no easy fix for this problem.

Some of the improvements were integrated into #2

But fixing it completly so that Frida doesn't need to use the preload hack is no simple fix and has a high chance of regressing existing apps.

[mnalmahmud]

@Samueru-sama hey! sorry had a very busy schedule last 2 weeks, what i missed?

Apparently there is no easy fix for this problem.

Some of the improvements were integrated into #2

But fixing it completly so that Frida doesn't need to use the preload hack is no simple fix and has a high chance of regressing existing apps.

okay, i checked, but for some reason the commit wasn't attached with my account.

[Samueru-sama]

okay, i checked, but for some reason the commit wasn't attached with my account.

fixed