Skip to content

[python] Report the dlopen failure reason from load_library - #63

Open
conrade-ctc wants to merge 1 commit into
compiler-research:mainfrom
chicagotrading:pr-f-dlerror-text
Open

conrade-ctc wants to merge 1 commit into
compiler-research:mainfrom
chicagotrading:pr-f-dlerror-text

Conversation

@conrade-ctc

@conrade-ctc conrade-ctc commented Sep 1, 2026 •

Copy link
Copy Markdown
Contributor

Cpp::LoadLibrary drops the loader's failure reason, so load_library raises a bare error. compiler-research/CppInterOp#1101 adds an optional reason out-parameter to Cpp::LoadLibrary. This change passes a std::string to it and reports the text in the RuntimeError. The CppInterOp pin moves to the commit that carries #1101. It adds a regression test for a missing library and a truncated ELF header.

@aaronj0 aaronj0 left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a nice improvement! Perhaps we could have Cpp::LoadLibrary give us the diagnostics on why it failed so we can avoid re-attempting dlopen with ctypes on the Python side.

edit: Just saw compiler-research/CppInterOp#1101, I think that would be the best solution here

@conrade-ctc

Copy link
Copy Markdown
Contributor Author

Agreed, #1101 is the right place for this, and I think a LoadLibrary that hands back the reason is the proper end state. #63 already prefers the captured stderr text and only falls back to the ctypes probe when that text is empty, so with #1101 in the pin the probe is idle. I kept it because the pin lags CppInterOp and the exception is what a notebook user sees, not stderr. Two options, your call: merge #63 as the interim and I remove the probe when the pin catches up, or I close it and follow #1101 with a CppInterOp API that returns the reason (an out-parameter overload of Cpp::LoadLibrary, say) and wire cppjit to that. I am happy either way.

@aaronj0

aaronj0 commented Sep 3, 2026 •

Copy link
Copy Markdown
Collaborator

Agreed, #1101 is the right place for this, and I think a LoadLibrary that hands back the reason is the proper end state. #63 already prefers the captured stderr text and only falls back to the ctypes probe when that text is empty, so with #1101 in the pin the probe is idle. I kept it because the pin lags CppInterOp and the exception is what a notebook user sees, not stderr. Two options, your call: merge #63 as the interim and I remove the probe when the pin catches up, or I close it and follow #1101 with a CppInterOp API that returns the reason (an out-parameter overload of Cpp::LoadLibrary, say) and wire cppjit to that. I am happy either way.

We can add an optional out param to LoadLibrary and once that lands, use that in this PR (you can bump the pinned commit here so it builds with latest CppInterOp containing #1101)

@conrade-ctc

Copy link
Copy Markdown
Contributor Author

Opened compiler-research/CppInterOp#1107 with the optional std::string* error out-parameter, stacked on #1101. The cppjit side is pushed here already: load_library passes a std.string and reports its text, and the ctypes probe is gone. CI on this PR stays red until the pin can point at a CppInterOp commit that carries #1107; I will bump CPPINTEROP_GIT_TAG and squash as soon as it lands.

Comment thread test/test_basic_api.py Outdated
shutil.copyfile(test_dct + ".so", tpath + "/test.so")
cppjit.load_library("test.so")

@mark.skipif(IS_LINUX == 0, reason="checks Linux dlerror text")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we make this xfail and not IS_LINUX

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done: the test now runs everywhere and is an xfail off Linux.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Turns out macOS passes too (strict xfail flagged it as XPASS), so I dropped the marker; the test now runs everywhere.

Cpp::LoadLibrary hands back the loader's reason through an optional
out-parameter (compiler-research/CppInterOp#1101). load_library puts it
in the RuntimeError, so the ctypes re-dlopen probe goes away. The
CppInterOp pin moves to the commit that carries it.

Co-developed-with-the-help-of: Claude Code (Fable 5.1, human in the loop)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants