Hi,
We're getting the QueueError "no system resources" error meaning the mqueue does not exist. We implemented a 12FA retry loop, and ended up running out of filehandles. Checking via /proc/PID/fd on Linux shows these filehandles are to blame:
lrwx------ 1 root root 64 Aug 4 23:47 994 -> /BarToFoo
lrwx------ 1 root root 64 Aug 4 23:47 995 -> /FooToBar
lrwx------ 1 root root 64 Aug 4 23:47 996 -> /BarToFoo
lrwx------ 1 root root 64 Aug 4 23:47 997 -> /FooToBar
lrwx------ 1 root root 64 Aug 4 23:47 998 -> /BarToFoo
lrwx------ 1 root root 64 Aug 4 23:47 999 -> /FooToBar
Between retries, we're not saving objects with the handles in them anywhere that would prevent the gc from releasing them. They're being created via posixmq.Queue("/FooToBar", serializer=...) and saved only as a private attribute of a class, ~2-4 functions calls deep from the exception handler that implements the retry loop. I wouldn't say I'm 100% certain but I know from having maintained a similar module that leaks are 99% of the maintenance of a dynamic language to C library binding module. Platform is linux/arm64hf.
Thanks for writing ipcqueue!
Hi,
We're getting the QueueError "no system resources" error meaning the mqueue does not exist. We implemented a 12FA retry loop, and ended up running out of filehandles. Checking via
/proc/PID/fdon Linux shows these filehandles are to blame:Between retries, we're not saving objects with the handles in them anywhere that would prevent the gc from releasing them. They're being created via
posixmq.Queue("/FooToBar", serializer=...)and saved only as a private attribute of a class, ~2-4 functions calls deep from the exception handler that implements the retry loop. I wouldn't say I'm 100% certain but I know from having maintained a similar module that leaks are 99% of the maintenance of a dynamic language to C library binding module. Platform is linux/arm64hf.Thanks for writing
ipcqueue!