Use DI container to resolve worker listeners and jobs

[alexvanniel]

Description

The WorkerCommand accepts a ContainerInterface and passes it to the Processor, which suggests that dependency injection is supported and intended for queue workers.

However, the configured worker listener is instantiated directly with new, which means the DI container is bypassed:

$listener = new $listenerClassName();

This makes it impossible to use constructor dependency injection in a custom worker listener.

There is a similar issue when a worker listener needs to instantiate a job dynamically based on the message target. If the listener uses:

$job = new $class();

the job is also instantiated outside the DI container, making constructor injection in jobs impossible.

Environment

• CakePHP: 5.3.7
• cakephp/queue: 2.3.1
• PHP: 8.3

Current behavior

WorkerCommand receives the DI container:

public function __construct(
    protected readonly ?ContainerInterface $container = null,
) {
}

and passes it to the processor:

return new $processorClass($logger, $this->container);

However, when the configured worker listener is created, the container is not used:

$listenerClassName = Configure::read(sprintf('Queue.%s.listener', $config));

/** @var \Cake\Event\EventListenerInterface $listener */
$listener = new $listenerClassName();

if ($processor instanceof Processor) {
    $processor->getEventManager()->on($listener);
}

Consequently, a listener such as:

final class WorkerListener implements EventListenerInterface
{
    public function __construct(
        private SomeService $service,
    ) {
    }
}

cannot be used as the configured worker listener, because it is instantiated without its required dependency.

The same problem occurs for jobs that are dynamically resolved from queue messages.

For example, a custom listener may determine the job class from the message and currently instantiate it with:

$job = new $class();
$job->{$handler}($message);

A job such as:

final class ImportUsersJob
{
    public function __construct(
        private ImportUsersService $importUsersService,
    ) {
    }
}

cannot therefore be instantiated using CakePHP's dependency injection container.

Expected behavior

Since the WorkerCommand already has access to the application DI container, I would expect configured worker listeners to be resolved through that container when one is available.

For example, instead of:

$listener = new $listenerClassName();

something along the lines of:

$listener = $this->container
    ? $this->container->get($listenerClassName)
    : new $listenerClassName();

would allow worker listeners to use constructor injection while preserving the existing behavior when no container is available.

Likewise, there should ideally be a supported way for jobs resolved from queue messages to be obtained from the same DI container, rather than requiring them to be instantiated manually with new.

This would allow applications to use normal CakePHP dependency injection throughout their queue worker infrastructure.

Why this matters

Queue jobs often need application services such as:

• application/domain services
• repositories
• API clients
• import/export services
• other infrastructure services

Requiring these dependencies to be manually constructed or passed through the queue message defeats the purpose of having the application's DI container available to the worker.

It also means that a job cannot simply use normal constructor injection:

final class SomeJob
{
    public function __construct(
        private SomeService $service,
    ) {
    }
}

even though the same class could be resolved normally through CakePHP's container.

Possible implementation

I don't want to prescribe the exact implementation, but it seems that the existing $container available in WorkerCommand could be used to resolve configured listeners.

For example:

$listener = $this->container
    ? $this->container->get($listenerClassName)
    : new $listenerClassName();

For jobs, perhaps the queue infrastructure could expose a job resolver/factory backed by the same container, or otherwise make the container available to the component responsible for resolving job classes.

The important part from an application developer's perspective is that dynamically resolved listeners and jobs should be able to participate in CakePHP's normal dependency injection mechanism.

Additional context

The current behavior is particularly noticeable when implementing a custom worker listener that reacts to the queue lifecycle events:

Processor.message.seen
Processor.message.start
Processor.message.success
Processor.message.failure
Processor.message.invalid
Processor.message.reject
Processor.message.exception

A custom listener can currently be registered through the queue configuration, but because it is instantiated directly, it cannot have constructor dependencies.

This seems somewhat inconsistent with the fact that WorkerCommand already accepts and passes around the application's ContainerInterface.

Would it be possible to resolve configured worker listeners through the DI container, and preferably provide a DI-aware mechanism for resolving jobs as well?