Skip to content

[Messenger] Be able to start a worker for multiple queues with custom consumption priorities #45882

Open
@d-ph

Description

@d-ph

Description

Hello,

The documentation says:

You can (...) instruct one worker to handle messages in a priority order:

> php bin/console messenger:consume async_priority_high async_priority_low

The worker will always first look for messages waiting on async_priority_high. If there are none, then it will consume messages from async_priority_low.

I would like to propose that a way to run the messenger:consume worker is added, where the end developer can define the priorities by which the worker should consume queue tasks (in particular: define that two or more queues have the same priority). Consider the following messenger:consume invocation that shows the proposed feature in practice:

php bin/console messenger:consume queue_a^1 queue_b^1 queue_b_low^2 queue_b_lowest^3 queue_c^1

That invocation would start a worker that consume tasks in the following priority:

  1. (queue_a, queue_b, queue_c)
  2. queue_b_low
  3. queue_b_lowest

It all boils down to the following if-statement in the code: link. The worker should continue to consume tasks from the same "priority group level" before breaking the consumption loop (because there was at least one task consumed from that group).

Example

A few more examples on top of the one mentioned above:

php bin/console messenger:consume queue_a queue_b queue_c

This would be implicitly equivalent to

php bin/console messenger:consume queue_a^1 queue_b^2 queue_c^3
php bin/console messenger:consume queue_a^1 queue_b^1 queue_b_low^2 queue_b_lowest^3 queue_c^1 queue_c_low^2 queue_d^10

This would start the worker with the following "priority groups":

  1. (queue_a, queue_b, queue_c)
  2. (queue_b_low, queue_c_low)
  3. queue_b_lowest
  4. queue_d

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions