Skip to content

Explicitly specify job queue #147

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 24, 2024
Merged

Conversation

Plytas
Copy link
Contributor

@Plytas Plytas commented May 30, 2024

Issue

Take this batch for example:

Bus::batch([new TestBatchedJob()])->onQueue('batch')->dispatch();

Running php artisan queue:work --queue=batch in a regular Laravel application successfully executes TestBatchedJob job on batch queue, even if the job has set a different queue, such as $this->queue = 'default'.


Using the same batch with this package is currently not possible, unless you specifically set the same queue on the job too. Batch jobs do get dispatched to GCT on correct queue (batch), but it fails the execution because it cannot pass existence check on TaskHandler. That's because it tries to resolve queue name from serialized payload, but it won't find one and default to default queue set in config, unless you specified queue directly on the job.

Solution

I found out that we could always set the queue on the job instance before dispatching it to GCT. Unfortunately it's not possible to to do that for batched closures, because they are transformed into CallQueuedClosure inside \Illuminate\Queue\Queue::createPayload() and then immediately serialized.

Only 1 test failed initially with this change and it was because it was expecting SimpleJob queue to be null, but with this change it is set to default queue. I added couple extra cases to test regular closure dispatch too. . I haven't found any batch tests in this repo, so I'm not sure how to best test batch dispatching to showcase the original issue.

Comment on lines -36 to +42
#[Override]
protected function tearDown(): void
{
parent::tearDown();
#[Override]
protected function tearDown(): void
{
parent::tearDown();

CloudTasksQueue::forgetWorkerOptionsCallback();
}
CloudTasksQueue::forgetWorkerOptionsCallback();
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran composer pint, thus this change.

@@ -82,7 +84,7 @@ protected function setConfigValue($key, $value)
$this->app['config']->set('queue.connections.my-cloudtasks-connection.'.$key, $value);
}

public function dispatch($job): DispatchedJob
public function dispatch($job, ?string $onQueue = null): DispatchedJob
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added an optional parameter that allows to set queue for a closure job.

Comment on lines -159 to +165
&& ($command['queue'] ?? null) === null
&& $command['queue'] === 'barbequeue'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This failed initially after the changes, because $command['queue'] is now always set.

@@ -146,17 +147,22 @@ public function it_posts_the_task_the_correct_queue()
// Arrange
CloudTasksApi::fake();

$closure = fn () => 'closure job';
$closureDisplayName = CallQueuedClosure::create($closure)->displayName();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Display name of a closure job looks like Closure (QueueTest.php:150). Added this line to keep the ability to assert displayName in CloudTasksApi::assertTaskCreated().

@Plytas
Copy link
Contributor Author

Plytas commented Jun 6, 2024

Sorry for pinging you @marickvantuil, but would you be able to look into this at your earliest convenience?

@marickvantuil marickvantuil added the safe-to-test Pull request has access to workflow secrets to run tests label Jun 24, 2024
@marickvantuil
Copy link
Member

Thanks! I'll check quickly if I can make a test for the batched job. Might have to rewrite some stuff in the TestCase.

Apologies for the later reply - I was on holiday :-)

@marickvantuil marickvantuil merged commit d6471a3 into stackkit:master Jun 24, 2024
12 of 13 checks passed
@Plytas
Copy link
Contributor Author

Plytas commented Jun 26, 2024

Thanks for taking your time to look into this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
safe-to-test Pull request has access to workflow secrets to run tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants