Using Laravel's Lottery helper to rate limit error reporting

I'm currently working on a project that is command line only, synchronising tens of thousands of products in one go. While everything worked great in testing, and even a test API account, large scale imports of data rarely go quite according to plan.

I've encountered a problem where a large scale import of many thousands of products is throwing an error on each iteration. Now this has been gracefully handled with a try/catch statement that simply logs an e for error out to the console.

While it's great that we're continuing the process without a catastrophic failure, I'm left with the problem that I'm unable to see exactly what is going wrong behind the scenes. The solution of course is to log this to our trusty error monitoring tool Flare. Great, so lets implement that...

$response->chunkMap(function (Product $product) {
    try {
        // store our product in the database
        $this->persist($product);
    } catch (WriteException $e) {
        // report our error and gracefully output an 'e' for error
        report($e);
        $this->output->write('e');
    }
});

Ok, so we're looping through our many thousands of products, and when we hit an error we'll output a nice message and report it to Flare. But wait, what about when out of the 10,000 products 3,000 are failing. We'll be spamming Flare with our errors in quick succession.

To overcome this challenge, let's implement Laravel's lottery helper. With this helper, we can provide the odds of winning, and when they are a winner, perform an action. So in our case, lets say that for one in every 100 errors, lets report it. Bringing our 3,000 errors down to just thirty.

$response->chunkMap(function (Product $product) {
    try {
        // store our product in the database
        $this->persist($product);
    } catch (WriteException $e) {
        // conditionally report our error and gracefully output an 'e' for error
        Illuminate\Support\Lottery::odds(1, 100)->winner(fn () => report($e))->choose();
        $this->output->write('e');
    }
});

And with that, we're done. Now, only one in every 100 errors will get reported to Flare, all thanks to Laravel and their Lottery helper. They've even got testing covered! Check out the documentation for this - https://laravel.com/docs/11.x/helpers#lottery

There's so many uses for Laravel's Lottery helper and I look forward to seeing what problems it can solve for me in the future.

But try/catch is ugly!

I'm not quite sure why, but I really dislike try/catch statements. A nice tryCatch helper API would be nice, something like this?

tryCatch(fn () => $this->persist($product), [
    WriteException::class => function (WriteException $e) {
        // conditionally report our error and gracefully output an 'e' for error
        Illuminate\Support\Lottery::odds(1, 100)->winner(fn () => report($e))->choose();
        $this->output->write('e');
    }
]);

If you enjoyed this short read, please consider signing up to my mailing list by clicking here so I can let you know of future articles you might be interested in!