Sidekiq 5x slower in JumpstartRails than in vanilla Rails app.
My app has a lot of background work that it does where it pulls data from external services and processes them.
It felt really slow because this was running on a MBP M1 Max w/ 64GB machine, so I tested running an empty job with literally no code in it to execute. It was really slow (~70 jobs/sec).
This job was using ActiveJob so I thought the problem may be with overhead of ActiveJob. So I removed the empty ActiveJob job and generated a native Sidekiq Job. Again, empty with no code in it to execute. It was faster (~130 jobs/sec) but still incredibly slower than I would expect.
I then figured I should see if Sidekiq is this slow in a fresh new Rails app. With a native Sidekiq Job in a new Rails app, Sidekiq was running the jobs about 5x faster (~780 jobs/sec).
Anyone know what's going on here in Jumpstart that is causing Sidekiq to be much slower?
Also, even the vanilla Rails app version feels very slow on a monster machine like this. Running more processes doesn't seem to improve total throughput. CPU and memory aren't being maxed out. I'm guessing there is some Redis configuration I'm not optimizing properly. Anyone have any clues on this?
It felt really slow because this was running on a MBP M1 Max w/ 64GB machine, so I tested running an empty job with literally no code in it to execute. It was really slow (~70 jobs/sec).
This job was using ActiveJob so I thought the problem may be with overhead of ActiveJob. So I removed the empty ActiveJob job and generated a native Sidekiq Job. Again, empty with no code in it to execute. It was faster (~130 jobs/sec) but still incredibly slower than I would expect.
I then figured I should see if Sidekiq is this slow in a fresh new Rails app. With a native Sidekiq Job in a new Rails app, Sidekiq was running the jobs about 5x faster (~780 jobs/sec).
Anyone know what's going on here in Jumpstart that is causing Sidekiq to be much slower?
Also, even the vanilla Rails app version feels very slow on a monster machine like this. Running more processes doesn't seem to improve total throughput. CPU and memory aren't being maxed out. I'm guessing there is some Redis configuration I'm not optimizing properly. Anyone have any clues on this?
Notifications
You’re not receiving notifications from this thread.