Rails Solid Queue: Migrating from Sidekiq to the Rails 8 Native Background Job Backend

Last month I helped a SaaS client delete their Redis cluster. Not downgrade it, not resize it — delete it. The only thing it was still doing was Sidekiq. Once we moved their background jobs to Rails Solid Queue, the last reason to keep Redis running quietly evaporated. Their AWS bill dropped by $180 a month, their runbook shrank by a page, and their on-call rotation stopped getting paged about Redis evictions at 3 a.m.

If you followed along with my Solid Cache post, you already know where this is going. Rails 8 shipped Solid Queue as the default ActiveJob backend for the same reason it shipped Solid Cache: most Rails apps do not need Redis. They just need a place to durably put a job description and a worker willing to pick it up. Postgres does that just fine.

After nineteen years of Rails, I have run Delayed Job, Resque, Sidekiq, GoodJob, and now Solid Queue in production. Solid Queue is the first of the group I would reach for by default on a greenfield Rails 8 app — and the first I would migrate an existing Sidekiq shop toward without wincing.

What Rails Solid Queue Actually Is

Rails Solid Queue is a database-backed ActiveJob adapter maintained by 37signals as part of the Rails “Solid Trifecta” (Solid Cache, Solid Queue, Solid Cable). It stores jobs as rows in a set of tables, uses FOR UPDATE SKIP LOCKED to pick up work without lock contention, and runs a supervisor process that spins up pools of workers, a scheduler, and a dispatcher.

Three things make it a serious Sidekiq replacement rather than “database-backed jobs, again”:

• FOR UPDATE SKIP LOCKED — the same Postgres primitive that makes queues like Que fast. Workers do not fight over the same row. One SELECT locks a row and moves on, others skip it.
• Concurrency controls out of the box — you can express “only one instance of this job per user at a time” as one line in the job class. No Redis-based mutex gem, no lua scripts, no expired locks.
• Recurring jobs — cron-style scheduling is built in via config/recurring.yml. You do not need sidekiq-cron, whenever, or a separate scheduler container.

The performance ceiling is lower than Sidekiq. A well-tuned Sidekiq worker can chew through 5,000-10,000 trivial jobs per second per process. A well-tuned Solid Queue worker on Postgres tops out closer to 1,000-2,000. For most Rails apps this is meaningless — you are running dozens or hundreds of jobs per second, not thousands. For the apps where it matters (payment processors, high-fanout notification pipelines), keep Sidekiq. For everyone else, Rails Solid Queue is the right default.

Solid Queue vs Sidekiq: When to Migrate

Before we go further, the honest tradeoff table I share with clients:

Migrate to Solid Queue when Redis is only there for Sidekiq. Keep Sidekiq when you are pushing more than a few thousand jobs per second, when you rely on Sidekiq Enterprise features like rate limiters or batches, or when Redis is already load-bearing for cache and Pub/Sub in ways you cannot unwind.

Setting Up Rails Solid Queue on a New App

Rails 8 generates new apps with Solid Queue as the default queue adapter. config/environments/production.rb contains:

config.active_job.queue_adapter = :solid_queue
config.solid_queue.connects_to = { database: { writing: :queue } }

Solid Queue expects its own database connection. Add it to config/database.yml:

production:
  primary:
    <<: *default
    database: myapp_production
  cache:
    <<: *default
    database: myapp_production_cache
    migrations_paths: db/cache_migrate
  queue:
    <<: *default
    database: myapp_production_queue
    migrations_paths: db/queue_migrate

Install the schema:

bin/rails db:create
bin/rails solid_queue:install:migrations
bin/rails db:migrate

The migrations create seven tables: solid_queue_jobs, solid_queue_ready_executions, solid_queue_claimed_executions, solid_queue_scheduled_executions, solid_queue_failed_executions, solid_queue_pauses, and solid_queue_processes. Do not be intimidated by the count — each one has a single, boring purpose. The one you will query most often is solid_queue_failed_executions when you are debugging.

Running Solid Queue in Production

Solid Queue does not run inside your Puma process by default. You run it as its own long-lived process, supervised the same way you supervise Puma. bin/jobs is generated for you:

bin/jobs start

For production, config/queue.yml defines the worker layout:

production:
  dispatchers:
    - polling_interval: 1
      batch_size: 500
  workers:
    - queues: [critical, default]
      threads: 5
      processes: 2
      polling_interval: 0.1
    - queues: [low, mailers]
      threads: 3
      processes: 1
      polling_interval: 1

I run two worker pools deliberately. The critical, default pool has short polling intervals and multiple processes so paying customers do not wait behind newsletter mailings. The low, mailers pool polls slowly and uses fewer threads — those jobs can wait five seconds without anyone noticing.

Under Kamal, add a role for the workers so you deploy them alongside the web app:

# config/deploy.yml
servers:
  web:
    - 10.0.0.10
  jobs:
    hosts:
      - 10.0.0.10
    cmd: bin/jobs

Kamal will boot a jobs container per host, restart it on deploys, and stream its logs. No separate Sidekiq systemd unit, no supervisord, no PM2.

Concurrency Controls: The Feature That Sold Me

The single feature that made me stop reaching for sidekiq-unique-jobs on new projects is Solid Queue’s built-in limits_concurrency.

class ImportSpreadsheetJob < ApplicationJob
  queue_as :default

  limits_concurrency to: 1, key: ->(user, _file) { user.id }, duration: 30.minutes

  def perform(user, file)
    Importer.new(user).run(file)
  end
end

Only one ImportSpreadsheetJob can run per user.id at any moment. If a second one is enqueued while the first is still working, it blocks (does not run, does not fail) until the first completes or the 30-minute duration expires. No Redis mutex, no zombie locks. The blocked state is stored in a solid_queue_blocked_executions row and released by the same supervisor that handles claims.

I use this constantly for import jobs, PDF generators, and any external API that has a per-account rate limit. It is one of the pieces of Rails Solid Queue that actually beats the Sidekiq ecosystem outright — with Sidekiq you would install a paid or third-party gem and hope its lock semantics matched what you thought they did.

Recurring Jobs Without Cron

config/recurring.yml replaces sidekiq-cron, whenever, or a separate scheduler service:

production:
  cleanup_expired_sessions:
    class: CleanupExpiredSessionsJob
    schedule: every hour
  send_daily_digest:
    class: SendDailyDigestJob
    args: ["marketing"]
    schedule: "0 8 * * *"
  refresh_analytics_cache:
    class: RefreshAnalyticsCacheJob
    schedule: every 15 minutes
    queue: low

The scheduler process (one of the dispatchers in config/queue.yml) reads this file, inserts scheduled executions at the right times, and workers pick them up like any other job. Because the schedule state lives in Postgres, you do not double-fire when you deploy or scale — a fresh scheduler notices “I already scheduled the 8:00 digest job” and moves on.

For anything I would have written a Rake task and a cron entry for in 2019, I now write a job and one YAML block. Rollbacks include the schedule; audit logs include the schedule; the schedule is code-reviewed. This alone justifies the migration for teams that had a crontab drifting between production servers.

Step-by-Step: Migrating from Sidekiq to Rails Solid Queue

Here is the actual playbook I follow when I move a Sidekiq shop to Solid Queue. The goal is a zero-loss cutover with the ability to roll back at any point.

1. Install Solid Queue alongside Sidekiq

# Gemfile
gem "sidekiq"       # keep for now
gem "solid_queue"
gem "mission_control-jobs"  # web UI

Run the installer:

bin/rails solid_queue:install
bin/rails db:migrate

Do not switch the adapter yet. Both backends coexist.

2. Route new jobs class-by-class

ActiveJob allows a per-class queue adapter:

class LowRiskReportJob < ApplicationJob
  self.queue_adapter = :solid_queue
  queue_as :low

  def perform(report_id)
    Report.find(report_id).generate!
  end
end

Start with jobs that are idempotent, low-frequency, and low-risk. Reports, weekly digests, cleanup tasks. If Solid Queue misbehaves for any reason, only these jobs are affected — the rest of your queue is still Sidekiq.

Deploy. Watch Mission Control (/jobs) and your APM. Give it a week.

3. Move recurring jobs to config/recurring.yml

Delete the equivalent entries from your sidekiq-cron config. Move them to config/recurring.yml. This is the highest-value step: it removes an entire class of “did we schedule this correctly?” ambiguity.

4. Flip the default adapter

Once you have a representative sample of jobs running on Solid Queue for a week without incident, flip the global default:

# config/environments/production.rb
config.active_job.queue_adapter = :solid_queue

Any job that still needs Sidekiq (high-throughput, uses a Sidekiq-specific feature) sets self.queue_adapter = :sidekiq explicitly. Everything else moves.

5. Drain and remove Sidekiq

After the flip, Sidekiq still holds jobs that were enqueued before the deploy. Let Sidekiq drain — do not delete the Redis keys. Once Sidekiq::Queue.all.map(&:size).sum is zero and Sidekiq::RetrySet.new.size is zero, remove the gem and the Redis dependency:

# Gemfile
# gem "sidekiq"   # gone

Delete config/sidekiq.yml, the Kamal sidekiq role, and the Redis env var. If Redis was only there for Sidekiq (like my client above), delete the Redis cluster.

Monitoring Rails Solid Queue in Production

Mission Control gives you a web UI that mirrors most of what sidekiq/web gave you — queue depths, failed jobs, retries, pauses, individual job inspection:

# config/routes.rb
authenticate :user, ->(u) { u.admin? } do
  mount MissionControl::Jobs::Engine, at: "/jobs"
end

Beyond the UI, three metrics belong on your dashboard:

# Queue depth per queue
SolidQueue::Job.where(finished_at: nil).group(:queue_name).count

# Failed job count
SolidQueue::FailedExecution.count

# Oldest ready job age (canary for worker starvation)
oldest = SolidQueue::ReadyExecution.minimum(:created_at)
Time.current - oldest if oldest

Ship those to Datadog, Grafana, or whatever you use. The oldest-ready-job age is the single most useful metric — it tells you whether workers are keeping up in a way that queue depth alone does not. A queue with 10,000 jobs and an oldest age of 2 seconds is healthy. A queue with 50 jobs and an oldest age of 4 minutes has a stuck worker.

For alerting, Sentry works well with Solid Queue — configure Rails.error.subscribe and every job failure will end up in Sentry with the arguments, backtrace, and retry count attached.

Common Pitfalls I Have Actually Hit

Not putting the queue database on its own connection pool. Solid Queue can generate a lot of connection churn. If it shares a pool with your web workers, Puma requests will start blocking on the pool. Use connects_to with a dedicated database.

Forgetting to run bin/jobs in development. Rails 8 does not automatically start Solid Queue during bin/dev. Either add a worker line to your Procfile.dev or accept that jobs sit in the queue during local development. I usually add:

worker: bin/jobs

Setting polling_interval too aggressively. A 0.01-second polling interval on eight worker threads is 800 SELECTs per second against your Postgres, doing nothing but asking “any work?”. Start at 0.1 for hot queues and 1.0 for cold ones. The dispatcher wakes workers via LISTEN/NOTIFY when new jobs arrive, so polling is a fallback, not the primary path.

Migrating unique-jobs semantics wrong. limits_concurrency on Solid Queue blocks; sidekiq-unique-jobs typically drops. Read your existing lock configuration carefully before moving — a job that was silently dropped under Sidekiq will now block a worker thread for the full duration if you translate the config naively.

FAQ

Is Rails Solid Queue production-ready?

Yes. 37signals has been running HEY and Basecamp on Solid Queue for over a year. Rails 8 ships it as the default ActiveJob adapter for a reason. For workloads under a few thousand jobs per second, on a decently sized Postgres, it is production-ready today.

Do I need a separate database for Solid Queue?

Not required, but strongly recommended. Solid Queue is write-heavy (every job insert, every claim, every finish is a write). Putting it on your primary application database is fine for small apps but will start showing up in your write IOPS at scale. A separate database on the same Postgres cluster is the sweet spot for most.

How does Rails Solid Queue compare to GoodJob?

Both are Postgres-backed, both use SKIP LOCKED, both are excellent. GoodJob is more mature and has a richer web UI; Solid Queue is the officially-blessed Rails 8 default and has the concurrency-controls-and-recurring-jobs story built in. For a new Rails 8 app I would pick Solid Queue for the alignment with the Rails core team’s direction. For an existing GoodJob deployment, there is no urgent reason to switch.

Can I use Rails Solid Queue with MySQL?

Yes. Solid Queue supports MySQL 8+ and SQLite in addition to Postgres. SKIP LOCKED has been in MySQL since 8.0. Throughput is comparable. On SQLite, it is intended for development and small single-server deployments — combine it with Litestream for durability if you go that route.

Migrating from Sidekiq to Solid Queue, or standing up background jobs on a new Rails 8 app? TTB Software specializes in Rails architecture, upgrades, and production DevOps. We have been doing this for nineteen years.