[Rails] I tried to implement batch processing with Rake task
Introduction
environment
Rails: 5.2.3 Ruby: 2.6.3 MySQL: 5.6.40
Contents
--There is a bug that the question_count column of the ʻusers` table is not updated.
I created a batch and made it possible to update to the correct value in batch.
――Because you are using an existing method, it may be difficult to understand, but please forgive me.
user.rb
has_many :questions, dependent: false
has_many :only_questions_visible_to_user, -> { where(question_id: nil).visible_to_user }, class_name: 'Question', dependent: false
scope :visible_to_user, -> { where(is_invalid: false).where(deleted: false) }
def update_question_count!
update!(question_count: count_questions)
end
private
# question_Method used for aggregation when updating count
#
# @return [Integer]
def count_questions
only_questions_visible_to_user.size
end
question.rb
belongs_to :user, optional: true
Implemented code
--I tried to make it easier to recognize the progress and update failure by outputting the log. --Since it is executed only once and will not be needed after that, put it under the ʻone_shot` directory and delete it after execution.
lib/tasks/one_shot/update_users_question_count.rake
# frozen_string_literal: true
namespace :user do
desc 'Update users_question_count'
task update_question_count: :environment do |_task|
index = 0
updated_question_counts = 0
user_count = User.count
logger = Logger.new($stdout)
logger.info 'Start'
User.find_each do |user|
begin
if user.question_count != user.only_questions_visible_to_user.size
user.update_question_count!
updated_question_counts += 1
end
rescue ActiveRecord::RecordInvalid => e
logger.info "Failed to update user_id: #{user.id}"
logger.info("#{e.class}: #{e.message}")
next
end
index += 1
case index
when 4000, 8000, 12_000, 16_000
logger.info "Progress: #{index}/#{user_count}"
end
end
logger.info "Updated User Count: #{updated_user_counts}"
logger.info 'Finished'
end
end
Batch execution
bundle exec rake user:update_question_count
――It took about 45 seconds to update about 19,000 records.
--Since the column that failed to update is displayed, it is easier to handle it separately.
Recommended Posts