[Rails] I tried to implement batch processing with Rake task

Introduction

environment

Rails: 5.2.3 Ruby: 2.6.3 MySQL: 5.6.40

Contents

--There is a bug that the question_count column of the ʻusers` table is not updated. I created a batch and made it possible to update to the correct value in batch. ――Because you are using an existing method, it may be difficult to understand, but please forgive me.

user.rb


has_many :questions, dependent: false
has_many :only_questions_visible_to_user, -> { where(question_id: nil).visible_to_user }, class_name: 'Question', dependent: false

scope :visible_to_user, -> { where(is_invalid: false).where(deleted: false) }

def update_question_count!
  update!(question_count: count_questions)
end

private

# question_Method used for aggregation when updating count
#
# @return [Integer]
def count_questions
  only_questions_visible_to_user.size
end

question.rb


belongs_to :user, optional: true

Implemented code

--I tried to make it easier to recognize the progress and update failure by outputting the log. --Since it is executed only once and will not be needed after that, put it under the ʻone_shot` directory and delete it after execution.

lib/tasks/one_shot/update_users_question_count.rake


# frozen_string_literal: true

namespace :user do
  desc 'Update users_question_count'
  task update_question_count: :environment do |_task|
    index = 0
    updated_question_counts = 0
    user_count = User.count
    logger = Logger.new($stdout)
    logger.info 'Start'

    User.find_each do |user|
      begin
        if user.question_count != user.only_questions_visible_to_user.size
          user.update_question_count!
          updated_question_counts += 1
        end
      rescue ActiveRecord::RecordInvalid => e
        logger.info "Failed to update user_id: #{user.id}"
        logger.info("#{e.class}: #{e.message}")
        next
      end

      index += 1
      case index
      when 4000, 8000, 12_000, 16_000
        logger.info "Progress: #{index}/#{user_count}"
      end
    end
    logger.info "Updated User Count: #{updated_user_counts}"
    logger.info 'Finished'
  end
end

Batch execution

bundle exec rake user:update_question_count

――It took about 45 seconds to update about 19,000 records. --Since the column that failed to update is displayed, it is easier to handle it separately. スクリーンショット 2020-10-26 午後2.36.25.png

Recommended Posts

[Rails] I tried to implement batch processing with Rake task
I tried to implement ModanShogi with Kinx
I tried to implement Ajax processing of like function in Rails
I tried to implement the image preview function with Rails / jQuery
[Rails] Implement rake task
How to make batch processing with Rails + Heroku configuration
I tried to implement file upload with Spring MVC
I tried to implement Stalin sort with Java Collector
I tried to increase the processing speed with spiritual engineering
[Rails] I tried to create a mini app with FullCalendar
I tried to interact with Java
[Rails] I tried to implement "Like function" using rails and js
I tried to implement flexible OR mapping with MyBatis Dynamic SQL
I tried to make a group function (bulletin board) with Rails
I tried to introduce CircleCI 2.0 to Rails app
I tried migrating Processing to VS Code
I tried to get started with WebAssembly
I tried to implement the Iterator pattern
[Rails] I tried to implement a transaction that combines multiple DB processes
[iOS] I tried to make a processing application like Instagram with Swift
Rails: How to write a rake task nicely
I tried to verify AdoptOpenJDK 11 (11.0.2) with Docker image
I tried to make Basic authentication with Java
I tried to implement polymorphic related in Nogizaka.
I tried to manage struts configuration with Coggle
[Rails] I tried to raise the Rails version from 5.0 to 5.2
I tried to manage login information with JMX
I tried writing CRUD with Rails + Vue + devise_token_auth
I want to play with Firestore from Rails
[Rails] How to easily implement numbers with pull-down
Rails beginners tried to get started with RSpec
I tried to implement a server using Netty
I tried to break a block with java (1)
Rails API mode I tried to implement the keyword multiple search function using arrays and iterative processing.
I want to perform aggregation processing with spring-batch
[Rails] I want to load CSS with webpacker
I tried to implement a function equivalent to Felica Lite with HCE-F of Android
I tried Spring Batch
I tried what I wanted to try with Stream softly.
How to implement login request processing (Rails / for beginners)
I tried to read and output CSV with Outsystems
I tried to implement Firebase push notification in Java
[Java 11] I tried to execute Java without compiling with javac
[Rails] I tried playing with the comment send button
I started MySQL 5.7 with docker-compose and tried to connect
I tried to get started with Spring Data JPA
I want to authenticate users to Rails with Devise + OmniAuth
I tried to draw animation with Blazor + canvas API
Rails6 I tried to introduce Docker to an existing application
[Java] I tried to implement Yahoo API product search
I tried to implement the Euclidean algorithm in Java
roman numerals (I tried to simplify it with hash)
A memo that I was addicted to when making batch processing with Spring Boot
[Swift] I tried to implement Instagram profile-like UI with UICollectionView only with code without storyboard
[Swift] I tried to implement exception handling for vending machines
I tried installing Ruby on Rails related plugin with vim-plug
I tried to implement the like function by asynchronous communication
I tried to create a java8 development environment with Chocolatey
I tried to introduce Bootstrap 4 to the Rails 6 app [for beginners]
I tried DI with Ruby
I want to push an app made with Rails 6 to GitHub