This time, I have summarized the solutions for the phenomenon of rollback when running on the console. This time, I will explain using scraping as an example. There is a commentary in the second half.
scraping.rb
class Scraping < ApplicationRecord
def self.get_infomation
require 'mechanize'
agent = Mechanize.new
links = []
current_page = agent.get("https://talent-dictionary.com/s/jobs/3/20")
elements = current_page.at('.home_talent_list_wrapper')
boxs = elements.search('.item')
roks = boxs.search('.right')
qqqs = roks.search('a')
eees = qqqs.search('.title')
eees.each do |eee|
links << eee.inner_text
end
links.each do |link|
get_personal_infomation('https://talent-dictionary.com/' + link)
end
end
def self.get_personal_infomation(link)
agent = Mechanize.new
personal_page = agent.get(link)
aaas = personal_page.at('.talent_name_wrapper')
ages = aaas.at('.age').inner_text.delete('age').to_i if aaas.at('.age')
names = aaas.at('h1').inner_text if aaas.at('h1')
image_urls = personal_page.at('.main_image img').get_attribute('src') if personal_page.at('.main_image img')
infomation = Infomation.where(name: names).first_or_initialize
infomation.age = ages
infomation.image_url = image_urls
infomation.save
byebug
end
end
Run get_infomation in the console.
[1] pry(main)> Scraping.get_infomation
(0.3ms) SET NAMES utf8, @@SESSION.sql_mode = CONCAT(CONCAT(@@sql_mode, ',STRICT_ALL_TABLES'), ',NO_AUTO_VALUE_ON_ZERO'), @@SESSION.sql_auto_is_null = 0, @@SESSION.wait_timeout = 2147483
Infomation Load (0.2ms) SELECT `infomations`.* FROM `infomations` WHERE `infomations`.`name` = 'Hirose Suzu' ORDER BY `infomations`.`id` ASC LIMIT 1
(0.1ms) BEGIN
(0.1ms) ROLLBACK
Return value is: nil
[23, 32] in /home/ec2-user/environment/filebook/app/models/scraping.rb
23: ages = aaas.at('.age').inner_text.delete('age').to_i if aaas.at('.age')
24: names = aaas.at('h1').inner_text if aaas.at('h1')
25: image_urls = personal_page.at('.main_image img').get_attribute('src') if personal_page.at('.main_image img')
26: infomation = Infomation.where(name: names).first_or_initialize
27: infomation.age = ages
28: infomation.image_url = image_urls
29: infomation.save
30: byebug
=> 31: end
32: end
(byebug)
Looking at this result, I don't understand why rollback is done. There is a command that will spit out the reason for this rollback! !! !!
There are two stages. The first street is
aplication.rb
require_relative 'boot'
require 'rails/all'
# Require the gems listed in Gemfile, including any gems
# you've limited to :test, :development, or :production.
Bundler.require(*Rails.groups)
module Filebook
class Application < Rails::Application
# Initialize configuration defaults for originally generated Rails version.
config.load_defaults 5.2
config.logger = Logger.new(STDOUT)← Description of this column
# Settings in config/environments/* take precedence over those specified here.
# Application configuration can go into files in config/initializers
# -- all .rb files in that directory are automatically loaded after loading
# the framework and any gems in your application.
end
end
Second stage
scraping.rb
class Scraping < ApplicationRecord
def self.get_infomation
require 'mechanize'
agent = Mechanize.new
links = []
current_page = agent.get("https://talent-dictionary.com/s/jobs/3/20")
elements = current_page.at('.home_talent_list_wrapper')
boxs = elements.search('.item')
roks = boxs.search('.right')
qqqs = roks.search('a')
eees = qqqs.search('.title')
eees.each do |eee|
links << eee.inner_text
end
links.each do |link|
get_personal_infomation('https://talent-dictionary.com/' + link)
end
end
def self.get_personal_infomation(link)
agent = Mechanize.new
personal_page = agent.get(link)
aaas = personal_page.at('.talent_name_wrapper')
ages = aaas.at('.age').inner_text.delete('age').to_i if aaas.at('.age')
names = aaas.at('h1').inner_text if aaas.at('h1')
image_urls = personal_page.at('.main_image img').get_attribute('src') if personal_page.at('.main_image img')
infomation = Infomation.where(name: names).first_or_initialize
infomation.age = ages
infomation.image_url = image_urls
infomation.save
logger.debug infomation.errors.inspect ← This description
byebug
end
end
If you put these two steps in between
[1] pry(main)> Scraping.get_infomation
D, [2020-11-02T13:39:29.459445 #5812] DEBUG -- : (0.3ms) SET NAMES utf8, @@SESSION.sql_mode = CONCAT(CONCAT(@@sql_mode, ',STRICT_ALL_TABLES'), ',NO_AUTO_VALUE_ON_ZERO'), @@SESSION.sql_auto_is_null = 0, @@SESSION.wait_timeout = 2147483
D, [2020-11-02T13:39:29.464122 #5812] DEBUG -- : Infomation Load (0.2ms) SELECT `infomations`.* FROM `infomations` WHERE `infomations`.`name` = 'Hirose Suzu' ORDER BY `infomations`.`id` ASC LIMIT 1
D, [2020-11-02T13:39:29.470351 #5812] DEBUG -- : (0.1ms) BEGIN
D, [2020-11-02T13:39:29.476168 #5812] DEBUG -- : (0.1ms) ROLLBACK
D, [2020-11-02T13:39:29.476801 #5812] DEBUG -- : #<ActiveModel::Errors:0x0000000004383ef8 @base=#<Infomation id: 14, age: 22, name: "Hirose Suzu", image_url: "https://images.talent-dictionary.com/uploads/image...", created_at: "2020-11-01 07:14:44", updated_at: "2020-11-01 07:14:44">, @messages={:user=>["must exist"]}, @details={:user=>[{:error=>:blank}]}>
Return value is:nil ↑ Description above
[24, 33] in /home/ec2-user/environment/filebook/app/models/scraping.rb
24: names = aaas.at('h1').inner_text if aaas.at('h1')
25: image_urls = personal_page.at('.main_image img').get_attribute('src') if personal_page.at('.main_image img')
26: infomation = Infomation.where(name: names).first_or_initialize
27: infomation.age = ages
28: infomation.image_url = image_urls
29: infomation.save
30: logger.debug infomation.errors.inspect
31: byebug
=> 32: end
33: end
(byebug)
<ActiveModel::Errors:0x0000000004383ef8 @base=#<Infomation id: 14, age: 22, name: "Hirose Suzu", image_url: "https://images.talent-dictionary.com/uploads/image...", created_at: "2020-11-01 07:14:44", updated_at: "2020-11-01 07:14:44">, @messages={:user=>["must exist"]}, @details={:user=>[{:error=>:blank}]}>
I spit out an error like this.
This error translates to {: user => ["must exist"] user is required, but {: user => [{: error =>: blank}] But it can be translated that the user column is empty.
Seems to be an association error.
Latter half
A logger spits out logs (indicating what you did) from controllers, models, views, etc. The logger functionality consists of the Logger class, which inherits from the ActiveSupport class.
・ Config / application.rb ・ Config / evironments / xxxxxxx.rb -In a file that has a class that inherits ActiveSupport You can use it.
You can specify the file to save the log by writing Logger.new ('filename'). (STDOUT) is saved to the'log / development.log' file by default
This is one step. In the second stage, we will write the code in the file to be processed. logger.debug xxxxx.errors.inspect Writing errors.inspect will inspect for errors.
Please try it out.
Recommended Posts