Create a LINEnews-style tech news summary app with Rails x LineBot! [Part 1]

What I want to do this time

Create a linenews-like app with linebot and Rails

What I made this time

ezgif-6-d27fd089a1f4.gif

When you type in news, a list of news articles will be sent. (A function like delivery at what time in the morning will be implemented next time.)

Why did you decide to make it

It's been less than four months since I started working as an engineer intern, but I wanted to be more sensitive to tech news. And I thought that if I could receive it like linenews, I would get into the habit of reading it, so I decided to make it. The image is like a morning newspaper

This implementation environment

Rails 6.0.3.1
ruby 2.6.1

Use WSL on surface pro 6 Deploy the app itself to heroku

Production flow

1. Register with Line Developpers-Deploy to heroku and check the operation

This time it is not a lecture article on how to make it, so I will omit it. This time, I registered by referring to the following article.

[Rails] Let's easily develop LINE bots in about an hour-Questionnaire total bot basics-[With images]

2. Scraping

I. Introduction

This time, we will scrape the new news from TechCrunch and display the top 10 news items.

A gem called mechanize is used for scraping. For the basic usage of this gem, I referred to "How to use Rails scraping method Mechanize".

Ⅱ. HTML tag analysis

Analyze the tags with Chrome's developer tools to determine which parts to scrape.

As a result, we implemented the method to get the title, image and URL of the news article as follows.


  def get_tech_crunch_news
    title_ary = []
    src_ary = []
    href_ary = []
    agent = Mechanize.new
    page = agent.get("https://jp.techcrunch.com/")
    news_contents = page.search('div.block-content')
    news_contents.search('h2.post-title').each do |title|
      title_ary << title.inner_text
      href_ary << title.at('a')['href']
    end
    news_contents.each do |news_content|
      src_ary << news_content.at('img')['data-src']
    end
    return title_ary, src_ary, href_ary
  end

This time, as a countermeasure against LazyLoad of the img tag, get the data-src attribute instead of the src attribute. This completes the scraping part.

3. Create a line Flex Message

This time, we will use something called flexmessage to deliver lineNews-like messages.

スクリーンショット (109).png ↑ It's such a guy

See the official API for details.

Send Flex Message

This flex message can be easily created using the tools on the WEB provided by line.

FLEX MESSAGE SIMULATOR

スクリーンショット (111).png

You may feel a little stress in the operation, but I think that you can operate it with knowledge that you have some experience with CSS.

When you're done, click the view as json button in the upper right and copy the Json. The JSON this time is as follows.

{
  "type": "bubble",
  "size": "giga",
  "header": {
    "type": "box",
    "layout": "vertical",
    "contents": [
      {
        "type": "text",
        "text": "TechCrunch news",
        "size": "xl",
        "weight": "bold"
      }
    ]
  },
  "body": {
    "type": "box",
    "layout": "vertical",
    "contents": [
      {
        "type": "box",
        "layout": "horizontal",
        "margin": "lg",
        "spacing": "sm",
        "contents": [
          {
            "type": "box",
            "layout": "vertical",
            "contents": [
              {
                "type": "box",
                "layout": "vertical",
                "contents": [
                  {
                    "type": "image",
                    "url": "https://scdn.line-apps.com/n/channel_devcenter/img/fx/01_1_cafe.png ",← Here src
                    "margin": "none",
                    "size": "full",
                    "aspectMode": "cover",
                    "gravity": "top"
                  }
                ],
                "position": "relative",
                "cornerRadius": "12px"
              },
              {
                "type": "box",
                "layout": "vertical",
                "contents": [
                  {
                    "type": "text",
                    "text": "WHILL introduces an automatic driving system to the boarding gate using an electric wheelchair at Haneda Airport",← Title here
                    "size": "md",
                    "weight": "bold",
                    "align": "center",
                    "wrap": true,
                    "maxLines": 4,
                    "color": "#ffffff"
                  }
                ],
                "position": "absolute",
                "width": "100%",
                "backgroundColor": "#00bfff80",
                "offsetBottom": "10px"
              }
            ],
            "action": {
              "type": "uri",
              "label": "action",
              "uri": "http://linecorp.com/"← here href
            },
            "width": "50%",
            "position": "relative"
          },

     ~~~~~~~~~~~~~~~~~~~~~~(Abbreviation)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

              "uri": "http://qiita.com/"
            },
            "width": "50%",
            "position": "relative"
          }
        ]
      }
    ]
  }
}

Then, just put the data acquired by scraping this time in the corresponding place and the news distribution function is completed for the time being.

At the end of the first part

So far, if you type "news" into linebot, you'll be able to pull 10 new articles from TechCrunch from the top. Next time, I would like to be able to deliver new news every morning at 8:00.

Finally, I will put the source of the controller.

linebot_controller.rb


class LinebotController < ApplicationController
  require 'line/bot'

  protect_from_forgery :except => [:callback]

  def client
    @client ||= Line::Bot::Client.new { |config|
      config.channel_secret = ENV["LINE_CHANNEL_SECRET"]
      config.channel_token = ENV["LINE_CHANNEL_TOKEN"]
    }
  end

  def callback
    body = request.body.read

    signature = request.env['HTTP_X_LINE_SIGNATURE']
    unless client.validate_signature(body, signature)
      head :bad_request
    end

    events = client.parse_events_from(body)

    events.each { |event|
      case event
      when Line::Bot::Event::Message
        case event.type
        when Line::Bot::Event::MessageType::Text
          if event.message['text'].eql?('news')
            client.reply_message(event['replyToken'], template)
          end
        end
      end
    }

    head :ok
  end

  private

  def get_tech_crunch_news
    title_ary = []
    src_ary = []
    href_ary = []
    agent = Mechanize.new
    page = agent.get("https://jp.techcrunch.com/")
    news_contents = page.search('div.block-content')
    news_contents.search('h2.post-title').each do |title|
      title_ary << title.inner_text
      href_ary << title.at('a')['href']
    end
    news_contents.each do |news_content|
      src_ary << news_content.at('img')['data-src']
    end
    return title_ary, src_ary, href_ary
  end

  def template
    title, src, href = get_tech_crunch_news
    {
      "type": "flex",
      "altText": "this is a flex message",
      "contents":{
        "type": "bubble",
        "size": "giga",
        "header": {
          "type": "box",
          "layout": "vertical",
          "contents": [
            {
              "type": "text",
              "text": "TechCrunch news",
              "size": "xl",
 
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Abbreviation
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

end

Reference URL

The URL that I referred to during this production is attached below.

[Rails] Let's easily develop LINE bots in about an hour-Questionnaire total bot basics-[With images]

How to use Rails scraping method Mechanize

Recommended Posts

Create a LINEnews-style tech news summary app with Rails x LineBot! [Part 1]
[Rails 5] Create a new app with Rails [Beginner]
Tutorial to create a blog with Rails for beginners Part 1
[Rails] I tried to create a mini app with FullCalendar
Tutorial to create a blog with Rails for beginners Part 2
Tutorial to create a blog with Rails for beginners Part 0
Create a new app in Rails
[Rails withdrawal] Create a simple withdrawal function with rails
Create a simple search app with Spring Boot
[Rails] rails new to create a database with PostgreSQL
Create a team chat with Rails Action Cable
Create an EC site with Rails 5 ⑨ ~ Create a cart function ~
Create a Chat app with WebSocket (Tyrus) + libGDX + Kotlin
How to get started with creating a Rails app
Summary of initial work when creating an app with Rails
Create a Hello World web app with Spring framework + Jetty
A series of steps to create portfolio deliverables with Rails
Until you create a Web application with Servlet / JSP (Part 1)
The story of the first Rails app refactored with a self-made helper
[Rails] I made a simple calendar mini app with customized specifications.
Create a memo app with Tomcat + JSP + Servlet + MySQL using Eclipse
Create a JVM for app distribution with JDK9 modules and jlink
Create a periodical program with Ruby x AWS Lambda x CloudWatch Events
Create a restaurant search app with IBM Watson + Gurunavi API (with source)