[PYTHON] The story of developing a web application that automatically generates catchphrases [MeCab]

Service site

MakeLike https://melikeke.sakura.ne.jp/malike/intro

Overview

Use Python's Mecab to generate "like sentences" from various teacher data ・ Generated from category ・ Generated from a file ・ Generated from direct input Teacher data can be specified from the three patterns of.

function

Automatic catch phrase generation

・ Generated from category ・ Generated from a file ・ Generated from direct input

Login function

Member registration Login Member registration notification reset Password

Favorite function

Implementation

environment

Sakura server

front end

Laravel

Basic part

Utilization of templates

Make the layout part common in Laravel's Blade template http://cly7796.net/wp/php/to-a-common-layout-in-blade-template-of-laravel/


Login authentication system

How to implement a quick login function in Laravel https://php-junkie.net/framework/laravel/login/

When "Command" make: auth "is not defined." Appears https://qiita.com/daisu_yamazaki/items/a914a16ca1640334d7a5

[User authentication with Laravel7_8] Authenticate email when changing email address https://qiita.com/crosawassant/items/018b29ab770c0a373bc9

Let's examine the behavior of Laravel's standard Authentication (Auth) https://qiita.com/zaburo/items/9fcf0f4c771e011a4d35


Automatic notification when registering as a member

Email notifications and Slack notifications are possible by either of the following methods

Laravel If you read this, you can set Laravel events and listeners https://reffect.co.jp/laravel/laravel-event-listener#i-8

Google Analytics Receive alerts from Google Analytics https://www.ad-market.jp/column/2020/04/20200420.html


Mail function

Set up Laravel's email feature to send password reset emails.

"MailTrap" during local development In the production environment, we set the SMTP of "Sakura Server" and confirmed the communication.

E-mail transmission test with Laravel mail Trap Memorandum https://qiita.com/ryomaDsakamoto/items/e9d3a2c258dbfc66c524

Set the SMTP of Sakura server to the SMTP server of Laravel https://laraweb.net/tutorial/1265/

Try using the sophisticated "Laravel" email sending function https://liginc.co.jp/369690

Laravel5.7: Send Japanese password reset email https://qiita.com/sutara79/items/0ea48847f5565aacceea


Japanese setting

When setting Japanese in Laravel, You can adapt the settings by writing in ja.json.

/resources/lang/ja.json


{
    "Click link below and reset password.": "Please access the URL below and reset your password.",
    "If you did not request a password reset, no further action is required.": "If you don't know this email, just delete it.",
}

blade.php


<p>
    {{ __('Click link below and reset password.') }}<br>
    {{ __('If you did not request a password reset, no further action is required.') }}
</p>

Actual display image.png


Generated part

Implementation of "Generate from category" display part

The category pull-down makes the choices of "Category 2" change in association with "Category 1".


・ When category 1 is "lyrics"


・ When category 1 is "novel"


relation

category1.id = category2.parent_id


categori1


categori2


reference

How to make a parent-child select box like category / subcategory in laravel5.1 https://www.messiahworks.com/archives/12202


Implementation of "Generate from category" processing part

Set the path of teacher data to be read by Storage based on the value specified in the category.

contoloer.php


$input = $request->input();
$category1  = Arr::get($input, 'category1');
$category2  = Arr::get($input, 'category2');
$path = 'public/constText/';
$path .= $category1 . '/';
$path .= $category2 . '.txt';

Structure of constText image.png


Implementation of "Generate from file" part

Use Laravel's Storage feature to implement the upload process. This time, the extension is set as follows to limit it to text data only. ·TXT ・ Csv ・ Xls ・ Xlsx

reference Detailed explanation on how to upload files with Laravel https://reffect.co.jp/laravel/how_to_upload_file_in_laravel

I tried to summarize the operation of files and directories using Storage in Laravel https://qiita.com/t1k2a/items/50081988363cf2fa1bca


Implementation of "Generate from direct input" part

"Generate from file" is executed in the above flow, but "Generate from direct input" is also executed by utilizing the Storage function.

Generation flow from direct input Save the value of textarea as a text file in Storage Below, the same process as "Generate from file"

Server end

Python The automatically generated part by MeCab is summarized in the following article.

reference Automatically generated catch phrase [Python] https://qiita.com/SyogoSuganoya/items/ba542f686104811e2d6b


Python execution from PHP

In a firing event such as input button implemented in Laravel Allows you to implement Python.

Use PHP's exec function to execute Python.

reference PHP and Python integration from scratch on Laravel https://qiita.com/SwitchBlade/items/96ed4ea425ef2d758f71

PHP --Summary of the relationship between exec () error handling and standard error output https://qiita.com/smd8122/items/65b552f1d53bfb7fad9a


Ajax

Basic

Creating a dynamic comment viewing screen using Laravel and Ajax https://www.merges.co.jp/archives/1980

Use Ajax (jQuery) with Laravel https://pointsandlines.jp/server-side/php/laravel-ajax

Asynchronous processing by passing values with Laravel and Ajax. https://niwacan.com/1619-laravel-ajax/


Display of generated text

image.png


Ajax is used to display the "generated text" area after pressing the generate button. Display the PHP exec function "\ $ outputs" in the generated text. $ outputs contains the printed string while running Python

controller.php


exec($command , $outputs, $return_var);

$data = [
	'outputs' => $outputs,
	'return_var' => $return_var,
];
return response()->json($data);

pythonCall.js


var params = {
	headers: {  
		'X-CSRF-TOKEN': $('meta[name="csrf-token"]').attr('content')
	},
	url: url,
	method: 'post',
	dataType: 'json',
	data: paramData,
	//Runs until the request is complete
	beforeSend: function(){
		$('h3').text('Loading');
		$("input").attr("disabled","true"); 
		$('.wrapper .button').addClass('pushed');
		$('.heart-check').prop('checked', false); //Remove all checked items
	}
};

$.ajax(
	params
).done(function( data ) {
	//Initialization of generated text
	$('input.sentence').val('');
	
	//Input of generated sentences
	for(var i = 0; i < data.outputs.length && i < 5; i++){
		$('input.sentence').eq(i).val(data.outputs[i]);
	}
}

Favorite function

Favorite registration of generated sentences and Manage your favorite texts on My Page image.png


The favorite function uses Ajax and processes asynchronously.

controller.php


//Display of list screen
public function index(Request $request) {

	$posts = Favorite_sentence::where('user_id', $request->user()->id) //$Get posts by user
		->orderBy('created_at', 'desc') //Arrange posts in chronological order
		->paginate(10); //Pagination; 

	return view('user.mypage', [
		'posts' => $posts, // $Pass the article written by user to view
	]);
}

//Click on the heart symbol(Register or delete)
public function ajaxlike(Request $request) {

	// $user_id = $request->user()->id;
	$user_id = Auth::user()->id;
	$post_id = $request->post_id;
	$sentence = $request->sentence;

	$like = new Favorite_sentence;
	$exist = Favorite_sentence::where('id', $post_id)->get();
	$isExist = $exist->isEmpty();

	if ($isExist) {
		//Create a new record in the likes table if empty (not yet liked)
		$like = new Favorite_sentence;
		$like->user_id = $user_id;
		$like->sentence = $sentence;
		$like->save();

		$post_id = $like->id;
		$command = 'insert';

	} else {
		//Confirmation of the existence of your favorite phrase
		Favorite_sentence::findOrFail($post_id);
		//Delete records in likes table
		$like = Favorite_sentence::where('id', $post_id)
		->delete();

		$command = 'delete';
	}

	$data = [
		'user_id' => $user_id,
		'post_id' => $post_id,
		'sentence' => $sentence,
		'command' => $command,
	];
	//Return the argument value to ajax with the following description
	return response()->json($data);
}

likeSentence.js


var like = $('.heart-label');
var likePostId;
var sentence;

like.on('click', function () {

	var $this = $(this);
	//Favorite sentence ID
	likePostId = $this.parent().parent().find(".sentence").attr('data-postid');
	//Favorite sentences
	sentence = $this.parent().parent().find(".sentence").val();
	
	//Favorite text is blank
	if (!sentence) {
		return;
	}
	
	$.ajax({
		headers: {
			'X-CSRF-TOKEN': $('meta[name="csrf-token"]').attr('content')
		},
		url: '/ajaxlike',
		type: 'POST',
		data: {
			'post_id': likePostId ,
			'sentence': sentence , 
		},
	}).done(function (data) {            
		if(data.command = 'insert') {
			$this.parent().parent().find(".sentence").attr('data-postid', data.post_id);
		}
});

release

Use "Sakura Server" in production environment

Python / MeCab environment construction

Output the result of morphological analysis with Mecab to a WEB browser compatible with Sakura server / UTF-8 https://qiita.com/Jshirius/items/ac3ca66a2d5262b98b58

Install mecab on Sakura shared server and call it from python https://qiita.com/Jshirius/items/3a067486e2a693544c32


When MeCab can be run from a terminal but not from a PHP call ...

First try calling Python with the full path

A story that I had a hard time trying to call python with the exec function from php on the X server https://hazukei.com/1259/


Add site-packages

If that doesn't work, compare "site-packages" between terminal execution and PHP execution.

import site
site.USER_SITE

Add diffs to make installed packages executable

import sys
sys.path.append('vendor\Lib\site-packages')

reference Install Python module on Sakura rental server https://emptypage.jp/notes/sakura-python.html

An active engineer explains how to check and set the installation destination of the Python module [for beginners] https://techacademy.jp/magazine/46510

Laravel [0001] SSH into Sakura's rental server and install Laravel https://www.failibere.com/development_memo/lalavel/0001

Make a symbolic link to storage on the Sakura server https://blog.hiroyuki90.com/articles/laravelで作成したwebアプリをレンタルサーバに公開する/

Recommended Posts

The story of developing a web application that automatically generates catchphrases [MeCab]
The story of making a web application that records extensive reading with Django
A story about creating a web application that automatically generates Minecraft sound block performances
A story that reduces the effort of operation / maintenance
A story that analyzed the delivery of Nico Nama.
A tool that automatically turns the gacha of a social game
The story of remounting the application server
The story of writing a program
The story of the release work of the application that Google does not tell
The story of creating a site that lists the release dates of books
The story of making a module that skips mail with python
The story of adding MeCab to ubuntu 16.04
The story of blackjack A processing (python)
I made a calendar that automatically updates the distribution schedule of Vtuber
The story of making a package that speeds up the operation of Juman (Juman ++) & KNP
[Python] I made a web scraping code that automatically acquires the news title and URL of Nikkei Inc.
The story of IPv6 address that I want to keep at a minimum
The story of making a box that interconnects Pepper's AL Memory and MQTT
The story of making a lie news generator
I want to create a web application that uses League of Legends data ①
The story of making a Line Bot that tells us the schedule of competitive programming
The story of making a mel icon generator
A python script that generates a sample dataset for checking the operation of a classification tree
The story of introducing a multi-factor authentication function using a one-time password into a Java application
The story of Linux that I want to teach myself half a year ago
I made a calendar that automatically updates the distribution schedule of Vtuber (Google Calendar edition)
The story of launching a Minecraft server from Discord
[Python] A program that counts the number of valleys
# Function that returns the character code of a string
The story of making a music generation neural network
A story that struggled with the common set HTTP_PROXY = ~
Generate that shape of the bottom of a PET bottle
A story about changing the master name of BlueZ
The story that the return value of tape.gradient () was None
Zip 4 Gbyte problem is a story of the past
[Python] A program that compares the positions of kangaroos.
Clustering G-means that automatically determines the number of clusters
The story of sys.path.append ()
Create a bot that only returns the result of morphological analysis with MeCab on Discord
Create a web application that recognizes numbers with a neural network
The story of creating a VIP channel for in-house chatwork
The story of a Django model field disappearing from a class
The story of creating a database using the Google Analytics API
The story of making a question box bot with discord.py
A Python script that compares the contents of two directories
A story that verified whether the number of coronas is really increasing rapidly among young people
Note that I was addicted to accessing the DB with Python's mysql.connector using a web application.
Output the result of morphological analysis with Mecab to a WEB browser compatible with Sakura server / UTF-8
A story that is a little addicted to the authority of the directory specified by expdp (for beginners)
The story of making a tool that runs on Mac and Windows at the game development site
The story of creating a bot that displays active members in a specific channel of slack with python
A memo that reproduces the slide show (gadget) of Windows 7 on Windows 10.
When incrementing the value of a key that does not exist
A story stuck with the installation of the machine learning library JAX
The story that the version of python 3.7.7 was not adapted to Heroku
pandas Fetch the name of a column that contains a specific character
[python, ruby] fetch the contents of a web page with selenium-webdriver
The story that a hash error came out when using Pipenv
A formula that simply calculates the age from the date of birth
The story of making a standard driver for db with python.
A function that measures the processing time of a method in python