The title may be suspicious and difficult to understand, but in simple terms, it is an implementation of a function that displays the progress of batch processing performed on the server side in real time in the form of a progress bar. As for the method, it seems that there are various methods when I try to google, but among them, there was no implementation method like me this time, so I will post it for the time being.
--Browser: ** Parameter input ** (Language: php) --Server: -** Start management script ** (Language: python) --Launch batch processing script and monitoring script --Pass the monitoring result to the progress bar -** Batch script ** (Language: shell) --Batch processing is performed according to the parameters from the browser. -** Monitoring script ** (Language: shell) --Monitor the log file generated by batch processing in real time and output the progress status -** Progress bar generation and display ** (Language: php, javascript) --Get the result of the monitoring script in real time and display it on the progress bar. We will still notify the administrator of the processing result by email.
This time I needed to enter the ISBN of the product, so enter the ISBN number and send it to the server in the form of post.
convert.php
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Convert WAV File!</title>
</head>
<body>
<form action="result.php" method="post">
<p><label>ISBN (Numeric Only) : <input type="number" name="isbn"></label></p>
<p><input type="submit" value="Submit"></p>
</form>
</body>
</html>
On the server side, it is necessary to perform batch processing according to the parameters obtained from the browser side, and at the same time, run a monitoring script to grasp the situation at the same time. Therefore, two scripts, batch processing and monitoring, will be started in parallel.
Note: When the batch processing is finished, in order to end the monitoring script, the batch processing status is notified by passing the ID of the batch processing process to the monitoring script.
process.py
#! /usr/bin/env python3
# -*- coding: utf-8 -*-
import os, sys, subprocess, shlex
from multiprocessing import Pool, Process, Queue
def f(logPID, q, scriptName, isbn):
q.put([os.getpid()])
os.system("sh "+scriptName+".sh "+logPID + " " + isbn)
def main():
jobs = []
q = Queue()
isbn = sys.argv[1]
#Batch processing part
mainJob = Process(target=f, args=("",q,"test", isbn, ))
jobs.append(mainJob)
mainJob.start()
logPID = str(q.get()[0])
#Monitoring part
monitorJob = Process(target=f, args=(logPID,q,"monitor", "", ))
jobs.append(monitorJob)
monitorJob.start()
for job in jobs:
job.join()
main()
Originally, the purpose of this program is to perform batch processing based on ISBN information, but this time, as a test case, we will replace it with a script that outputs the processing progress as a numerical value at regular time intervals.
test.sh
#!/bin/sh
isbn=$1
touch log.txt
for i in 1 2 3 4 5 6 7 8 9 10; do
sleep 2
echo $i >> log.txt
done
The monitoring script uses the tail command to retrieve and analyze the lines added to the log file in real time. This time, since the progress status is directly output as a number in batch processing, no processing is done, but in reality, since various information is mixed in the log file as well as the progress status, only the progress status is used in the monitoring script. Processing to extract is required.
Note: By adding the batch processing process ID to the tail command, it is possible to terminate the monitoring script when the batch processing is completed.
monitor.sh
#!/bin/sh
pID=$1 #Batch processing process ID
tail -f -n 1 --pid=$pID log.txt | while read line
do
if [ -n "$line" ] ; then
echo $line
fi
done
The monitoring result is acquired in real time and displayed on the progress bar. Finally, we will notify the administrator by e-mail that it has been processed.
Note: Use the `` `popen``` command to get the python processing result in real time in php.
result.php
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html lang="en">
<head>
<title>Progress Bar</title>
</head>
<body>
<!-- Progress bar holder -->
<div id="progress" style="width:500px;border:1px solid #ccc;"></div>
<!-- Progress information -->
<div id="information" style="width"></div>
<?php
$isbn = $_POST['isbn'];
echo "ISBN: ".$isbn;
// Total processes
$total = 10;
// Loop through process
$cmd = "python /var/www/html/service/progressbar/process.py ".$isbn;
$proc = popen($cmd, 'r');
//rea one line of the last oparation, and do nothing
$i = (int) fread($proc, 4096);
while (!feof($proc)) {
// Calculate the percentation
$i = (int) fread($proc, 4096);
$percent = intval($i/$total * 100)."%";
// Javascript for updating the progress bar and information
echo '<script language="javascript">
document.getElementById("progress").innerHTML="<div style=\"width:'.$percent.';background-color:#ddd;\"> </div>";
document.getElementById("information").innerHTML="'.$percent.' processed.";
</script>';
// Send output to browser immediately
flush();
// Sleep one second so we can see the delay
sleep(1);
if($percent=="100%")
break;
}
// Tell user that the process is completed
echo '<script language="javascript">
document.getElementById("information").innerHTML="Process completed!";
</script>';
//send mail when the process is over
$to = "[email protected]";
$subject = "Batch processed";
$message = "Admin,\n Item number: 「".$isbn."It has been processed.\n Thank you.";
$headers = 'From: [email protected]' . "\r\n";
if(mail($to, $subject, $message, $headers))
echo "Notification email has been sent!";
else
echo "The notification email could not be sent.\n Please inform Admin directly of the item number."
?>
</body>
</html>
The source code of this program can also be obtained from the link below. Implementation of progress bar