Introduce postfix to WSL to relay mail addressed to Office365 and process it with Python

Introduction

I needed to investigate, so I tried the method of relaying the sent mail, processing it, and then forwarding it to Office365.

WSL → telnet → Send mail → postfix → Process with Python → sendmail → It's like Office365.

I am using WSL (Ubuntu 18.04 LTS) of Windows 10 as an environment because it is only a confirmation of the implementation method. E-mail is sent from within WSL (localhost), and postfix settings are prioritized for operation check.

environment

I started from an environment where WSL (Ubuntu 18.04 LTS) can be used on Windows 10 Pro.

Installation of WSL (Windows Subsystem for Linux) and initial settings of Ubuntu

Referenced information

http://www.postfix.org/FILTER_README.html https://qiita.com/kanedaq/items/aab362e20bdafea83fff

Postfix installation and relay settings on WSL (Ubuntu 18.04 LTS)

$ sudo apt install postfix
$ sudo cp /etc/postfix/main.cf.proto /etc/postfix/main.cf
$ sudo vi /etc/postfix/main.cf
#Mail system host name (FQDN)
myhostname = hostname.example.jp
#Mail system domain name
mydomain = example.jp
#Domain name (domain name) that is automatically completed when an email is delivered
myorigin = $mydomain
#Network interface to receive mail (all)
inet_interfaces = all
#Allowed network range
mynetworks = 127.0.0.0/8

#The full path of the Postfix sendmail command
sendmail_path = /usr/sbin/sendmail
#Full path of Postfix's newaliases command
newaliases_path = /usr/bin/newaliases
#Full path of Postfix mailq command
mailq_path = /usr/bin/mailq
#Group for sending email and managing queues
setgid_group = postdrop
#Postfix HTML document location
html_directory = no
#Postfix manual location
manpage_directory = /usr/share/man
#Location of Postfix sample configuration files
sample_directory = /etc/postfix
#Postfix README file location
readme_directory = /usr/share/doc/postfix

#Add from here to the bottom of the configuration file
#Top mail server of the forwarding destination (Office365 this time)
relayhost = [smtp.office365.com]:587
# SMTP-Enable AUTH
smtp_sasl_auth_enable = yes
#A file that describes the address of the transfer destination server and the user name and password to be authenticated.
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
#Do not allow anonymous login
smtp_sasl_security_options = noanonymous
#Filter for SASL mechanism on destination server
smtp_sasl_mechanism_filter = LOGIN
#TLS encryption if possible
smtp_tls_security_level = may
#CA certificate (ca-certificates)
smtp_tls_CApath = /etc/ssl/certs/ca-certificates.crt
$ sudo vi /etc/postfix/sasl_passwd
[smtp.office365.com]:587 Username (email address):password
$ sudo chmod 600 /etc/postfix/sasl_passwd
$ sudo postmap hash:/etc/postfix/sasl_passwd
$ sudo service postfix restart
$ tail -f /var/log/mail.log
$ cat /var/log/mail.err
#Check the process
$ ps aux|grep rsyslog|grep -v grep
#Check settings
$ sudo /usr/sbin/rsyslogd -n -d
$ cat /etc/rsyslog.d/50-default.conf
#Check the status
$ sudo /etc/init.d/rsyslog status
#Start the process
$ sudo /etc/init.d/rsyslog start

error: open database /etc/aliases.db: No such file or directory

#Stop Postfix and rebuild and start the mail alias database
$ sudo service postfix stop
$ sudo newaliases
$ sudo service postfix start
#Confirmation of existence of mail alias database
$ ls /etc/aliases.db

Confirmation to relay the sent mail and forward it to Office365

$ telnet hostname.example.jp 25
Trying 127.0.1.1...
Connected to hostname.example.jp.
Escape character is '^]'.
220 hostname.example.jp ESMTP Postfix (Ubuntu)
(input) HELO hostname.example.jp
250 hostname.example.jp
(input) MAIL From:Sender email address
250 2.1.0 Ok
(input) RCPT To:Destination email address
250 2.1.5 Ok
(input) DATA
354 End data with <CR><LF>.<CR><LF>
(input) This is test mail.
(input) .
250 2.0.0 Ok: queued as EAD291C000000084498
(input) quit
221 2.0.0 Bye
Connection closed by foreign host.
$ tail -f /var/log/mail.log
$ cat /var/log/mail.err

Filter with Python program using Postfix content_filter

$ sudo apt install python3
$ sudo apt install python3-pip
$ python3 -V
$ pip3 -V
$ cd ~
$ mkdir postfix
$ cd postfix
$ touch transfer_mail.py
$ chmod 755 transfer_mail.py
#!/usr/bin/python3

import sys
import os
import email
import email.parser
from os.path import join, dirname
from subprocess import Popen, PIPE

#Get from standard input
mime_str = sys.stdin.read()
# mime_str = open(sys.argv[4]).read()
#Analyze as email
message = email.parser.Parser().parsestr(mime_str)
#A file that spits out the contents of an email
filepath = join(dirname(__file__), 'result.txt')
#Exhale the contents of the email to a file
with open(filepath, mode='w') as f:
    #Parameters
    for arg in sys.argv:
        f.write(f"{arg}\n")
    #Email header
    for key in message.keys():
        value = message.get(key)
        f.write(f"{key} : {value}\n")
    #the content of the email
    payload = message.get_payload()
    f.write(payload)
#Set source, destination, subject
message["From"] = sys.argv[1]
message["To"] = sys.argv[2]
message["Subject"] = "Test mail !!!"
#Forward email
command = ["/usr/sbin/sendmail", "-t", "-oi"]
stdout = ''
stderr = ''
retval = 0
process = Popen(command, stdin=PIPE)
(stdout, stderr) = process.communicate(message.as_string().encode())
retval = process.wait()
if retval == 0:
    sys.exit(0)
else:
    sys.exit(1)
$ sudo vi /etc/postfix/master.cf
# content_Specify filter
smtp      inet  n       -       n       -       -       smtpd -o content_filter=filter:dummy

# content_Define a filter (user to run, program to run, sender, recipient, size)
filter    unix  -       n       n       -       10      pipe
  flags=Rq user=User argv=/home/user/postfix/transfer_mail.py ${sender} ${recipient} ${size}
$ sudo service postfix restart
$ tail -f /var/log/mail.log
$ cat /var/log/mail.err

Confirmation that the sent mail is relayed, processed with Python and forwarded to Office365

$ telnet hostname.example.jp 25
Trying 127.0.1.1...
Connected to hostname.example.jp.
Escape character is '^]'.
220 hostname.example.jp ESMTP Postfix (Ubuntu)
(input) HELO hostname.example.jp
250 hostname.example.jp
(input) MAIL From:Sender email address
250 2.1.0 Ok
(input) RCPT To:Destination email address
250 2.1.5 Ok
(input) DATA
354 End data with <CR><LF>.<CR><LF>
(input) This is test mail.
(input) .
250 2.0.0 Ok: queued as EAD291C000000084498
(input) quit
221 2.0.0 Bye
Connection closed by foreign host.
$ cat ~/postfix/result.txt
/home/user/postfix/transfer_mail.py ← Executed program
Sender email address ← Filter parameter 1
Destination email address ← Filter parameter 2
Size ← Filter parameter 3
Return-Path : <Relay email address>
Received : from hostname.example.jp (localhost [127.0.0.1])
	by hostname.example.jp (Postfix) with SMTP id 923AE9E00000003FBF1
	for <Destination email address>; Fri,  9 Oct 2020 23:55:00 +0900 (JST)
Message-Id : <[email protected]>
Date : Fri, 10 Oct 2020 23:55:00 +0900 (JST)
From :Sender email address
This is test mail.(Text)
$ tail -f /var/log/mail.log
Oct 10 23:55:00 hostname postfix/qmgr[10858]: 923AE9E00000003FBF1: from=<Sender email address>, size=362, nrcpt=1 (queue active)
Oct 10 23:55:00 hostname postfix/pipe[10951]: 923AE9E00000003FBF1: to=<Destination email address>, relay=filter, delay=591, delays=591/0.01/0/0.08, dsn=2.0.0, status=sent (delivered via filter service)
Oct 10 23:55:00 hostname postfix/qmgr[10858]: 923AE9E00000003FBF1: removed
Oct 10 23:55:00 hostname postfix/pickup[10857]: 2047A9F00000003FBF1: uid=1000 from=<user>
Oct 10 23:55:00 hostname postfix/cleanup[10955]: 2047A9F00000003FBF1: message-id=<[email protected]>
Oct 10 23:55:00 hostname postfix/qmgr[10858]: 2047A9F00000003FBF1: from=<[email protected]>, size=567, nrcpt=1 (queue active)
Oct 10 23:55:00 hostname postfix/smtp[10957]: warning: vstream_tweak_tcp: getsockopt TCP_MAXSEG: Protocol not available
Oct 10 23:55:00 hostname postfix/smtp[10957]: 2047A9F00000003FBF1: to=<Destination email address>, relay=smtp.office365.com[XXX.XXX.XXX.XXX]:587, delay=1.6, delays=0.03/0.02/1.1/0.4, dsn=2.0.0, status=sent (250 2.0.0 OK <[email protected]> [Hostname=XXXXXXXXXXXXX.xxxxxxxx.xxxx.outlook.com])
Oct 10 23:55:00 hostname postfix/qmgr[10858]: 2047A9F00000003FBF1: removed

in conclusion

I'm curious about the vstream_tweak_tcp warning in the log, but I was able to relay and process the email I sent and then forward it to Office365. It seems that it can be used for cooperation with other services and surveys during development.

Recommended Posts

Introduce postfix to WSL to relay mail addressed to Office365 and process it with Python
Get mail from Gmail and label it with Python3
I tried to make a periodical process with Selenium and Python
Process the gzip file UNLOADed with Redshift with Python of Lambda, gzip it again and upload it to S3
Fractal to make and play with Python
Read CSV file with Python and convert it to DataFrame as it is
I tried to make a periodical process with CentOS7, Selenium, Python and Chrome
How to get started with the 2020 Python project (windows wsl and mac standardization)
Scraping tabelog with python and outputting to CSV
MessagePack-Try to link Java and Python with RPC
[Python] I introduced Word2Vec and played with it.
[Introduction to system trading] I drew a Stochastic Oscillator with python and played with it ♬
Quickly create a Python data analysis dashboard with Streamlit and deploy it to AWS
It is easy to execute SQL with Python and output the result in Excel
Procedure to load MNIST with python and output to png
Python Ver. To introduce WebPay with a little code.
I want to handle optimization with python and cplex
Introduce pipe operators and function synthesis to Python (provisional)
Install selenium on Mac and try it with python
Try to operate DB with Python and visualize with d3
Read json file with Python, format it, and output json
Email hipchat with postfix, fluentd and python on Azure
Something to enjoy with Prim Pro (X-Play) and Python
Recursively get the Excel list in a specific folder with python and write it to Excel.
I made a server with Python socket and ssl and tried to access it from a browser
Return the image data with Flask of Python and draw it to the canvas element of HTML
[Python / Ruby] Understanding with code How to get data from online and write it to CSV
[Python] What is a tuple? Explains how to use without tuples and how to use it with examples.
Precautions when inputting from CSV with Python and outputting to json to make it an exe
Easy to use Nifty Cloud API with botocore and python
Try to make it using GUI and PyQt in Python
After all it is wrong to cat with python subprocess.
screen and split screen with python and ssh login to remote server
I tried various methods to send Japanese mail with Python
Associate Python Enum with a function and make it Callable
[Python] How to play with class variables with decorator and metaclass
[pyqtgraph] Add region to the graph and link it with the graph region
Send experiment results (text and images) to slack with Python
Try to bring up a subwindow with PyQt5 and Python
How to do Bulk Update with PyMySQL and notes [Python]
How to convert Youtube to mp3 and download it super-safely [Python]
[Let's play with Python] Image processing to monochrome and dots
The process of making Python code object-oriented and improving it
Object-oriented in C: Refactored "○ ✕ game" and ported it to Python
Convert video to black and white with ffmpeg + python + opencv
I tried to make GUI tic-tac-toe with Python and Tkinter
It's Halloween so I'll try to hide it with Python
Install CaboCha in Ubuntu environment and call it with Python.
Get additional data to LDAP with python (Writer and Reader)
How to log in to AtCoder with Python and submit automatically
Overview of Python virtual environment and how to create it
[Python] What is pip? Explain the command list and how to use it with actual examples
Find the white Christmas rate by prefecture with Python and map it to a map of Japan