I compared the calculation time of the moving average written in Python

Previous article Implementing LWMA of MetaTrader with FIR filter function of scipy So, I tried to rewrite LWMA (Linear Weighted Moving Average) with the lfilter () function, but I'm not thankful for it just being neat, so I compared the calculation time.

I thought that there would be not much difference as expected, so I tried to add a little more time series data to take the moving average. I read the 1-minute data for one year of EUR / USD by referring to the following (my blog article).

Read FX historical data with Python

import numpy as np
import pandas as pd

dataM1 = pd.read_csv('DAT_ASCII_EURUSD_M1_2015.csv', sep=';',
                     names=('Time','Open','High','Low','Close', ''),
                     index_col='Time', parse_dates=True)

There are about 370,000 each of the opening price, high price, low price, and closing price.

Comparison of LWMA

I tried using the % time command in Jupyter Notebook to measure the time. First, the older version of LWMA. Calculate the output one by one.

def LWMA(s, ma_period):
    y = pd.Series(0.0, index=s.index)
    for i in range(len(y)):
        if i<ma_period-1: y[i] = 'NaN'
        else:
            y[i] = 0
            for j in range(ma_period):
                y[i] += s[i-j]*(ma_period-j)
            y[i] /= ma_period*(ma_period+1)/2
    return y

%time MA = LWMA(dataM1['Close'], 10)

As a result

Wall time: 3min 35s

What a three and a half minutes. Even if there is a lot of data, it takes 370,000 pieces so much, is it not possible to use Python? .. ..

Take a second look and test a new version of LWMA. Using scipy's filter function.

from scipy.signal import lfilter
def LWMAnew(s, ma_period):
    h = np.arange(ma_period, 0, -1)*2/ma_period/(ma_period+1)
    y = lfilter(h, 1, s)
    y[:ma_period-1] = 'NaN'
    return pd.Series(y, index=s.index)

%time MA = LWMAnew(dataM1['Close'], 10)

As a result

Wall time: 6 ms

What! 6 milliseconds. Is it 35,000 times faster? Is the original too late? You can use scipy.

By the way, if you run it with % timeit instead of% time,

100 loops, best of 3: 3.4 ms per loop

Completed within 1 second even if executed 100 times. It seems that it can be executed in about 3 milliseconds when it is fast.

SMA comparison

Since scipy is fast, I compared it with the pandas function. SMA can be written as a method function of pandas as follows.

def SMA(s, ma_period):
    return s.rolling(ma_period).mean()

%timeit MA = SMA(dataM1['Close'], 10)

As a result

100 loops, best of 3: 16 ms per loop

Run 100 times and it's fast 16ms. It's pretty fast. Now, let's write this using scipy's filter function.

def SMAnew(s, ma_period):
    h = np.ones(ma_period)/ma_period
    y = lfilter(h, 1, s)
    y[:ma_period-1] = 'NaN'
    return pd.Series(y, index=s.index)

%timeit MA = SMAnew(dataM1['Close'], 10)

As a result

100 loops, best of 3: 3.44 ms per loop

fast. About 5 times as much as pandas. After all scipy is fast, so it may be better to rewrite other moving averages using lfilter ().