Mémo de base Python - Partie 2

Table of content

  1. Environment

    • 1.1. Command line
    • 1.2. pip
    • 1.3. pyenv
    • 1.4. virtualenv
    • 1.5. iPython Notebook
  2. Module

  3. File access

  4. Database access

  5. Plotting with matplotlib

  6. Environment

1.1. Command line

$ python myscript.py
$ python

1.2. pip

$ wget https://bootstrap.pypa.io/get-pip.py
$ python get-pip.py

1.3. pyenv

1.4. virtualenv

1.5. iPython Notebook

Install iPython Notebook with pip

$ pip install jupyter

Start iPython Notebook

$ jupyter notebook

Using iPython Notebook

  1. Module

Basic on module

def mysum(a, b):
    return a + b
import mymod
print(mymod.mysum(3, 4))
> python myscript.py
7
import mymod as mm
print(mm.mysum(3, 4))
# mm works as alias to mymod
from mymod import mysum
print(mysum(3, 4))

Import patterns

# Import a module and access the module's elements through it name
import modulename
modulename.func1(params)
# Use alias to a module
import modulename as alias
alias.func1(params)
# Import specific names from a module
from modulename import func1, func2
func1(params)
func2(params)
# Import specific names and use alias
from modulename import func1 as f
f(params)
# Import all names from a module except those beginning with an underscore (_)
from modulename import *
func1(params)
func2(params)

Global variable __name__

Module can be executed directly

def mysum(a, b):
    return a + b
if __name__ == "__main__":
    print("Mymod is executed directly")

Module search path

import sys
print(sys.path)
  1. File access

Reading from file

Read all data from file

f = open('stock1.csv', 'r')
data = f.read()
f.close()
print(data)

sec_code,opn,high,low,now,time 6702,602,610,599,608,2016-01-04 00:00:00 6702,608,611,608,610,2016-01-04 00:10:00 6758,2958,3000,2940,2998,2016-01-04 00:00:00 6758,2998,3011,2992,3009,2016-01-04 00:10:00

Read one line

f = open('stock1.csv', 'r')
data = f.readline()
f.close()
print(data)

sec_code,opn,high,low,now,time

Read all lines

f = open('stock1.csv', 'r')
for line in f:
    print(line)
f.close()

sec_code,opn,high,low,now,time

6702,602,610,599,608,2016-01-04 00:00:00

6702,608,611,608,610,2016-01-04 00:10:00

6758,2958,3000,2940,2998,2016-01-04 00:00:00

6758,2998,3011,2992,3009,2016-01-04 00:10:00

Write to file

f = open('sample_out.csv', 'wb')
f.write('Programming with Python \n')
f.write('Write 2 numbers: %d %f' % (5, 7.95))
f.close()

Reading from CSV

import csv
linecount = 0
with open('stock1.csv', 'rb') as csvfile:
    stockreader = csv.reader(csvfile, delimiter=',')
    for row in stockreader:
        print(row)
        linecount += 1

print("%d lines" % linecount)

['sec_code', 'opn', 'high', 'low', 'now', 'time '] ['6702', '602', '610', '599', '608', '2016-01-04 00:00:00'] ['6702', '608', '611', '608', '610', '2016-01-04 00:10:00'] ['6758', '2958', '3000', '2940', '2998', '2016-01-04 00:00:00'] ['6758', '2998', '3011', '2992', '3009', '2016-01-04 00:10:00'] 5 lines

Writing to CSV

import csv
with open('sample_out.csv', 'wb') as csvfile:
    csvwriter = csv.writer(csvfile, delimiter=',', lineterminator="\n")
    csvwriter.writerow(['Product', 'Price', 'Amount'])
    csvwriter.writerow(['Table', 12000, 5])
    csvwriter.writerow(['Chair', 4700, 25])

Serialization with pickle

Writing data with pickle

import pickle
f = open('sample_out.pkl', 'wb')
mixed = ['abc', 1, 5, 7.39]
str = "String data in pickle"
pickle.dump(mixed, f)
pickle.dump(str, f)
f.close()

Reading data with pickle

f = open('sample_out.pkl', 'rb')
new_mixed = pickle.load(f)
new_str = pickle.load(f)
f.close()
print(new_mixed)
print(new_str)

['abc', 1, 5, 7.39] String data in pickle

  1. Database access with mysql-connector-python

Install mysql-connector-python package Install manually

Install through pip

$ echo https://cdn.mysql.com/Downloads/Connector-Python/mysql-connector-python-2.1.3.tar.gz >> requirements.txt
$ pip install -r requirements.txt
$ pip list | grep mysql

Connector/Python API Reference

Connect to DB

import mysql.connector
from mysql.connector import errorcode

config = {
    'user': 'us',
    'password': 'pwd',
    'host': 'localhost',
    'database': 'stocks'
}

try:
    mysql_cnn = mysql.connector.connect(**config)
except mysql.connector.Error as err:
    if err.errno == errorcode.ER_ACCESS_DENIED_ERROR:
        print("Something is wrong with your user name or password")
    elif err.errno == errorcode.ER_BAD_DB_ERROR:
        print("Database does not exist")
    else:
        print(err)

SELECT data

cursor = mysql_cnn.cursor(buffered=True)
sql = 'SELECT * FROM stocks_10minute WHERE sec_code = 9984'
cursor.execute(sql)
print('Row count: %d' % cursor.rowcount)

Row count: 608

rec = cursor.fetchone()
print(rec)

(26911001, 9984, 6050, 6081, 6030, 6046, datetime.datetime(2016, 1, 4, 0, 0))

records = [rec for rec in cursor]
price = [rec[5] for rec in records] # now column
time = [rec[6] for rec in records] # time column
cursor.close()

True

UPDATE data

upd_cursor = mysql_cnn.cursor()
id = 29269249
value = 5266
upd_sql = "UPDATE stocks_10minute SET now = %d WHERE id = %d" % (value, id)
upd_cursor.execute(upd_sql)
mysql_cnn.commit()
upd_cursor.close()

True

INSERT data

import datetime
ins_cursor = mysql_cnn.cursor()
ins_sql = (
  "INSERT INTO stocks_10minute (sec_code, open, high, low, now, time) "
  "VALUES (%s, %s, %s, %s, %s, %s)"
)
data = (9984, 5245, 5288, 5243, 5265, datetime.datetime(2016, 2, 1, 0, 10))
ins_cursor.execute(ins_sql, data)

new_id = ins_cursor.lastrowid
new_id

29278575

mysql_cnn.commit()
ins_cursor.close()

True

DELETE data

del_cursor = mysql_cnn.cursor()
del_sql = (
  "DELETE FROM stocks_10minute WHERE id = %d " % new_id
)
del_cursor.execute(del_sql)
mysql_cnn.commit()
del_cursor.close()

True

Disconnect

mysql_cnn.close()
  1. Plotting with matplotlib

Introduction

Install

$ pip install matplotlib

Basic

import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
plt.plot([2, 3, 4, 5])
plt.ylabel('some numbers')
plt.show()

output_82_0.png

plt.plot([1, 2, 3, 4], [1, 4, 9, 16])
plt.xlabel('Some numbers')
plt.ylabel('Their squares')
plt.show()

output_84_0.png

plt.plot([1, 2, 3, 4], [1, 4, 9, 16], 'r^--')
plt.xlabel('Some numbers')
plt.ylabel('Their squares')
plt.axis([0, 6, 0, 20])
plt.show()

output_86_0.png

# Numpy's arange function receive the start, end & interval value to generate an array of evenly spaced values.
# Here, the values range from 0 -> 5, evenly spaced by 0.2
t = np.arange(0., 5., 0.2)

plt.plot(t, t, 'r--', t, t ** 2, 'bs', t, t ** 3, 'g^')
plt.show()

output_88_0.png

Controlling line properties

x = np.arange(0., 5., 0.2)
y = x ** 2
plt.plot(x, y, linewidth=4.0)
plt.show()

output_91_0.png

line1, line2, line3 = plt.plot(x, x, x, x ** 2, x, x ** 3)
line1.set_color('red')
line2.set_color('green')
line3.set_color('blue')
line3.set_alpha(0.2)
plt.show()

output_93_0.png

plt.setp(line1)
plt.getp(line1)

Working with multiple figures and axes

def f(t):
    return np.exp(-t) * np.cos(2 * np.pi * t)

t1 = np.arange(0.0, 5.0, 0.1)
t2 = np.arange(0.0, 5.0, 0.02)

# Select the 1st figure
plt.figure(1)
plt.subplot(2, 1, 1)
plt.plot(t1, f(t1), 'bo', t2, f(t2), 'r')

plt.subplot(2, 1, 2)
plt.plot(t2, np.cos(2 * np.pi * t2), 'r--')

# Select the 2nd figure
plt.figure(2)
# 3 rows, 2 cols & set the (2, 2) as active
plt.subplot(324)
plt.plot(t2, np.cos(2 * np.pi * t2), 'b-')

# 3 rows, 2 cols & set the (1, 1) as active
plt.subplot(321)
plt.plot(t2, np.cos(2 * np.pi * t2), 'g-')

plt.show()

output_99_0.png

output_99_1.png

Working with text

mu, sigma = 100, 15
x = mu + sigma * np.random.randn(10000)

n, bins, patches = plt.hist(x, 50, normed=1, facecolor='g', alpha=0.75)
plt.xlabel('Smarts')
plt.ylabel('Probability')
plt.title('Histogram of IQ as normal distribution of $\mu=100,\ \sigma=15$')
plt.text(57, .025, r'$\mu=100,\ \sigma=15$', color='r')
plt.axis([40, 160, 0, 0.03])
plt.grid(True)
plt.show()

output_102_0.png

Annotating text

ax = plt.subplot(111)
t = np.arange(0.0, 5.0, 0.01)
s = np.cos(2 * np.pi * t)
line, = plt.plot(t, s, lw=2)
plt.annotate('local max', xy=(2, 1), xytext=(3, 1.5), arrowprops=dict(facecolor='k', shrink=0.05))
plt.ylim(-2, 2)
plt.show()

output_105_0.png

Working with legend

x = np.array(range(100))
line1, line2 = plt.plot(x, x, 'r', x, x ** 2, 'b')
line1.set_label('$Y=x$')
line2.set_label('$Y=x^2$')
plt.legend(loc=2, title='Explanation')
# plt.legend((line1, line2), ('For $Y=x$', 'For $Y=x^2$'))  # Or use legend function to set labels for lines
plt.show()

output_108_0.png

Creating artists specifically for adding to the legend (aka. Proxy artists)

import matplotlib.patches as mpatches

red_patch = mpatches.Patch(color='r', label='The red data')
plt.legend(handles=[red_patch])

plt.show()

output_110_0.png

import matplotlib.lines as mlines

line1, = plt.plot([1,3,4,5], label='Real line', color='r')
blue_line = mlines.Line2D([], [], color='blue', marker='*', linestyle='',
                          markersize=15, label='Blue stars')
plt.legend(handles=[line1, blue_line])
plt.show()

output_111_0.png

Legend location

x = np.array(range(100))
line1, = plt.plot(x, x, 'r', label='$Y=x$')
line2, = plt.plot(x, x ** 2, 'g--', label='$Y=x^2$')
plt.legend([line1, line2], ['For $Y=x$', 'For $Y=x^2$'], loc=2, bbox_to_anchor=(1.05, 1), borderaxespad=0.0)
plt.show()

output_114_0.png

Multiple legends on the same Axes

line1, = plt.plot([1, 2, 3], label='Line1', linestyle='--')
line2, = plt.plot([3, 2, 1], label='Line2', linewidth=4)

# Create the legend for 1st line
first_legend = plt.legend(handles=[line1], loc=1)

# Add a new legend
ax = plt.gca().add_artist(first_legend)

# Create another legend for the 2nd line
plt.legend(handles=[line2], loc=2)

plt.show()

output_116_0.png

Financial charts

Candle stick chart

from matplotlib.dates import DateFormatter, WeekdayLocator,\
    DayLocator, MONDAY
from matplotlib.finance import quotes_historical_yahoo_ohlc, candlestick_ohlc


# (Year, month, day) tuples suffice as args for quotes_historical_yahoo
date1 = (2004, 2, 1)
date2 = (2004, 4, 12)


mondays = WeekdayLocator(MONDAY)        # major ticks on the mondays
alldays = DayLocator()              # minor ticks on the days
weekFormatter = DateFormatter('%b %d')  # e.g., Jan 12
dayFormatter = DateFormatter('%d')      # e.g., 12

quotes = quotes_historical_yahoo_ohlc('INTC', date1, date2)
if len(quotes) == 0:
    raise SystemExit

fig, ax = plt.subplots()
fig.set_size_inches(10, 5)
fig.subplots_adjust(bottom=0.2)
ax.xaxis.set_major_locator(mondays)
ax.xaxis.set_minor_locator(alldays)
ax.xaxis.set_major_formatter(weekFormatter)
#ax.xaxis.set_minor_formatter(dayFormatter)

#plot_day_summary(ax, quotes, ticksize=3)
candlestick_ohlc(ax, quotes, width=0.6)

ax.xaxis_date()
ax.autoscale_view()
plt.setp(plt.gca().get_xticklabels(), rotation=45, horizontalalignment='right')

plt.show()

output_119_0.png

(From http://matplotlib.org/examples/pylab_examples/finance_demo.html)

Stock prices with indicators

import datetime
import numpy as np
import matplotlib.colors as colors
import matplotlib.finance as finance
import matplotlib.dates as mdates
import matplotlib.ticker as mticker
import matplotlib.mlab as mlab
import matplotlib.pyplot as plt
import matplotlib.font_manager as font_manager


startdate = datetime.date(2006, 1, 1)
today = enddate = datetime.date.today()
ticker = 'SPY'


fh = finance.fetch_historical_yahoo(ticker, startdate, enddate)
# a numpy record array with fields: date, open, high, low, close, volume, adj_close)

r = mlab.csv2rec(fh)
fh.close()
r.sort()


def moving_average(x, n, type='simple'):
    """
    compute an n period moving average.

    type is 'simple' | 'exponential'

    """
    x = np.asarray(x)
    if type == 'simple':
        weights = np.ones(n)
    else:
        weights = np.exp(np.linspace(-1., 0., n))

    weights /= weights.sum()

    a = np.convolve(x, weights, mode='full')[:len(x)]
    a[:n] = a[n]
    return a


def relative_strength(prices, n=14):
    """
    compute the n period relative strength indicator
    http://stockcharts.com/school/doku.php?id=chart_school:glossary_r#relativestrengthindex
    http://www.investopedia.com/terms/r/rsi.asp
    """

    deltas = np.diff(prices)
    seed = deltas[:n+1]
    up = seed[seed >= 0].sum()/n
    down = -seed[seed < 0].sum()/n
    rs = up/down
    rsi = np.zeros_like(prices)
    rsi[:n] = 100. - 100./(1. + rs)

    for i in range(n, len(prices)):
        delta = deltas[i - 1]  # cause the diff is 1 shorter

        if delta > 0:
            upval = delta
            downval = 0.
        else:
            upval = 0.
            downval = -delta

        up = (up*(n - 1) + upval)/n
        down = (down*(n - 1) + downval)/n

        rs = up/down
        rsi[i] = 100. - 100./(1. + rs)

    return rsi


def moving_average_convergence(x, nslow=26, nfast=12):
    """
    compute the MACD (Moving Average Convergence/Divergence) using a fast and slow exponential moving avg'
    return value is emaslow, emafast, macd which are len(x) arrays
    """
    emaslow = moving_average(x, nslow, type='exponential')
    emafast = moving_average(x, nfast, type='exponential')
    return emaslow, emafast, emafast - emaslow


plt.rc('axes', grid=True)
plt.rc('grid', color='0.75', linestyle='-', linewidth=0.5)

textsize = 9
left, width = 0.1, 0.8
rect1 = [left, 0.7, width, 0.2]
rect2 = [left, 0.3, width, 0.4]
rect3 = [left, 0.1, width, 0.2]


fig = plt.figure(figsize=(10, 5), facecolor='white')
axescolor = '#f6f6f6'  # the axes background color

ax1 = fig.add_axes(rect1, axisbg=axescolor)  # left, bottom, width, height
ax2 = fig.add_axes(rect2, axisbg=axescolor, sharex=ax1)
ax2t = ax2.twinx()
ax3 = fig.add_axes(rect3, axisbg=axescolor, sharex=ax1)


# plot the relative strength indicator
prices = r.adj_close
rsi = relative_strength(prices)
fillcolor = 'darkgoldenrod'

ax1.plot(r.date, rsi, color=fillcolor)
ax1.axhline(70, color=fillcolor)
ax1.axhline(30, color=fillcolor)
ax1.fill_between(r.date, rsi, 70, where=(rsi >= 70), facecolor=fillcolor, edgecolor=fillcolor)
ax1.fill_between(r.date, rsi, 30, where=(rsi <= 30), facecolor=fillcolor, edgecolor=fillcolor)
ax1.text(0.6, 0.9, '>70 = overbought', va='top', transform=ax1.transAxes, fontsize=textsize)
ax1.text(0.6, 0.1, '<30 = oversold', transform=ax1.transAxes, fontsize=textsize)
ax1.set_ylim(0, 100)
ax1.set_yticks([30, 70])
ax1.text(0.025, 0.95, 'RSI (14)', va='top', transform=ax1.transAxes, fontsize=textsize)
ax1.set_title('%s daily' % ticker)

# plot the price and volume data
dx = r.adj_close - r.close
low = r.low + dx
high = r.high + dx

deltas = np.zeros_like(prices)
deltas[1:] = np.diff(prices)
up = deltas > 0
ax2.vlines(r.date[up], low[up], high[up], color='black', label='_nolegend_')
ax2.vlines(r.date[~up], low[~up], high[~up], color='black', label='_nolegend_')
ma20 = moving_average(prices, 20, type='simple')
ma200 = moving_average(prices, 200, type='simple')

linema20, = ax2.plot(r.date, ma20, color='blue', lw=2, label='MA (20)')
linema200, = ax2.plot(r.date, ma200, color='red', lw=2, label='MA (200)')


last = r[-1]
s = '%s O:%1.2f H:%1.2f L:%1.2f C:%1.2f, V:%1.1fM Chg:%+1.2f' % (
    today.strftime('%d-%b-%Y'),
    last.open, last.high,
    last.low, last.close,
    last.volume*1e-6,
    last.close - last.open)
t4 = ax2.text(0.3, 0.9, s, transform=ax2.transAxes, fontsize=textsize)

props = font_manager.FontProperties(size=10)
leg = ax2.legend(loc='center left', bbox_to_anchor=(1.05, 1.3), shadow=True, fancybox=True, prop=props)
leg.get_frame().set_alpha(0.5)


volume = (r.close*r.volume)/1e6  # dollar volume in millions
vmax = volume.max()
poly = ax2t.fill_between(r.date, volume, 0, label='Volume', facecolor=fillcolor, edgecolor=fillcolor)
ax2t.set_ylim(0, 5*vmax)
ax2t.set_yticks([])


# compute the MACD indicator
fillcolor = 'darkslategrey'
nslow = 26
nfast = 12
nema = 9
emaslow, emafast, macd = moving_average_convergence(prices, nslow=nslow, nfast=nfast)
ema9 = moving_average(macd, nema, type='exponential')
ax3.plot(r.date, macd, color='black', lw=2)
ax3.plot(r.date, ema9, color='blue', lw=1)
ax3.fill_between(r.date, macd - ema9, 0, alpha=0.5, facecolor=fillcolor, edgecolor=fillcolor)


ax3.text(0.025, 0.95, 'MACD (%d, %d, %d)' % (nfast, nslow, nema), va='top',
         transform=ax3.transAxes, fontsize=textsize)

#ax3.set_yticks([])
# turn off upper axis tick labels, rotate the lower ones, etc
for ax in ax1, ax2, ax2t, ax3:
    if ax != ax3:
        for label in ax.get_xticklabels():
            label.set_visible(False)
    else:
        for label in ax.get_xticklabels():
            label.set_rotation(30)
            label.set_horizontalalignment('right')

    ax.fmt_xdata = mdates.DateFormatter('%Y-%m-%d')


class MyLocator(mticker.MaxNLocator):
    def __init__(self, *args, **kwargs):
        mticker.MaxNLocator.__init__(self, *args, **kwargs)

    def __call__(self, *args, **kwargs):
        return mticker.MaxNLocator.__call__(self, *args, **kwargs)

# at most 5 ticks, pruning the upper and lower so they don't overlap
# with other ticks
#ax2.yaxis.set_major_locator(mticker.MaxNLocator(5, prune='both'))
#ax3.yaxis.set_major_locator(mticker.MaxNLocator(5, prune='both'))

ax2.yaxis.set_major_locator(MyLocator(5, prune='both'))
ax3.yaxis.set_major_locator(MyLocator(5, prune='both'))

plt.show()

output_122_0.png

(From http://matplotlib.org/examples/pylab_examples/finance_work2.html)

XKCD-style sketch plots

with plt.xkcd():
    line1, = plt.plot([1, 2, 3], label='Line1', linestyle='--')
    line2, = plt.plot([3, 2, 1], label='Line2', linewidth=4)

    # Create the legend for 1st line
    first_legend = plt.legend(handles=[line1], loc=1)

    # Add a new legend
    ax = plt.gca().add_artist(first_legend)

    # Create another legend for the 2nd line
    plt.legend(handles=[line2], loc=2)

    plt.show()

output_126_0.png

with plt.xkcd():
    ax = plt.subplot(111)
    t = np.arange(0.0, 5.0, 0.01)
    s = np.cos(2 * np.pi * t)
    line, = plt.plot(t, s, lw=2)
    plt.annotate('local max', xy=(2, 1), xytext=(3, 1.5), arrowprops=dict(facecolor='k', shrink=0.05))
    plt.ylim(-2, 2)
    plt.show()

output_127_0.png

with plt.xkcd():
    x = np.array(range(100))
    line1, = plt.plot(x, x, 'r', label='$Y=x$')
    line2, = plt.plot(x, x ** 1.5, 'g--', label='$Y=x^2$')
    plt.legend([line1, line2], ['For $Y=x$', 'For $Y=x^{1.5}$'], loc=2, bbox_to_anchor=(1.05, 1), borderaxespad=0.0)
    plt.show()

output_128_0.png

Credit

Recommended Posts

Mémo de base Python - Partie 2
Mémo de base Python - Partie 1
Mémo de grammaire de base Python (1)
Mémorandum de base Python partie 2
Notes de commande de base Python
Mémo de grammaire de base Python
Application Python: Pandas Partie 1: Basique
Mémo Python
mémo python
Mémo Python
Grammaire de base Python (divers) Mémo (3)
mémo python
Mémo Python
Grammaire de base Python (divers) Mémo (2)
Grammaire de base Python (divers) Mémo (4)
Mémo Python
Application Python: visualisation de données partie 1: basique
QGIS + Python Partie 2
[Python] Mémo sur le dictionnaire
RF Python Basic_01
mémo débutant python (9.2-10)
QGIS + Python Partie 1
mémo débutant python (9.1)
Mémo de base Flask
Écriture de base Python
★ Mémo ★ Python Iroha
Mémo opérateur Python 3
Python: grattage partie 1
Grammaire de base Python3
[Mon mémo] python
Mémo de métaclasse Python3
RF Python Basic_02
[Python] Mémo de fond de carte
Mémo débutant Python (2)
Python3 commence la partie 1
[Python] Mémo Numpy
Python: grattage, partie 2
Mémorandum de base Python Partie 3-A propos de l'orientation des objets-
Cours de base Python (12 fonctions)
Édition de base Python Memo
Classe Python (mémo d'apprentissage Python ⑦)
installation de python openCV (mémo)
Cours de base Python (2 installation Python)
Tri de base en Python
Mémo de visualisation par Python
Mémo de calcul de base Numpy
Cours de base Python (9 itérations)
Cours de base Python (11 exceptions)
[Python] Mémo sur les fonctions
Liste de rédaction de mémo partie 2
Cours de base Python (6 sets)
Aide-mémoire Python3 (basique)
mémo d'expression régulière python
Recherche de bisection (python2.7) mémo
Grammaire de base Python (divers)
[Mon mémo] python -v / python -V
Cours de base Python (Introduction)