Create wav file from GLSL shader using python3

I've been experimenting with music shader, but it's getting harder to develop with webGL because it doesn't work. So, I made a script to create a wav file from GLSL shader using python. I think it will be easier to adjust detailed settings if python is involved.

Things necessary

Please install pyopengl and pyaudio with pip. If it doesn't work http://www.lfd.uci.edu/~gohlke/pythonlibs/ Downloading the package from here usually works.

This is the script

The explanation here is difficult, so please use it as a library. The sampling rate is set at 48000. If you like the difference, please fix it appropriately.

from OpenGL.GL import *
from OpenGL.WGL import *
from ctypes import *
import numpy
import pyaudio
import wave
import array

def shader2data(duration, src): #Output time, shader music source
    sampleRate = 48000
    numSamples  = duration*sampleRate
    numSamplesC = numSamples*2
    samples = (c_float * numSamplesC)()
    hWnd = windll.user32.CreateWindowExA(0,0xC018,0,0,0,0,0,0,0,0,0,0)
    hDC = windll.user32.GetDC(hWnd)
    pfd = PIXELFORMATDESCRIPTOR(0,1,0,32,0,0,0,0,0,0,0,0,0,0,0,0,0,32,0,0,0,0,0,0,0)
    SetPixelFormat(hDC,ChoosePixelFormat(hDC, pfd), pfd)
    hGLrc = wglCreateContext(hDC)
    wglMakeCurrent(hDC, hGLrc)
    program = glCreateProgram()
    shader = glCreateShader(GL_VERTEX_SHADER)
    glShaderSource(shader, src)
    glCompileShader(shader)
    if glGetShaderiv(shader, GL_COMPILE_STATUS) != GL_TRUE:
      raise RuntimeError(glGetShaderInfoLog(shader).decode())
    glAttachShader(program, shader)
    outs =  cast((c_char_p*1)(b"gain"), POINTER(POINTER(c_char)))
    glTransformFeedbackVaryings(program, 1, outs, GL_INTERLEAVED_ATTRIBS)
    glLinkProgram(program)
    glUseProgram(program)
    vbo = glGenBuffers(1)
    glBindBuffer(GL_ARRAY_BUFFER, vbo)
    glBufferData(GL_ARRAY_BUFFER, sizeof(samples), None, GL_STATIC_DRAW)
    glBindBufferBase(GL_TRANSFORM_FEEDBACK_BUFFER, 0, vbo)
    glUniform1f(glGetUniformLocation(program, "sampleRate"), sampleRate)
    glEnable(GL_RASTERIZER_DISCARD)
    glBeginTransformFeedback(GL_POINTS)
    glDrawArrays(GL_POINTS, 0, numSamples)
    glEndTransformFeedback()
    glDisable(GL_RASTERIZER_DISCARD)
    glGetBufferSubData(GL_ARRAY_BUFFER, 0, sizeof(samples), byref(samples))
    wglMakeCurrent(0, 0);
    wglDeleteContext(hGLrc);
    windll.user32.ReleaseDC(hWnd, hDC);
    windll.user32.PostQuitMessage(0);
    return numpy.frombuffer(samples, dtype=numpy.float32)

def writeWav(filename, data):    
    sampleRate = 48000
    w = wave.Wave_write(filename)
    w.setnchannels(2)
    w.setsampwidth(2)
    w.setframerate(sampleRate)
    w.writeframes(array.array('h', data*32767).tobytes())
    w.close()

def readWav(filename): 
    w = wave.open(filename, 'rb')
    print("params  :", w.getparams())
    print("duration:", w.getnframes() / w.getframerate(), "sec")
    p = pyaudio.PyAudio()
    stream = p.open(format=p.get_format_from_width(w.getsampwidth()),
                channels=w.getnchannels(),
                rate=w.getframerate(),
                output=True)
    stream.write(w.readframes(w.getnframes()))
    stream.close()
    p.terminate()

def playSound(data):    
    sampleRate = 48000
    p = pyaudio.PyAudio()
    stream = p.open(rate=sampleRate, channels=2, format=pyaudio.paFloat32, output=True)
    stream.write(array.array('f', data).tobytes())
    stream.close()
    p.terminate()

Simple rules for writing shader

It uses the same function name and argument name as mainSound (), which is the main function of shadertoy's music shader. So, you can copy and paste the shader toy music shader for this part. Do not change other uniforms, out, main (), etc.

src = """
#version 300 es

out vec2 gain; 
uniform float sampleRate;    

vec2 mainSound( float time )
{
  return vec2(sin(6.2831*440.0*time)); ;
}

void main() {
  float time = float(gl_VertexID) / sampleRate;
  gain =mainSound(time); 
}
"""

This is the source for the shader. If you put this and shader2data (duration, src) with the output time as an argument, sample data will be output. Enter this sample data and file name in writeWav (filename, data) to create a wav file. You can hear the sound with readWav (filename). If you want to output the sound as it is without creating a file, use playSound (data).

data = shader2data(2,src)
filename = "output.wav"
writeWav(filename, data)
readWav(filename)
playSound(data)

I use it like this.

Finally

I don't think there is any demand, but I wrote an article that doubles as a memorandum. I put the ipython file here. https://gist.github.com/gaziya/e5d40b0df1a65510af81b124c4ecf4cc

Recommended Posts

Create wav file from GLSL shader using python3
Create a deb file from a python package
Create a GIF file using Pillow in Python
Run a Python file from html using Django
Create a MIDI file in Python using pretty_midi
Wav file generation from numeric text with python
Create an image file using PIL (Python Imaging Library).
Create your first GDSII file in Python using gdspy
Python script to create a JSON file from a CSV file
Flatten using Python yield from
Create JIRA tickets using Python
Execute Python script from batch file
Using Rstan from Python with PypeR
Create folders from '01' to '12' with python
Notes on using MeCab from Python
Create an Excel file with Python3
Using Cloud Storage from Python3 (Introduction)
Extract the targz file using python
Run Ansible from Python using API
Precautions when using phantomjs from python
Access spreadsheets using OAuth 2.0 from Python
Access Blender Shader Nodes from Python
[Python] File operation using if statement
Easily create homemade RPA using Python
Try using Amazon DynamoDB from Python
From Python to using MeCab (and CaboCha)
Create wordcloud from your tweet with python3
Download the file from S3 using boto.
Quickly create an excel file with Python #python
Create a tool to automatically furigana with html using Mecab from Python3
[Python] Create a Batch environment using AWS-CDK
I tried using UnityCloudBuild API from Python
Create Excel file with Python + similarity matrix
[GPS] Create a kml file in Python
Create a dataframe from excel using pandas
[Hyperledger Iroha] Create an account using Python library
Edit Excel from Python to create a PivotTable
[Python] Split a large Flask file using Blueprint
Create a C array from a Python> Excel sheet
Import Excel file from Python (register to DB)
[Python pandas] Create an empty DataFrame from an existing DataFrame
I want to email from Gmail using Python.
From file to graph drawing in Python. Elementary elementary
How to create a JSON file in Python
Create a New Todoist Task from Python Script
[python] Create table from pandas DataFrame to postgres
Create a phylogenetic tree from Biopyton using ClustalW2
Create 3D printer data (STL file) using CadQuery
Create a web map using Python and GDAL
I tried reading a CSV file using Python
Operate the schedule app using python from iphone
Create a decision tree from 0 with Python (1. Overview)
Create a datetime object from a string in Python (Python 3.3)
Load images from URLs using Pillow in Python 3
Create a Photoshop format file (.psd) with python
Create a Mac app using py2app and Python3! !!
Read line by line from a file with Python
Run a python script from excel (using xlwings)
Script python file
Start using Python
sql from python