It's the 21st day of the OpenCV Advent Calendar. I will post it late.
I think there are many people who want to use various C ++ code using OpenCV in python (fewer). I am one of them. So, in this article, I will make it possible to pass data such as cv :: Mat or std :: vector << cv :: Rect >> to python in numpy format. Of course, if you just call the C ++ code, there is nothing wrong with it. The point of this time is to convert Mat and vector to numpy and pass it, just like using OpenCV in Python.
Boost.Python is a well-known mechanism for calling libraries written in C ++ with python, but if you try to convert a complex class like Mat in C ++ with Boost.Python, it is quite difficult. On top of that, if you try to port not just Mat, but Point, Rect, and all the other classes, you'll die. On the other hand, OpenCV functions can be easily called from python in the first place, and Mat or Rect vector can be easily handled in python in the common format of numpy. The motivation is that if you use this mechanism, you can easily call your own C ++ library from python.
In the end, it looks like this.
c++~ side
cv::Mat aaaa() {
return cv::Mat();
}
std::vector<cv::Rect> bbbb() {
return std::vector<cv::Rect>(4);
}
python side
import xxxx
mat = xxxx.aaaa()
vec = xxxx.bbbb()
Also, I'm sorry, I haven't confirmed the operation on windows this time. I remember giving up in an instant, saying that it wouldn't work for a while. Maybe if you do your best. Ubuntu has a track record of working in the past. As the current test environment, we have confirmed the operation on the Mac at hand.
This time we will use Boost.Python. Boost.Python is a nice library that wraps C ++ classes and functions and makes them available to Python.
http://d.hatena.ne.jp/moriyoshi/20091214/1260779899
The mechanism and merits of Boost.Python are described in detail in, so I will omit it here. I'm not familiar with Boost.Python in the first place ...
By the way, suddenly, OpenCV is in modules / python / common.cmake at build time.
add_custom_command(
OUTPUT ${cv2_generated_hdrs}
COMMAND ${PYTHON_EXECUTABLE} "${PYTHON_SOURCE_DIR}/src2/gen2.py" ${CMAKE_CURRENT_BINARY_DIR} "${CMAKE_CURRENT_BINARY_DIR}/headers.txt"
DEPENDS ${PYTHON_SOURCE_DIR}/src2/gen2.py
DEPENDS ${PYTHON_SOURCE_DIR}/src2/hdr_parser.py
DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/headers.txt
DEPENDS ${opencv_hdrs})
ocv_add_library(${the_module} MODULE ${PYTHON_SOURCE_DIR}/src2/cv2.cpp ${cv2_generated_hdrs})
As shown in, gen2.py is called to convert each OpenCV function so that it can be used in Boost.Python. Furthermore, using cv2.cpp below it, Mat, Rect, etc. are converted from C ++ to numpy. For example, the conversion of Mat is as follows.
PyObject* pyopencv_from(const Mat& m)
{
if( !m.data )
Py_RETURN_NONE;
Mat temp, *p = (Mat*)&m;
if(!p->u || p->allocator != &g_numpyAllocator)
{
temp.allocator = &g_numpyAllocator;
ERRWRAP2(m.copyTo(temp));
p = &temp;
}
PyObject* o = (PyObject*)p->u->userdata;
Py_INCREF(o);
return o;
}
As you can see by reading it, Py_INCREF is used to increase the reference count, and g_numpyAllocator in the middle is used to allocate memory.
UMatData* allocate(int dims0, const int* sizes, int type, void* data, size_t* step, int flags, UMatUsageFlags usageFlags) const
{
if( data != 0 )
{
CV_Error(Error::StsAssert, "The data should normally be NULL!");
// probably this is safe to do in such extreme case
return stdAllocator->allocate(dims0, sizes, type, data, step, flags, usageFlags);
}
PyEnsureGIL gil;
int depth = CV_MAT_DEPTH(type);
int cn = CV_MAT_CN(type);
const int f = (int)(sizeof(size_t)/8);
int typenum = depth == CV_8U ? NPY_UBYTE : depth == CV_8S ? NPY_BYTE :
depth == CV_16U ? NPY_USHORT : depth == CV_16S ? NPY_SHORT :
depth == CV_32S ? NPY_INT : depth == CV_32F ? NPY_FLOAT :
depth == CV_64F ? NPY_DOUBLE : f*NPY_ULONGLONG + (f^1)*NPY_UINT;
int i, dims = dims0;
cv::AutoBuffer<npy_intp> _sizes(dims + 1);
for( i = 0; i < dims; i++ )
_sizes[i] = sizes[i];
if( cn > 1 )
_sizes[dims++] = cn;
PyObject* o = PyArray_SimpleNew(dims, _sizes, typenum);
if(!o)
CV_Error_(Error::StsError, ("The numpy array of typenum=%d, ndims=%d can not be created", typenum, dims));
return allocate(o, dims0, sizes, type, step);
}
And so on. It's hard to do this yourself, so this time let's use the code around here as it is and convert C ++ format Mat etc. to python.
I'm sorry, I've put a lot in my environment, so I'm not sure if the following is enough, but if it's not enough, please comment as much as possible. Comments are welcome that it worked if you put this in.
OpenCV3 with python
numpy
boost
boost-python
As a sample, let's create a new project. This time, we will call it sample_project. Create sample_project in a suitable location.
mkdir sample_project
cd sample_project
Create a directory containing the library code. This time with boost_opencv.
mkdir boost_opencv
Create a folder to put the python test script code.
mkdir scripts
This time it will be the main one. However, diversion is very easy. Copy cv2 from earlier to your library directory. There is no big reason, but rename the code of cv2.cpp as a header file.
cd boost_opencv
wget https://raw.githubusercontent.com/opencv/opencv/3.1.0/modules/python/src2/cv2.cpp
mv cv2.cpp cv2.hpp
I want to use only a part of this time, and I will not use the part automatically generated by gen2.py explained earlier, so I will modify it in various ways. First of all, I will comment out the following include parts scattered in various parts of the code.
#include "pyopencv_generated_types.h"
#include "pyopencv_generated_funcs.h"
#include "pyopencv_generated_ns_reg.h"
#include "pyopencv_generated_type_reg.h"
Next, comment out the entire initialization part. Initialization will be done separately, so you can scrape it. At the beginning of shaving
1351 #if PY_MAJOR_VERSION >= 3
1352 extern "C" CV_EXPORTS PyObject* PyInit_cv2();
1353 static struct PyModuleDef cv2_moduledef =
1354 {
1355 PyModuleDef_HEAD_INIT,
1356 MODULESTR,
1357 "Python wrapper for OpenCV.",
So, I will comment out everything from here to the end. However, this is information at this time, so I think it will change over time. In that case, I think that you can skip here once and remove the part where the error occurred in the build step later. This completes the diversion section.
Here, as a simple example, we will prepare a function that returns Mat and a function that returns a vector of Rect. Here, we will return a 3x3 zero initialization matrix and a vector with four Rects.
boost_opencv/main.cpp
#include <boost/python.hpp>
#include "cv2.hpp"
PyObject* GetImage()
{
cv::Mat cv_img(3, 3, CV_32F, cv::Scalar(0.0));
return pyopencv_from(cv_img);
}
PyObject* GetObject(int index) {
vector_Rect2d rects;
rects.push_back(cv::Rect2d(0,0,0,0));
rects.push_back(cv::Rect2d(0,0,0,0));
rects.push_back(cv::Rect2d(0,0,0,0));
rects.push_back(cv::Rect2d(0,0,0,0));
return pyopencv_from(rects);
}
BOOST_PYTHON_MODULE(libboost_opencv)
{
using namespace boost::python;
import_array();
def("GetImage", &GetImage);
def("GetObject", &GetObject);
}
The most important here is pyopencv_from. Here's the code that converts Mat and vector to numpy. By the way, vector_Rect2d is a vector of Rect2d and is defined somewhere in cv2.cpp. Also, don't forget to call import_array () as a way to use numpy. Also, as for Boost.Python, you need to put the correct library name in the argument of BOOST_PYTHON_MODULE.
Now that the basic preparation is complete, let's write CMakeLists.txt. First, make the root CMakeLists.txt as follows.
CMakeLists.txt
cmake_minimum_required(VERSION 3.7)
project(sample_project)
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${CMAKE_CURRENT_SOURCE_DIR}/cmake-modules)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/lib)
find_package(OpenCV REQUIRED)
find_package(Boost REQUIRED COMPONENTS python)
find_package(PythonLibs REQUIRED)
# in the case of homebrew python
#set(PYTHON_INCLUDE_DIRS "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/include/python2.7/")
#set(PYTHON_LIBRARIES "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib")
find_package(PythonInterp REQUIRED)
find_package(Numpy REQUIRED)
add_subdirectory(boost_opencv)
As a caveat, when loading PythonLibs, you can do it as it is with ubuntu, but if you have installed python using Homebrew on Mac, this does not seem to work. I entered the pass directly because it was troublesome ...
Well, if you leave it as it is, you will be told that Numpy cannot be found, so let's bring in the Numpy include path and library path. As long as NUMPY_INCLUDE_DIR is set in the end, any method is fine, but I used FindNumPy.cmake from Caffe.
mkdir cmake-modules
cd cmake-modules
wget https://raw.githubusercontent.com/BVLC/caffe/master/cmake/Modules/FindNumPy.cmake
In addition, FindNumPy.cmake of caffe seems to have the setting for caffe at the end, so I'm not sure, but let's comment it out.
# caffe_clear_vars(__result __output __error_value __values __ver_check __error_value)
Then the boost_opencv / CMakeLists.txt for the library looks like this: As it is, I will omit the details.
boost_opencv/CMakeLists.txt
project(boost_opencv)
include_directories(${OpenCV_INCLUDE_DIRS} ${Boost_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} ${NUMPY_INCLUDE_DIR})
add_library(boost_opencv SHARED main.cpp)
target_link_libraries(boost_opencv ${OpenCV_LIBS} ${Boost_LIBRARIES} ${PYTHON_LIBRARIES})
This is just that. Just create a build directory and cmake and make.
mkdir build
cd build
cmake ..
make
Then, libboost_opencv.so (libboost_opencv.dylib on Mac) will be created in the lib directory. All you have to do is import this with python and call it.
This is really simple, but I'll post it for now. There is no rule, but prepare a suitable script in the scripts directory.
scripts/test.py
import sys
sys.path.append('../lib')
import libboost_opencv
print(libboost_opencv.GetImage())
print(libboost_opencv.GetObject(0))
By the way, python does not import dylib made on Mac, so I changed the extension to so and it read it. Result is,
[[ 0. 0. 0.]
[ 0. 0. 0.]
[ 0. 0. 0.]]
[[ 0. 0. 0. 0.]
[ 0. 0. 0. 0.]
[ 0. 0. 0. 0.]
[ 0. 0. 0. 0.]]
So I was able to do it. Now you can convert C ++ Mat and Rect vectors to numpy and pass them to python. I'm happy.