Setting up a C++/Python project with pybind11 and CMake

This tutorial shows how to set up a pybind11 project with CMake for wrapping a C++ library into Python.

The final result will be:

A C++ project you can build independent of pybind11.
A Python library generated from wrapping the C++ code.
Both using CMake.

You can find the code for complete project here.

Requirements

Obviously, get pybind11:

conda install -c conda-forge pybind11

Create a C++ project ========================

We will use the outer (current) working directory to build python, and an inner directory called cpp to build the C++ code. First make a C++ directory.

mkdir cpp  
cd cpp

Next, we will initialize a C++ project. Two ways (of many more) are:

Using VS Code. Install the CMake Tools extension. Then, bring up the command pallette and select CMake: Quick start. Follow the prompts and enter a name — I chose automobile. When prompted for library or executable, choose library. Your directory should now look like this:

cpp/build/  
cpp/motorcycle.cpp  
cpp/CMakeLists.txt

We will separate the source and header files — this is always good practice. In the cpp directory, make two new directories:

cd cpp  
mkdir include  
mkdir src

and move the source file:

mv motorcycle.cpp src/

In the include directory, we would like to have a single header to import. This way, we could later simply #include <automobile>. We can organize it as follows:

cd cpp/include  
mkdir automobile\_bits  
touch automobile

Finally, let us create a header file in the cpp/include/automobile_bits directory:

cd cpp/include/automobile\_bits  
touch motorcycle.hpp

The final directory structure should now look like this:

cpp/build  
cpp/CMakeLists.txt  
cpp/include/automobile  
cpp/include/automobile\_bits/motorcycle.hpp  
cpp/src/motorcycle.cpp

Alternatively, manually create the files and directories, such that the final structure is:

cpp/build  
cpp/CMakeLists.txt  
cpp/include/automobile  
cpp/include/automobile\_bits/motorcycle.hpp  
cpp/src/motorcycle.cpp

We will need to edit the current CMakeLists.txt such that it can find the header and source files. I edited mine to read as follows:

Let’s also give the motorcycle.hpp and motorcycle.cpp files some reasonable content. For the header:

and the source:

Yes! I know they’re dumb. Note that we introduced a namespace vehicles — this is always a good idea.

We also need to have the header file find the actual library. Edit the include/automobile file to read:

We can now already build the library:

Using the command line:

cd cpp/build  
cmake ..  
make  
make install

Using your favorite IDE, e.g. XCode:

cd cpp/build  
cmake .. -GXcode

should generate automobile.xcodeproject in the build directory.

Either way, you should get the library to build and install.

Testing the C++ library

Before we go on to wrapping the library into Python, let’s create a test for the C++ library (not a real test, just somewhere for us to mess around!).

Create a new directory in cpp:

cd cpp  
mkdir tests

Here we will again set up a CMake project for our test. Make the directory structure look as follows:

cpp/tests/CMakeLists.txt  
cpp/tests/src/test.cpp

Edit the test.cpp file to read:

and edit the CMakeLists.txt file:

Make and run that bad boy using XCode as before, or from the command line:

mkdir build  
cd build  
cmake ..  
make  
cd ../bin  
./test

Note that the binary will be in the bindirectory. The output should be:

Made a motorcycle called: Yamaha  
Zoom Zoom on road: mullholland

Setting up the Python wrapper =================================

Finally, let’s get to wrapping the library into a Python. We’re moving up a directory! In the main directory, let’s make a new directory called python. It will hold all the glue code:

mkdir python

We also need a CMakeLists.txt file, with contents:

You should be ready to build your Python library! Try:

mkdir build  
cd build  
cmake .. -DPYTHON\_LIBRARY\_DIR=”/path/to/site-packages” -DPYTHON\_EXECUTABLE=”/path/to/executable/python3"  
make  
make install

As usual, you could also generate code using a generator for your favorite IDE, e.g. by adding -GXcode to the cmake command. My paths were:

DPYTHON\_LIBRARY\_DIR=”/Users/USERNAME/opt/anaconda3/lib/python3.7/site-packages”  
DPYTHON\_EXECUTABLE=”/Users/USERNAME/opt/anaconda3/bin/python3"

Note that if you are lazy like me, you can try to add for testing:

set(PYTHON\_LIBRARY\_DIR “/Users/USERNAME/opt/anaconda3/lib/python3.7/site-packages”)  
set(PYTHON\_EXECUTABLE “/Users/USERNAME/opt/anaconda3/bin/python3”)

in your CMakeLists.txt — obviously not a good trick for production!

Fire up python (make sure it’s the same as you specified in PYTHON_EXECUTABLE above) and try:

>>> import automobile  
Traceback (most recent call last):  
File “<stdin>”, line 1, in <module>  
ImportError: dynamic module does not define module export function (PyInit\_automobile)

You got a nice fat error, but that’s OK! We didn’t write the glue code yet, but at least your CMake is working and Python can find your library.

Wrapping the library into Python

Now for the actual logic of wrapping the C++ code into Python. It will take place in the python directory. First create a file which will define the “module export function” that python was complaning about in the last part:

touch python/automobile.cpp

Give it the following content:

Next, we will define the init_motorcycle method that was declared. We will do this in a separate file:

touch python/motorcycle.cpp

Edit it to read:

I always find the code itself to be the best explanation, but some pointers:

”Motorcycle” defines the name of the class in Python — you could change it if you want! Notice also the appearance of the namespace.
.def(py::init<std::string>(), py::arg(“name”)) defines the constructor. The py::arg(“name”) allows you to use named arguments in Python.
.def(“get_name”, py::overload_cast<>( &vehicles::Motorcycle::get_name, py::const_)) wraps the get_name method. Note how the const declaration is wrapped.
.def(“ride”, py::overload_cast<std::string>( &vehicles::Motorcycle::ride, py::const_), py::arg(“road”)); wraps the ride method. The arguments to the method are declared in py::overload_cast<std::string> (separated by commas if multiple), and can again be named using py::arg(“road”). Also note the semicolon at the end — often forgotten, but this should be proper C++ code.

You can now test your library. Run make and make install again to rebuild and install the library.

Fire up python and try it:

should give the same output as before:

Made a motorcycle called: Yamaha  
Zoom Zoom on road: mullholland

You could make another test script with those contents, located in a directory tests/test.py.

Conclusion

That’s it for this tutorial. You can find the complete code here.

The nice part about this setup is that you can build your C++ project in peace from the cpp directory, and then at the end in the outer layer worry about wrapping it into Python.

You can read about more advance pybind11 features in another tutorial I wrote here.

Thanks for reading!

Setting up a C++/Python project with pybind11 and CMake

Requirements

Testing the C++ library

Wrapping the library into Python

Conclusion

Contents