Three advanced pybind11 features for wrapping C++ code into Python

Elegant features from a more civilized language.

Three advanced pybind11 features to bring fancy C++ features into your Python code by wrapping:

Shared pointers.
Enum.
Abstract base classes (ABC) and pure virtual methods.

The starting point for this project is a previous project found here (the code for which is here)…. But you don’t have to read it — I’ll walk you through the setup.

All the code for this project can be found here.

Requirements

Obviously, you will need pybind11. On Mac:

brew install pybind11

will do it. Otherwise, go buy a Mac and install brew and then pybind11 . Or just use whatever.

Setup with CMake

We will start with the CMake based setup from a previous introduction found here.

Are you gonna check it out? Of course not. No time for that! I’ll walk you through it. Here is the setup:

The directory structure is as follows:

cpp/CMakeLists.txt  
cpp/include/automobile  
cpp/include/automobile\_bits/motorcycle.hpp  
cpp/src/motorcycle.cpp  
python/automobile.cpp  
python/automobile.hpp  
CMakeLists.txt

The idea here is:

The inner cpp folder contains a C++ project for a library. It can be built using the CMakeLists.txt as follows:

cd cpp  
mkdir build  
cd build  
cmake ..  
make  
make install

The outer folder contains the wrapping code in the python library, and a second CMakeLists.txt for building the python library as follows:

mkdir build  
cd build  
cmake .. -DPYTHON\_LIBRARY\_DIR=”/path/to/site-packages” -DPYTHON\_EXECUTABLE=”/path/to/executable/python3"  
make  
make install

My paths are:

DPYTHON\_LIBRARY\_DIR=”/Users/USERNAME/opt/anaconda3/lib/python3.7/site-packages”  
DPYTHON\_EXECUTABLE=”/Users/USERNAME/opt/anaconda3/bin/python3"

I won’t review all the files here — you can find them in this repo.

Shared pointers

C++ standard 11 introduced shared and unique pointers which do not require manual memory cleanup.

This is highly parallel to Python, where garbage collection is automatic.

Wrapping shared pointers into Python is therefore only natural.

We will add a static constructor method that returns a shared_ptr to a Motorcycle. Add to cpp/include/automobile_bits/motorcycle.hpp the constructor:

and the implementation in cpp/src/motorcycle.cpp:

There are two parts now: (1) we must allow a shared_ptr<Motorcycle> to be accessible by Python, and (2) we need to expose the create method.

For the first part, we will modify the glue code in python/motorcycle.cpp:

For the second part, to wrap the static method, we will also add:

Notice that we used def_static instead of def for a static method.

Build and install the library as before. The test python code:

works as expected with output:

Zoom Zoom on road: mullholland

Remember that the complete code is here if you got lost.

Enum

Enum are great for setting flags or options in a more verbose way than simply true/false or 1/2/3/4…. They are supported in both Python and C++.

Let’s create an enum in C++. In the header cpp/include/automobile_bits/motorcycle.hpp add:

above the Motorcycle class, as well as the public method of the Motorcycle class:

and it’s implementation in cpp/src/motorcycle.cpp:

Add the following glue code in python/motorcycle.cpp below the Motorcycle wrapper:

and expose the method in the Motorcycle class:

Finally, the proof in Python is:

returns

EngineType.TWO\_STROKE

Remember that the complete code is here if you got lost.

Abstract base classes

Python also has an abc module that is entirely underused in the Python community. I guess there are not enough projects that work extensively with inheritance? Clearly, in C++, it is all the rage.

As we shall see, using pybind11, the basic principles of abstract base classes will translate nicely from C++ to Python, but some behavior is missing.

Add to the header cpp/include/automobile_bits/motorcycle.hpp:

and of course, no implementation for is_beautiful (although you could have one!). Probably this should be in it’s own header file, but it doesn’t really matter here.

What happens when we try to add the glue code in python/motorcycle.cpp? If we try:

we’ll get the error

Allocating an object of abstract class type ‘autos::Photograph’

Uh oh! It looks like it is unhappy with the constructor. Of course, we could simply eliminate the constructor:

This compiles — but now consider the following example in Python:

This gives the error:

TypeError: YamahaPhoto: No constructor defined!

because of course, we deleted the constructor! So abstract base classes are no longer extensible.

The solution is to define what pybind11 refers to as a “trampoline” class. In python/motorcycle.cpp, define the trampoline at the top:

and change the glue code to:

Notice here the order in py::class_<autos::Photograph, autos::PhotographTrampoline> — first the parent class (the ABC), then the trampoline.

Everywhere else, we use just the name of the ABC, i.e. Photograph::is_beautiful, not PhotographTrampoline::is_beautiful.

Now we could also add the constructor without an error.

The python example will now run and produce a resounding True. Remember that the complete code is here if you got lost.

A limitation here is that the Photograh class in Python is no longer an abstract base class. That means, we can actually run the following:

which will construct a YamahaPhoto object, despite the fact that we did not implement the is_beautiful method.

This is unfortunate, as it breaks some of the design principles enforced in C++. At the moment, it seems we just cannot have everything — but maybe one day!

Final thoughts

That’s three advanced features of pybind11 — some things are not so obvious, but we can in the end port over much of the C++ code we love with minimal effort.