Saturday, January 26, 2013

Running c++ code from python using Boost.Python

I am currently reading a book "Production Volume Rendering". The book images were the main reason I started reading it. However, since the content of the book is written into a library"PVR" which is written in C++ but it is controlled using Python scripts. Thus, to get started, I needed to understand how the whole thing is chained from the c++ code to the python scripts. So I started to get my c++ code to work from a Python script so here it is. This is what I learned. I used VisualStudio 2010 and python27.

In visualstudio 2010, create a new empty win32 dll project named "hello". Next create an empty c++ file. The contents of the file are as follows:

#include < iostream >
using namespace std; 
void say_hello(const char* name) { 
   cout << "Hello " <<  name << "\n"; 


#include < boost/python/module.hpp >
#include < boost/python/def.hpp >


using namespace boost::python; 

BOOST_PYTHON_MODULE(hello) { 
   def("say_hello", say_hello); 


Nothing fancy here. I simply create a function that says hello world on screen usign standard IO routine. 

In order to have my function exported properly I added a module definition file (.def) to the project. This file contains: 
LIBRARY hello 
EXPORTS say_hello 

This tells the compiler to add the say_hello function to dll. 
Next, I compiled and build the dll if all goes well you have the dll created.

Now on the python side. I copied my hello.dll file to an empty directory. This directory contained another file i.e. my python script (hello.py). The python script contains:
import hello 
hello.say_hello("World")

I tried to run the python script using python hello.py 
This returned the following error 
d:\TestHello > python hello.py 
Traceback (most recent call last): 
File "hello.py", line 1, in import hello 
ImportError: No module named hello 

Now searching online I found that for my dll to be recognized by Python, it needs to be renamed to .pyd extension. So i changed the extension to .pyd and ran the python script again. This time, it gave me another error which was 
d:\TestHello > python hello.py 
Traceback (most recent call last): File "hello.py", line 1, in import hello 
ImportError: DLL load failed: The specified module could not be found. 

Now once again I went online to find the reason for this error. Fortunately, on one of the websites, I found the reason for this error. Basically, the Python interpreter is unable to load a dependency dll while loading my hello.pyd (dll). Which dll is it? This I came to know by opening my hello.pyd(dll) in the dependency walker (http://www.dependencywalker.com/). This showed me that I was missing the boost_python-vc100-mt-gd-1_51.dll which is needed. So i copied this dll to the same place as my hello.pyd (dll). 

Now running the python script again ran the code fine. 
d:\TestHello > python hello.py 
Hello World! 

Some notes to take away: 
1) Rename your c++ dll to .pyd otherwise Python will not load it. 
2) Name the module name the same as your dll name i.e. BOOST_PYTHON_MODULE(< your dll name >) 
3) Make sure that all of the dependency dlls are in the same folder or in the path. 

Hope this helps others. I went through a lot of pain with this one. But finally I got it working.

16 comments:

  1. Glad it has helped u too.
    Thanks,
    Mobeen

    ReplyDelete
  2. Excellent tips! I've been battling with VS for a few hours on this. Thanks!

    ReplyDelete
  3. You are welcome. Glad it helped you too ;).

    ReplyDelete
  4. You are always welcome. Glad it helped you too.

    ReplyDelete
  5. After hours of Blockage, Frustration, Irretation finaly Ur Trick helped me to use Boost Library. Kudos.

    ReplyDelete
  6. I ran into the same problems, very helpful! Thank you very much!

    ReplyDelete
  7. where does the dll file gets created? I mean the location path.

    ReplyDelete
  8. That depends on which IDE you are using. For visualstudio it stores the dll in the Debug/Release sub-folder of the current solution folder.

    ReplyDelete
  9. You are a lifesaver. I've been looking for this information for several days, and renaming my dll to `hello.pyd` instead of `hello.dll` was the crucial step I was missing. Thanks a ton!

    ReplyDelete
  10. You are welcome Jimmy. Glad it was helpful.

    ReplyDelete
  11. Thank you ...
    Copy/pasting the *.dll made me save hours ... After having already lost a bit of them !

    Have a nice day

    ReplyDelete
  12. HI Pierre,
    I am glad it helped you too.

    Regards,
    Mobeen

    ReplyDelete