-
Notifications
You must be signed in to change notification settings - Fork 29
How Python and C fits together
(aka seeing C in Python pipelines)
Bifrost is based on two maxims:
- by itself, C sucks.
- by itself, Python sucks.
Bifrost bridges the two languages using some clever python libraries, a customized numpy array class, and enforcing some pipeline friendly design choices.
Suppose you have some custom C code which you want to integrate into a bifrost pipeline. Here's a nice and simple snippet that adds two arrays together:
for(int i=0; i < nelements; i+=1)
{
x[i] = x[i] + y[i];
}
To get this into bifrost and use it in Python, you need to 'bifrostify' this code to
accept bifrost's special ndarray
class.
Bifrost has a special array class in python that plays nicely with C:
import bifrost as bf
a = bf.ndarray([1,2,3,4,5,6,7,8,9,10], dtype='f32')
b = bf.ndarray([2,3,4,5,6,7,8,9,10,11], dtype='f32')
This ndarray
object is very similar to the numpy.array
, but it has a special method, .as_BFarray()
.
Essentially, as_BFarray
returns a pointer to the numpy array's memory address, along with some other
useful stuff.
z = a.as_BFarray()
# Tab complete will show you this object has:
z.big_endian z.dtype z.shape
z.conjugated z.immutable z.space
z.data z.ndim z.strides
In the bifrost C++ code, there is a matching BFarray
that makes interfacing with Python
straightforward. This is essentially a struct that provides the same info as the Python ndarray
.
A usage example is worth a thousand words, so here is our simple snippet from before, after it's been bifrostified:
BFstatus AddStuff(BFarray *xdata, BFarray *ydata)
{
long nelements = num_contiguous_elements(xdata);
float* x = (float *)xdata->data;
float* y = (float *)ydata->data;
for(int i=0; i < nelements; i +=1)
{
x[i] = x[i] + y[i];
}
return BF_STATUS_SUCCESS;
}
A full code example (with headers etc) can be found at the end.
Now comes the cool part. You'll need to edit three files:
- create
add_stuff.cpp
insrc
- create
add_stuff.h
insrc/bifrost
- edit
src/Makefile
and addadd_stuff.o
in the LIBBIFROST_OBJS part
Now, run make
and make install
from the root directory to rebuild the libbifrost.so
library.
Once this is recompiled, open up iPython and try this:
import bifrost as bf
from bifrost.libbifrost import _bf
b = bf.ndarray([1,2,3,4,5,6,7,8,9,10], dtype='f32')
a = bf.ndarray([1,2,3,4,5,6,7,8,9,10], dtype='f32')
_bf.AddStuff(a.as_BFarray(), b.as_BFarray())
print a
That is: your AddStuff
function is available in python via some pyclibrary magic. You can't just pass a
and b
by themselves, but you can send their as_BFarray()
output.
Bravo, you've managed to run C++ code in bifrost! All the rest of the pipeline stuff is just python (to be continued...)
#include <bifrost/cpu_add.h>
#include <bifrost/array.h>
#include <bifrost/common.h>
#include <bifrost/ring.h>
#include <utils.hpp>
#include <stdlib.h>
#include <stdio.h>
#include <iostream>
extern "C" {
BFstatus AddStuff(BFarray *xdata, BFarray *ydata)
{
long nelements = num_contiguous_elements(xdata);
float* x = (float *)xdata->data;
float* y = (float *)ydata->data;
for(int i=0; i < nelements; i +=1)
{
x[i] = x[i] + y[i];
}
return BF_STATUS_SUCCESS;
}
}
#include <bifrost/common.h>
#include <bifrost/array.h>
extern "C" {
BFstatus AddStuff(BFarray *xdata, BFarray *ydata);
}
- Home
- Introductions and examples
- Getting started guide
- Common installation and execution problems
- Some helpful tips and warnings
- Reference guides
- Python API
- Ring() API
- C++ Development