Subsections

 
12.6 Numeric emulation API

These notes describe the Numeric compatability functions which enable numarray to utilize a subset of the extensions written for Numeric (NumPy). Not all Numeric C-API features and therefore not all Numeric extensions are currently supported. Users should be able to utilize suitable extensions written for Numeric within the numarray environment by:

  1. Writing a numarray setup.py file.
  2. Scanning the extension C-code for all instances of array creation and return and making corrections as needed and specified below.
  3. Re-compiling the Numeric C-extension for numarray.

Numarray's compatability with Numeric consists of 3 things:

  1. A replacement header file, "arrayobject.h" which supplies emulation functions and macros for numarray just as the original arrayobject.h supplies the C-API for Numeric.
  2. Layout and naming of the fundamental numarray C-type, PyArrayObject, in Numeric compatible way.
  3. A set of "emulation" functions. These functions have the same names and parameters as the original Numeric functions, but operate on numarrays. The emulation functions are also incomplete; features not currently supported should result in compile time warnings.

PyArrayObject was discussed in an earlier section. The header file, ``arrayobject.h'' is now fairly minimal, mainly just including libnumarray.h, so needs little discussion. The following section will discuss the Numeric compatible functions.

 
12.6.1 Emulation Functions

The basic use of numarrays by Numeric extensions is achieved in the extension function's wrapper code by:

  1. Ensuring creation of array objects by calls to emulation functions.
  2. DECREFing each array and/or calling PyArray_Return.

Unlike prior versions of numarray, this version *does* support access to array objects straight out of PyArg_ParseTuple. This is a consequence of a change to the underlying object model, where a class instance has been replaced by PyArrayObject. Nevertheless, the ``right'' way to access arrays is either via the high level interface or via emulated Numeric factory functions. That way, access to other python sequences is supported as well.

The creation of array objects is illustrated by the following of wrapper code for a 2D convolution function:

static PyObject *
Py_Convolve2d(PyObject *obj, PyObject *args)
{
        PyObject   *okernel, *odata, *oconvolved=Py_None;
        PyArrayObject *kernel, *data, *convolved;

        if (!PyArg_ParseTuple(args, "OO|O", &okernel, &odata, &oconvolved)) {
                return PyErr_Format(_Error, 
                                    "Convove2d: Invalid parameters.");  
                goto _fail;
        }

The first step was simply to get object pointers to the numarray parameters to the convolution function: okernel, odata, and oconvolved. Oconvolved is an optional output parameter, specified with a default value of Py_None which is used when only 2 parameters are supplied at the python level. Each of the ``o'' parameters should be thought of as an arbitrary sequence object, not necessarily an array.

The next step is to call emulation functions which convert sequence objects into PyArrayObjects. In a Numeric extension, these calls map tuples and lists onto Numeric arrays and assert their dimensionality as 2D. The numarray emulation functions first map tuples, lists, and misbehaved numarrays onto well-behaved numarrays. Thus, calls to the emulation factory functions transparently use the numarray high level interface and provide visibility only to aligned and non-byteswapped array objects.

        kernel = (PyArrayObject *) PyArray_ContiguousFromObject(
                okernel, tFloat64, 2, 2);
        data = (PyArrayObject *) PyArray_ContiguousFromObject(
                odata, tFloat64, 2, 2);

        if (!kernel || !data) goto _fail;

Extra processing is required to handle the output array convolved, cloning it from data if it was not specified. Code should be supplied, but is not, to verify that convolved and data have the same shape.

        if (convolved == Py_None)
                convolved = (PyArrayObject *) PyArray_FromDims(
                        data->nd, data->dimensions, tFloat64);
        else
                convolved = (PyArrayObject *) PyArray_ContiguousFromObject(
                        oconvolved, tFloat64, 2, 2);
        if (!convolved) goto _fail;

After converting all of the input paramters into PyArrayObjects, the actual convolution is performed by a seperate function. This could just as well be done inline:

        Convolve2d(kernel, data, convolved);

After processing the arrays, they should be DECREF'ed or returned using PyArray_Return. It is generally not possible to directly return a numarray object using Py_BuildValue because the shadowing of mis-behaved arrays needs to be undone. Calling PyArray_Return destroys any temporary and passes the numarray back to Python.

        Py_DECREF(kernel);
        Py_DECREF(data);
        if (convolved != Py_None) {
                Py_DECREF(convolved);
                Py_INCREF(Py_None);
                return Py_None;
        } else
                return PyArray_Return(convolved);
_fail:
        Py_XDECREF(kernel);
        Py_XDECREF(data);
        Py_XDECREF(convolved);
        return NULL;
}

Byteswapped or misaligned arrays are handled by a process of shadowing which works like this:

  1. When a "misbehaved" numarray is accessed via the Numeric emulation functions, first a well-behaved temporary copy (shadow) is created by NA_IoArray.
  2. Operations performed by the extension function modifiy the data buffer belonging to the shadow.
  3. On extension function exit, the shadow array is copied back onto the original and the shadow is freed.
All of this is transparent to the user; if the original array is well-behaved, it works much like it always did; if not, what would have failed altogether works at the cost of extra temporary storage. Users which cannot afford the cost of shadowing need to use numarray's native elementwise or 1D APIs.

 
12.6.2 Numeric Compatible Functions

The following functions are currently implemented:

PyArrayObject* PyArray_FromDims(int nd, int *dims, int type)
This function will allocate a new numarray.

An array created with PyArray_FromDims can be used as a temporary or returned using PyArray_Return.

Used as a temporary, calling Py_DECREF deallocates it.

PyObject* PyArray_FromDimsAndData(int nd, int *dims, int type, char *data)
This function will allocate a numarray of the specified shape and type, and it's contents will be copied from the specified data pointer.

PyObject* PyArray_ContiguousFromObject(PyObject *op, int type, int min_dim, int max_dim)
Returns an emulation object for a contiguous numarray of 'type' created from the sequence object 'op'. If 'op' is a contiguous, aligned, non-byteswapped numarray, then the emulation object refers to it directly. Otherwise a well-behaved numarray will be created from 'op' and the emulation object will refer to it. min_dim and max_dim bound the expected rank as in Numeric. min_dim==max_dim specifies an exact rank. min_dim==max_dim==0 specifies any rank.

PyObject* PyArray_CopyFromObject(PyObject *op, int type, int min_dim, int max_dim)
Returns a contiguous array, similar to PyArray_FromContiguousObject, but always returning an emulation object referring to a new numarray copied from the original sequence.

PyObject* PyArray_FromObject(PyObject *op, int type, int min_dim, int max_dim)
Returns and emulation object based on 'op', possibly discontiguous. The strides array must be used to access elements of the emulation object.

If 'op' is a byteswapped or misaligned numarray, FromObject creates a temporary copy and the emulation object refers to it.

If 'op' is a nonswapped, aligned numarray, the emulation object refers to it.

If 'op' is some other sequence, it is converted to a numarray and the emulation object refers to that.

PyObject* PyArray_Return(PyArrayObject *apr)
Returns emulation object 'apr' to python. The emulation object itself is destructed. The numarray it refers to (base) is returned as the result of the function.

An additional check is (or eventually will be) performed to guarantee that rank-0 arrays are converted to appropriate python scalars.

PyArray_Return has no net effect on the reference count of the underlying numarray.

int PyArray_As1D(PyObject **op, char **ptr, int *d1, int typecode)
Copied from Numeric verbatim.

int PyArray_As2D(PyObject **op, char ***ptr, int *d1, int *d2, int typecode)
Copied from Numeric verbatim.

int PyArray_Free(PyObject *op, char *ptr)
Copied from Numeric verbatim. Note: This means including bugs and all!

int PyArray_Check(PyObject *op)
This function returns 1 if op is a PyArrayObject.

int PyArray_Size(PyObject *op)
This function returns the total element count of the array.

int PyArray_NBYTES(PyArrayObject *op)
This function returns the total size in bytes of the array, and assumes that bytestride == itemsize, so that the size is product(shape)*itemsize.

PyObject* PyArray_Copy(PyArrayObject *op)
This function returns a copy of the array 'op'.

int PyArray_CanCastSafely(PyArrayObject *op, int type)
This function returns 1 IFF the array 'op' can be safely cast to 'type', otherwise it returns 0.

PyArrayObject* PyArray_Cast(PyArrayObject *op, int type)
This function casts the array 'op' into an equivalent array of type 'type'.

PyArray_Descr* PyArray_DescrFromType(int type)
This function returns a pointer to the array descriptor for 'type'. The numarray version of PyArray_Descr is incomplete and does not support casting, getitem, setitem, one, or zero.

int PyArray_isArray(PyObject *o)(T)
his macro is designed to fail safe and return 0 when numarray is not installed at all. When numarray is installed, it returns 1 iff object 'o' is a numarray, and 0 otherwise. This macro facilitates the optional use of numarray within an extension.

 
12.6.3 Unsupported Numeric Features

Send comments to the NumArray community.