Blink Reference Guide

Introduction

The Foundry’s Blink is a C++-like language designed for Rapid Image Processing. It uses standard C++ syntax with a few keyword changes. At runtime, it uses the LLVM compiler to generate the code required to run on a particular device.

Currently, we can generate the following code types:

• Standard C++ for the CPU;
• SIMD-optimised C++ for the CPU;
• OpenCL for the GPU.

Runtime compilation allows us to avoid generating output code for every possible circumstance. For example, we don’t need to generate code for every possible combination of image data types and component counts, but can make code only for the specific input and output types we require. We can also produce code to target the hardware available at runtime. As an example of this, the SIMD code will use AVX instructions if the CPU supports them, or SSE2 if not. Once generated, the code is cached on disk for future use to avoid the overhead of compiling every time.

Runtime compilation also enables fast algorithm prototyping. An example of this is the BlinkScript node for Nuke, which provides a convenient interface for testing out your image processing code, and allows you to combine Blink operations together in Nuke’s node graph.

Kernel Basics

The cornerstone of our Blink framework is the concept of a Blink “kernel”. A kernel is a simply piece of code that will be run at each position in the iteration space. For example, in image processing your iteration space will usually be the bounds of the output image you want to produce. The kernel will be run at every position in the output image in order to produce the output picture.

Creating a Kernel

A Blink kernel is written in a similar manner to a C++ class, inheriting from the class Kernel which takes Granularity as a a template parameter. For example:

kernel BasicKernel : ImageComputationKernel<Granularity>
{
  //Kernel body goes here
};

The kernel body should contain the following:

At least one image specification

Image specifications define the images that are read from and written to by the kernel. By convention, these are the first members of the kernel.

A single process() method

The process() method is called for each point in the iteration space. This is where the kernel reads from its inputs and writes to its outputs.

Kernel Granularity

A kernel can be iterated in either a componentwise or pixelwise manner. Componentwise iteration means that the kernel will be executed once for each component at every point in the iteration space. Pixelwise means it will be called once only for every point in the iteration space.

The options for the kernel granularity are:

eComponentWise

The kernel processes the image one component at a time. Only the current component’s value can be accessed in any of the input images, or written to in the output image.

ePixelWise

The kernel processes the image one pixel at a time. All component values can be read from and written to.

Image Specification

A kernel image specification takes the form:

Image<ReadSpec, AccessPattern, EdgeMethod> myImage;

The template arguments describe how we will access myImage:

ReadSpec

This describes how the data in the image can be accessed. The options are: - eRead: Read-only access to the image data. - eWrite: Write-only access to the image data. - eReadWrite: Both read and write access to the image data.

AccessPattern

This describes how the kernel will access pixels in the image. The options are:

• eAccessPoint: Access only the current position in the iteration space.
• eAccessRanged1D: Access a one-dimensional range of positions relative to the current position in the iteration space.
• eAccessRanged2D: Access a two-dimensional range of positions relative to the current position in the iteration space.
• eAccessRandom: Access any pixel in the iteration space.

The default value is eAccessPoint.

EdgeMethod

The edge method for an image defines the behaviour if a kernel function tries to access data outside the image bounds. The options are:

• eEdgeClamped: The edge values will be repeated outside the image bounds.
• eEdgeConstant: Zero values will be returned outside the image bounds.
• eEdgeNone: Values are undefined outside the image bounds and no within-bounds checks will be done when you access the image. This is the most efficient access method to use when you do not require access outside the bounds, because of the lack of bounds checks.

The default value is eEdgeNone.

The process() Method

The process() method is a function that will be run at every point in the iteration space. It can have one of three signatures:

void process()

Use this for kernels which do the same processing regardless of where they are in the iteration space.

void process(int2 pos)

Use this for kernels which need to know the x (pos.x) and y (pos.y) coordinates of their position in the iteration space.

void process(int3 pos)

Only available for kernels with eComponentWise granularity. Here, (pos.x, pos.y) gives the coordinates of the current position in the iteration space, while pos.z is the current component.

If you define multiple kernel functions with different signatures, only the first one will be used.

A Basic Kernel Example

This kernel simply copies its input image, src, to its output image, dst.

kernel CopyKernel : ImageComputationKernel<eComponentWise>
{
  Image<eRead, eAccessPoint, eEdgeClamped> src;
  Image<eWrite> dst;

  void process()
  {
    dst() = src();
  }
};

To access src and dst at the current position and component, you can just use the () operator, as above. You could write a pixelwise version of the same kernel in exactly the same way:

kernel CopyKernel : ImageComputationKernel<ePixelWise>
{
  Image<eRead, eAccessPoint, eEdgeClamped> src;
  Image<eWrite> dst;

  void process()
  {
    dst() = src();
  }
};

In this case, src() returns a value of type SampleType(src), which is actually a vector of values, one per component. Each value is of type ValueType(src).

You could also write the pixelwise version of the kernel as follows:

kernel CopyKernel : ImageComputationKernel<ePixelWise>
{
  Image<eRead, eAccessPoint, eEdgeClamped> src;
  Image<eWrite> dst;

  void process()
  {
    for (int component = 0; component < dst.kComps; component++)
      dst(component) = src(component);
  }
};

To access a single component inside a pixelwise kernel, you can put the component index inside the () operator, as above. dst.kComps gives you the number of components inside the dst image.

For more on how to access images, see Image Access.

Kernel Variables

Variable Visibility

Kernel variables can have two different levels of visibility. These are:

param

Param variables are similar to public member variables in C++. Values for param variables can be accessed from outside the kernel.

local

Local variables are the equivalent of private member variables. They’re used and accessed only from within the kernel.

Both variable types are declared inside variable blocks with the same visibility, as they would be in C++. For example:

class MyKernel : public Kernel<eComponentWise>
{
  Image<eWrite> dst;

  param:
    float myParameter1;
    int myParameter2;

  local:
     int myVariable1;
     float myVariable2;

  void process()
  {
    dst() = (float)(myParameter2 * myVariable1) + myParameter2 * myVariable2;
  }
};

Variable Types

Both param and local variables can be standard C++ types such as float, int and bool. Arrays of C++ types are also supported: float[], int[], bool[]. In addition, there are some standard vector types: int2, int3, int4, float2, float3 and float4. For completeness, we also provide the vector types int1 and float1.

Individual components of vector types can be accessed using .x, .y, .z and .w for the first, second, third and fourth components respectively. For example, if you have a variable of a vector type called vec, the first component can be accessed using vec.x.

The special types recti and rectf represent rectangles with integer and floating-point co-ordinates respectively. For example,

recti myRect(int x1, int y1, int x2, int y2)

represents a rectangle with its bottom left corner at (x1, y1) and its top right corner at (x2, y2). The co-ordinates for the rectangle can then be accessed using myRect.x1, myRect.y1, myRect.x2 and myRect.y2.

The Define Method

define() is a special method that’s used to provide more information about kernel parameters. This method should only contain calls to the function defineParam(), which has the following form:

defineParam(paramName, "externalParamName", defaultValue);

For example, if you had a floating-point param named “size” that you wanted to refer to externally as “Radius” and give a default value of 10, you would call defineParam like so:

defineParam(size, "Radius", 10.0f);

The define() function will be called just once, when the kernel is first created, to define the parameters.

Image Access

Image access from inside the process(…) function depends on both the kernel granularity and the image specification. Certain types of image access must first be set up in the kernel’s init() method.

The Init Method

The init() method is called before the process() function is run. It’s used for setting up access to images and initialising local variables. If you don’t need to do any set up or initialisation, you need not provide an init() function in your kernel.

Configuring ranged image access

Ranged image access must be set up inside the init() function. eRanged1D access first needs to have an axis specified to indicate whether the range is vertical or horizontal. The setAxis() function is used to do this:

setAxis(Axis axis)

axis can be one of eX (horizontal) or eY (vertical).

Both eAccessRanged1D and eAccessRanged2D access need to have a range specified. Both ranges can be specified with the following call:

setRange(int rangeMin, int rangeMax)

This sets the minimum and maximum extent of the range (inclusive on both sides). In the 2D case, it will set the minimum and maximum values to be the same for both axes.

In the eAccessRanged2D case, the range can also be set separately for each axis:

setRange(int xMin, int yMin, int xMax, int yMax)

This allows you to define asymmetric bounds for 2D ranged access. Again, the ranges are inclusive on both sides.

Neither eAccessPoint nor eAccessRandom image access require any initialisation inside the init() function.

Setting local variables

The init() function can also be used to set the value of local variables in the kernel, to avoid repeating expensive computation at every point in the iteration space.

At this stage, both the image specifications and the parameter values with which the kernel will be executed are known, and can be used in local variable calculations. The following information is available from an image, image, in both the init() and process() functions:

image.kMin

The minimum possible value for any component of the image data.

image.kMax

The maximum possible value for any component of the image data.

image.kWhitePoint

The minimum value for any component of the image data which is considered to be white. All values above this will be what are known as “super-whites”. For example, a floating-point image will usually have a white point of 1, though values greater than 1 are also valid.

image.kComps

The number of components in the image.

image.kClamps

Whether the image data should be clamped or not. For example, floating point data can take any value and therefore image.kClamps will be false.

image.bounds

The bounds of the image. Individual lower and upper bounds can be accessed using image.bounds.x1, image.bounds.y1 and image.bounds.x2, image.bounds.y2 respectively.

ValueType(image)

The data type of the image components.

SampleType(image)

The data type for the pixels. For example, if ValueType(image) is float and there are three components in your image, SampleType(image) will be float3.

Image Access from the Kernel Method

Both read and write access are performed by calling the image with a variable number of parameters, according to the image specification and the granularity of the kernel. Parameters related to the image specification are passed in first:

eAccessPoint access:

No parameters related to the image specification are needed.

image()

eAccessRanged1D access:

The first parameter is an integer offset along the chosen axis for the point in the image to be accessed. The offset is relative to the current position in the iteration space.

image(int offset)

eAccessRanged2D access:

The first two parameters are integer offsets in the x- and y-axes respectively for the point in the image to be accessed. Both offsets are relative to the current position in the iteration space.

image(int horizontalOffset, int verticalOffset)

eAccessRandom access:

The first two parameters are the x and y coordinates of the position in the image you wish to access.

image(int x, int y)

This will always give you access to point (x, y) in the image, whatever the current position in the iteration space.

Finally, with a kernel granularity of ePixelWise you can specify an extra parameter at the end of the list which is the index, c, of the component you wish to access:

//eAccessPoint component access
image(int c)

//eAccessRanged1D component access
image(int offset, int c)

//eAccessRanged2D component access
image(int horizontalOffset, int verticalOffset, int c)

//eAccessRandom component access
image(int x, int y, int c)

Return Types for Image Access

All image accesses are by reference. With pixelwise access, if you don’t specify a component to access, you will get back a reference to a vector containing the values for all components at the position requested. The vector will be of type image.SampleType. If you do specify a component, or you have specified componentwise access, you will get back a reference to a component of type image.ValueType.

Bilinear Interpolation

There is also a bilinear function which can be used to access pixels or components at non-integer positions within an Image. It does bilinear interpolation to estimate the appropriate value from the four pixels or components nearest to the requested position. In a pixelwise kernel,

bilinear(Image img1, float x, float y)

will return a value of type SampleType(img1), while in a componentwise kernel the return type will be ValueType(img1). Within a pixelwise kernel you can also use

bilinear(Image img1, float x, float y, int c)

to perform bilinear interpolation at position (x, y) on just the component c.

Functions

The functions below are available within a Blink kernel.

Vector Functions

Here, vec can be any of int1, int2, int3, int4, float1, float2, float3, float4 and scalar is the corresponding scalar type for the vector (i.e. int for the former four types and float for the latter).

scalar dot(vec a, vec b);     // Returns the dot product of vector *a* with vector *b*.
vec3 cross(vec3 a, vec3 b);   // Returns the cross product of vector *a* with vector *b* (3-component vector types only).
scalar length(vec a);         // Returns the length of vector *a*.
vec normalize(vec a);         // Returns the result of dividing vector *a* by its length.

Maths Functions

The following standard maths functions are available, where type can be any of the scalar or vector types we support, and int_type can be any of int, int1, int2, int3 or int4.

Trigonometric functions

type sin(type a);
type cos(type a);
type tan(type a);
type asin(type a);
type acos(type a);
type atan(type a);
type atan2(type a, type b);

Logarithmic and exponential functions

type exp(type a);
type log(type a);
type log2(type a);
type log10(type a);

Other maths functions

//Rounding:
type floor(type a);
type ceil(type a);
int_type round(type a);  //Rounds a to the nearest integer

//Powers and square roots
type pow(type a, type b);
type sqrt(type a);
type rsqrt(type a);     //Returns 1 over the square root of a

//Absolute functions
type fabs(type a);
int_type abs(int_type a);

//Integer and fractional parts
type fmod(type a, type b);
type modf(type a, type *b);

//Sign function
type sign(type a);

//Min and max functions
type min(type a, type b);
type max(type a, type b);
type clamp(type a, type min, type max);  //Clamp a to be between min and max

//Reciprocal function
type rcp(type a);

Atomic Functions (ints only)

void atomicAdd(int_type* ptr, int val);  //Atomically increment the value at the address pointed to by ptr by val
void atomicInc(int_type* ptr);           //Atomically increment the value at the address pointed to by ptr by 1

Statistical Functions

scalar median(scalar data[], int size);  //Finds the median value in an array of data of length size

Rectangle Functions

The following functions are available on the rectangle type. Here rect could be recti or rectf. vec and scalar will be integer types for a recti and floating-point types for a rectf.

//Constructors
rect();                                            //Construct without initialising
rect(scalar x1, scalar y1, scalar x2, scalar y2);  //Construct a rectangle that goes from (x1, y1) to (x2, y2)

//Bounds functions
rect grow(scalar x, scalar y);   //Grow the bounds by x in the horizontal axis and y in the vertical
bool inside(scalar x, scalar y); //Return whether or not the position (x, y) is inside the rectangle
bool inside(vec v);              //Return whether or not the position represented by the vector v is inside the rectangle

//Size functions
scalar width();                  //Return the width of the rectangle
scalar height();                 //Return the height of the rectangle
vec    height();                 //Return a vector containing the width and height of the rectangle