The Foundry’s Blink is a C++-like language designed for Rapid Image Processing. It uses standard C++ syntax with a few keyword changes. At runtime, it uses the LLVM compiler to generate the code required to run on a particular device.
Runtime compilation allows us to avoid generating output code for every possible circumstance. For example, we don’t need to generate code for every possible combination of image data types and component counts, but can make code only for the specific input and output types we require. We can also produce code to target the hardware available at runtime. As an example of this, the SIMD code will use AVX instructions if the CPU supports them, or SSE2 if not. Once generated, the code is cached on disk for future use to avoid the overhead of compiling every time.
Runtime compilation also enables fast algorithm prototyping. An example of this is the BlinkScript node for Nuke, which provides a convenient interface for testing out your image processing code, and allows you to combine Blink operations together in Nuke’s node graph.
The cornerstone of our Blink framework is the concept of a Blink “kernel”. A kernel is a simply piece of code that will be run at each position in the iteration space. For example, in image processing your iteration space will usually be the bounds of the output image you want to produce. The kernel will be run at every position in the output image in order to produce the output picture.
A Blink kernel is written in a similar manner to a C++ class, inheriting from the class Kernel which takes Granularity as a a template parameter. For example:
kernel BasicKernel : ImageComputationKernel<Granularity>
{
//Kernel body goes here
};
The kernel body should contain the following:
A kernel can be iterated in either a componentwise or pixelwise manner. Componentwise iteration means that the kernel will be executed once for each component at every point in the iteration space. Pixelwise means it will be called once only for every point in the iteration space.
The options for the kernel granularity are:
A kernel image specification takes the form:
Image<ReadSpec, AccessPattern, EdgeMethod> myImage;
The template arguments describe how we will access myImage:
The default value is eAccessPoint.
The default value is eEdgeNone.
The process() method is a function that will be run at every point in the iteration space. It can have one of three signatures:
If you define multiple kernel functions with different signatures, only the first one will be used.
This kernel simply copies its input image, src, to its output image, dst.
kernel CopyKernel : ImageComputationKernel<eComponentWise>
{
Image<eRead, eAccessPoint, eEdgeClamped> src;
Image<eWrite> dst;
void process()
{
dst() = src();
}
};
To access src and dst at the current position and component, you can just use the () operator, as above. You could write a pixelwise version of the same kernel in exactly the same way:
kernel CopyKernel : ImageComputationKernel<ePixelWise>
{
Image<eRead, eAccessPoint, eEdgeClamped> src;
Image<eWrite> dst;
void process()
{
dst() = src();
}
};
In this case, src() returns a value of type SampleType(src), which is actually a vector of values, one per component. Each value is of type ValueType(src).
You could also write the pixelwise version of the kernel as follows:
kernel CopyKernel : ImageComputationKernel<ePixelWise>
{
Image<eRead, eAccessPoint, eEdgeClamped> src;
Image<eWrite> dst;
void process()
{
for (int component = 0; component < dst.kComps; component++)
dst(component) = src(component);
}
};
To access a single component inside a pixelwise kernel, you can put the component index inside the () operator, as above. dst.kComps gives you the number of components inside the dst image.
For more on how to access images, see Image Access.
Kernel variables can have two different levels of visibility. These are:
Both variable types are declared inside variable blocks with the same visibility, as they would be in C++. For example:
class MyKernel : public Kernel<eComponentWise>
{
Image<eWrite> dst;
param:
float myParameter1;
int myParameter2;
local:
int myVariable1;
float myVariable2;
void kernel()
{
dst() = (float)(myParameter2 * myVariable1) + myParameter2 * myVariable2;
}
};
Both param and local variables can be standard C++ types such as float, int and bool. Arrays of C++ types are also supported: float[], int[], bool[]. In addition, there are some standard vector types: int2, int3, int4, float2, float3 and float4. For completeness, we also provide the vector types int1 and float1.
Individual components of vector types can be accessed using .x, .y, .z and .w for the first, second, third and fourth components respectively. For example, if you have a variable of a vector type called vec, the first component can be accessed using vec.x.
The special types recti and rectf represent rectangles with integer and floating-point co-ordinates respectively. For example,
recti myRect(int x1, int y1, int x2, int y2)
represents a rectangle with its bottom left corner at (x1, y1) and its top right corner at (x2, y2). The co-ordinates for the rectangle can then be accessed using myRect.x1, myRect.y1, myRect.x2 and myRect.y2.
define() is a special method that’s used to provide more information about kernel parameters. This method should only contain calls to the function defineParam(), which has the following form:
defineParam(paramName, "externalParamName", defaultValue);
For example, if you had a floating-point param named “size” that you wanted to refer to externally as “Radius” and give a default value of 10, you would call defineParam like so:
defineParam(size, "Radius", 10.0f);
The define() function will be called just once, when the kernel is first created, to define the parameters.
Image access from inside the kernel(...) function depends on both the kernel granularity and the image specification. Certain types of image access must first be set up in the kernel’s init() method.
The init() method is called before the kernel() function is run. It’s used for setting up access to images and initialising local variables. If you don’t need to do any set up or initialisation, you need not provide an init() function in your kernel.
Ranged image access must be set up inside the init() function. eRanged1D access first needs to have an axis specified to indicate whether the range is vertical or horizontal. The setAxis() function is used to do this:
Both eAccessRanged1D and eAccessRanged2D access need to have a range specified. Both ranges can be specified with the following call:
In the eAccessRanged2D case, the range can also be set separately for each axis:
Neither eAccessPoint nor eAccessRandom image access require any initialisation inside the init() function.
The init() function can also be used to set the value of local variables in the kernel, to avoid repeating expensive computation at every point in the iteration space.
At this stage, both the image specifications and the parameter values with which the kernel will be executed are known, and can be used in local variable calculations. The following information is available from an image, image, in both the init() and kernel() functions:
Both read and write access are performed by calling the image with a variable number of parameters, according to the image specification and the granularity of the kernel. Parameters related to the image specification are passed in first:
No parameters related to the image specification are needed.
image()
The first parameter is an integer offset along the chosen axis for the point in the image to be accessed. The offset is relative to the current position in the iteration space.
image(int offset)
The first two parameters are integer offsets in the x- and y-axes respectively for the point in the image to be accessed. Both offsets are relative to the current position in the iteration space.
image(int horizontalOffset, int verticalOffset)
The first two parameters are the x and y coordinates of the position in the image you wish to access.
image(int x, int y)
This will always give you access to point (x, y) in the image, whatever the current position in the iteration space.
Finally, with a kernel granularity of ePixelWise you can specify an extra parameter at the end of the list which is the index, c, of the component you wish to access:
//eAccessPoint component access
image(int c)
//eAccessRanged1D component access
image(int offset, int c)
//eAccessRanged2D component access
image(int horizontalOffset, int verticalOffset, int c)
//eAccessRandom component access
image(int x, int y, int c)
All image accesses are by reference. With pixelwise access, if you don’t specify a component to access, you will get back a reference to a vector containing the values for all components at the position requested. The vector will be of type image.SampleType. If you do specify a component, or you have specified componentwise access, you will get back a reference to a component of type image.ValueType.
There is also a bilinear function which can be used to access pixels or components at non-integer positions within an Image. It does bilinear interpolation to estimate the appropriate value from the four pixels or components nearest to the requested position. In a pixelwise kernel,
bilinear(Image img1, float x, float y)
will return a value of type SampleType(img1), while in a componentwise kernel the return type will be ValueType(img1). Within a pixelwise kernel you can also use
bilinear(Image img1, float x, float y, int c)
to perform bilinear interpolation at position (x, y) on just the component c.
The functions below are available within a Blink kernel.
Here, vec can be any of int1, int2, int3, int4, float1, float2, float3, float4 and scalar is the corresponding scalar type for the vector (i.e. int for the former four types and float for the latter).
scalar dot(vec a, vec b); // Returns the dot product of vector *a* with vector *b*.
vec3 cross(vec3 a, vec3 b); // Returns the cross product of vector *a* with vector *b* (3-component vector types only).
scalar length(vec a); // Returns the length of vector *a*.
vec normalize(vec a); // Returns the result of dividing vector *a* by its length.
The following standard maths functions are available, where type can be any of the scalar or vector types we support, and int_type can be any of int, int1, int2, int3 or int4.
type sin(type a);
type cos(type a);
type tan(type a);
type asin(type a);
type acos(type a);
type atan(type a);
type atan2(type a, type b);
type exp(type a);
type log(type a);
type log2(type a);
type log10(type a);
//Rounding:
type floor(type a);
type ceil(type a);
int_type round(type a); //Rounds a to the nearest integer
//Powers and square roots
type pow(type a, type b);
type sqrt(type a);
type rsqrt(type a); //Returns 1 over the square root of a
//Absolute functions
type fabs(type a);
int_type abs(int_type a);
//Integer and fractional parts
type fmod(type a, type b);
type modf(type a, type *b);
//Sign function
type sign(type a);
//Min and max functions
type min(type a, type b);
type max(type a, type b);
type clamp(type a, type min, type max); //Clamp a to be between min and max
//Reciprocal function
type rcp(type a);
void atomicAdd(int_type* ptr, int val); //Atomically increment the value at the address pointed to by ptr by val
void atomicInc(int_type* ptr); //Atomically increment the value at the address pointed to by ptr by 1
scalar median(scalar data[], int size); //Finds the median value in an array of data of length size
The following functions are available on the rectangle type. Here rect could be recti or rectf. vec and scalar will be integer types for a recti and floating-point types for a rectf.
//Constructors
rect(); //Construct without initialising
rect(scalar x1, scalar y1, scalar x2, scalar y2); //Construct a rectangle that goes from (x1, y1) to (x2, y2)
//Bounds functions
rect grow(scalar x, scalar y); //Grow the bounds by x in the horizontal axis and y in the vertical
bool inside(scalar x, scalar y); //Return whether or not the position (x, y) is inside the rectangle
bool inside(vec v); //Return whether or not the position represented by the vector v is inside the rectangle
//Size functions
scalar width(); //Return the width of the rectangle
scalar height(); //Return the height of the rectangle
vec height(); //Return a vector containing the width and height of the rectangle