What's New in Katana 3.0

3Delight Renderer

Katana now bundles 3Delight: a uni-directional path-tracer designed to withstand the high demands of production rendering. Workflow options for the 3Delight plug-in include:

  • Fully interactive live rendering (add, subtract, modify lights, objects, materials and cameras)
  • Simplified render pass management and sampling controls
  • Interactive Light Mixing via the Multilight tools and the GafferThree node
  • Fast atmosphere rendering for all light types with deep output
  • Unified light sampling across all light types
  • Support for Cryptomatte workflows
  • Fully OSL shading engine

The Katana installer presents 3Delight as an optional component of the Katana installation. 3Delight can be used for free in preview and live renders. Disk and batch rendering requires a 3delight_r license to remove watermarks; please contact sales@foundry.com for further information. Katana's example projects now use 3Delight where a renderer if required.

Note:  Katana continues to support other renderers via the renderer-agnostic Renderer Plug-in API.

Hydra-Powered Viewer

Katana 3.0 includes a new 3D scene viewer tab that is substantially more interactive and scalable than the existing Viewer tab.

The new viewer is powered by Hydra, a modern, scalable OpenGL rendering architecture that was developed at Pixar Animation Studios as a key component of the open source Universal Scene Description (USD) project.

The Viewer (Hydra) tab is implemented using Katana's Viewer API, which has been revised and expanded from Katana 2.6.

Performance Improvements

Performance improvements for Katana 3.0 include:

  • Improvements to Katana's scene processing library, Geolib3.

    • Greatly reduced memory and computation overhead for 'no-op' scene graph locations.

    • Experimental support for abort-free cooking of the scene dependencies. Normally Geolib3 aborts and re-schedules a cook operation if location data is requested from a location that has not been cooked, or is otherwise not in Geolib3's cache. When this feature is enabled, when Geolib3 encounters missing location data it will instead recursively cook that data and then return to the original Op at the point the location data was requested. This feature can be enabled by setting the environment variable, KATANA_ENABLE_RECURSIVE_COOKS=1.

    • Optimizations to FnAttribute:

      • Commonly-created attributes (such as integers [0, 256], floats 0.0f and 1.0f, doubles 0.0 and 1.0, the empty string and the empty group) are now created only once and then reused subsequently. This avoids the overheads of memory allocation and manipulating an FnAttribute object's reference count.

      • FnAttribute classes are now move-aware, defining move constructors and move-assign operators. With supported compilers, this can reduce the overhead of manipulating an FnAttribute object's reference count.

      • DataAttribute classes are now able to take ownership of the data buffers they are constructed with, rather than creating a separate copy. This can help reduce peak memory usage and reduce overall render times as the data does not need to be copied into the FnAttribute's internal buffers.
    • Optimization of several performance-critical Geolib3 Ops, including AlembicIn, AttributeSet, ConstraintResolve, LookFileResolve and OpScript.

    • Optimizations to avoid making temporaries copies of strings at API boundaries. For instance, many methods on the GeolibCookInterface now take string parameters by StringView instead of const std::string&.

  • Improvements to the performance of the Render Log when processing large volumes of log output, Monitor when handling image updates from the renderer, and Scene Graph when a large number of locations are open or pinned.

  • Improvements to the handling of the node graph's namespace, reducing time taken to load, search and manipulate the node graph.

  • Improvements to the synchronization time between the node graph and Geolib3 Op tree.

  • Improvements to application start-up time and CPU usage.

User Interface Improvements

Katana 3 includes a number of feature enhancements and bug fixes that improve the overall user experience of the application. Notable changes are:

  • A revised color palette for a more modern and flat look, to allow artists to focus on their work.

  • Larger color swatches.

  • Tooltips for parameters, attributes, project settings, and preferences, which make information like the type of an attribute more easily accessible.

  • Redesigned checkboxes and radio buttons to differentiate between the two types of controls.

  • A reorganized Tabs main menu and Add Tab menu, which clearly differentiates between built-in types of tabs, tab types being loaded from Katana resource locations, and custom types of tabs that may be loaded from a user's home directory.

  • Many more UI elements that scale with the application font size preference.

We will continue to refine aspects of the user interface in Katana releases to come, incorporating feedback we receive from our customers.

Deprecated Features

  • The AttributeScript node type has been deprecated, and hidden from node creation menus of Node Graph tabs. AttributeScript nodes in existing Katana projects still work, and AttributeScript nodes can still be created via NodegraphAPI scripting, but for new projects, we advise using OpScript nodes and Lua instead. Support for AttributeScript nodes will be removed in a future release.

  • The classic Gaffer node type, which was superseded by GafferThree in Katana 2.0v1, is now hidden from node creation menus in the Node Graph tab.

  • The following little-used 2D nodes have been hidden from menus in the Node Graph tab: ImageAddMix, ImageAtop, ImageAverage, ImageBlack, ImageDifference, ImageDistort, ImageDivide, ImageExclusion, ImageFrom, ImageGeometric, ImageHypot, ImageMatte, ImageMatteMix, ImageMax, ImageMin, ImageMinus, ImageMix, ImageMultiply, ImageOver, ImagePlus, ImageScreen, ImageUnder, ImageWhite. These node types can still be created through scripting. Nodes in existing projects continue to function as before.

  • The color/useSingleComponentRGB preference, which was not used in commercial Katana, has been deprecated.

Third-party Software Changes

The table below summarizes changes to versions of third-party software shipped with Katana.

Version in Katana 2.6 Version in Katana 3.0v9
3Delight Not shipped 13.4.9
Alembic 1.5.3 1.6.1
LuaJIT Not shipped 2.1.0
Numpy 1.5.1 (Linux only) Not shipped
Python 2.7.3 2.7.13
RLM 9.3 12.2
TBB 4.4.6 2017 Update 8
USD Not shipped 0.8.0
PyQt 4.10.4 4.11.4
ZeroMQ 3.2.5 4.2.1
PyZeroMQ 13.0.2 16.0.2
VS CRT 2010 (Windows only) Not shipped

Compiler Changes

Katana 3.0 is compiled with GCC 4.8 on Linux, and Visual Studio 2015 on Windows.

Katana 2.5 and Katana 2.6 were built with GCC 4.1 on Linux, and Visual Studio 2010 on Windows.

The change in compilers should not affect most customers. For compatibility reasons, Katana has long separated its APIs into a set of C++ headers and sources compiled directly into customer plug-in libraries and a stable, compiler-agnostic C-based API that's wrapped by the C++ layer. Plug-in authors writing Geolib Ops, for example, are still free to build their plug-ins with their choice of toolchain.

This compatibility note does not apply for Windows plug-in authors building plug-ins that link against Python or Qt: customers are required to re-build these plug-ins using Visual Studio 2015.

Note:  The standard Windows compiler for building Python 2.7 C extensions is Visual Studio 2008, but this compiler must not be used for building Python C extensions for Katana. Customers may find it necessary to override distutils/setuptools' choice of compiler in order to build native Python extensions that are compatible with Katana. See https://docs.python.org/2.7/distutils/apiref.html#module-distutils.msvccompiler for instructions.

API Changes

Katana 3.0's APIs continue to build with older compilers that do not support the C++11 standard. Support for C++98 compilers will be removed in a future release of Katana.

Katana C++ plug-ins built against Katana 2.5 or later should be binary compatible with Katana 3.0 (with the exception of Windows plug-ins that link to Python or Qt, and plug-ins using the in-development Viewer API).

Katana C++ plug-ins built against Katana 2.5 or later should be source compatible with the Katana 3.0 APIs, with a few exceptions, described below.

Example Plug-ins

Katana's example plug-ins are being progressively migrated to C++11, and are no longer guaranteed to build in older compilers that do not support the C++11 standard.

Viewer API

The Viewer API was developed in parallel with the Hydra Viewer, and has diverged substantially from the version found in Katana 2.6v4. Information on the Viewer API in Katana 3.0 can be found in the Katana Developer Guide.

Asset Plug-ins

Python-based AssetAPI plug-ins have been deprecated, and support for them will be removed in a future release. Moving forward, for performance and stability reasons, AssetAPI plug-ins should be written in C++.

Cook Interface

GeolibCookInterface::execOp() now supports passing private, user-defined data (a.k.a. "private data") to the executed Op. This also resolves an issue where the caller's private data would be inherited by the callee. In the revised behavior, if the caller of execOp() does not specify any private data then getPrivateData() will return NULL in the callee. Ops that are built against the 2.5 or 2.6 SDK retain the ability to leak private data into an Op executed by execOp(). However, we recommend re-building against the 3.0 SDK and using the privateData parameter of execOp() instead. There have been no change to OpScript, which does not support private data.

FnAttribute

The implementation of FnAttribute has been made more performant:

  • Static attributes: certain attributes (such as integers [0, 256], floats 0.0f and 1.0f, doubles 0.0 and 1.0, the empty string and the empty group) are now created only once and then reused subsequently. That means their reference counts don't have to be incremented and decremented.
  • Move constructor and operators: these new operators (C++11) have been implemented in FnAttribute. If a new compiler is used, it will detect when an object can be moved, meaning that its reference count won't have to be incremented and decremented.

Note:  Katana 3.0v9 uses a different algorithm to calculate attribute hashes, and so attribute hashes will differ from those in Katana 2.x.

The behaviour of DataAttribute's getNearestSample() method has changed, and in some cases this will require code changes in order to build C++ plug-ins against the Katana 3.0 APIs.

In previous releases of Katana it was possible to obtain a raw pointer to a sample buffer by calling getNearestSample(time).data(). For performance reasons, as of Katana 3.0 this idiom is disallowed, and results in a compile-time error. FnAttribute now requires that the ConstVector object returned by getNearestSample() is assigned to a local variable before calling its data() method. Some examples follow:

// Works with both Katana 2 and Katana 3. `p` is valid until either of `sample`
// or `myIntAttr` are destructed.
auto sample = myIntAttr.getNearestSample(.0f);
const int* p = sample.data();

// Results in a compile-time error in Katana 3.
const int* p = myIntAttr.getNearestSample(.0f).data();

To facilitate getting raw pointers to sample buffers of multi-sampled DataAttribute objects, Katana 3.0 introduces a new C++ API in the form of the getSamples() method. Some examples follow:

for (const auto& sample : myIntAttr.getSamples())
{
    ProcessSample(sample.getSampleTime(), sample.data(), sample.size());
}

// Assign the result of `getSamples` to a local variable if you want pointers to
// remain valid for longer. Here, the raw pointers stored in `vec` remain valid
// until either the `myIntAttr` or `samples` objects are destructed.
auto samples = myIntAttr.getSamples();
vector<tuple<float /*time*/, const int* /*data*/, size_t /*size*/>> vec;
for (const auto& sample : samples)
{
    vec.emplace_back(sample.getSampleTime(), sample.data(), sample.size());
}

Additionally, a new getValuesAs() function has been added that returns a C++ object constructed with a fixed number of values from the attribute's primary time sample. For example, if you have a FloatAttribute representing a 4x4 matrix, you can convert it to an Imath matrix object using code similar to:

auto m44f = myFloatAttr.getValuesAs<Imath::M44f, 16>();

FnAttributes are now able to take ownership of the data buffers they are constructed with. This can help reduce peak memory usage and reduce overall render times as the data does not need to be copied into the FnAttribute's internal buffers.

For an FnAttribute to take ownership of the data buffer callers must pass a deleter function and non-null data pointer (context) to the FnAttribute's constructor. This will result in the following:

  1. The FnAttribute will take copies of the values of the sample times i.e. {-0.5, 0.0, 0.5}.
  2. It will copy the value of the pointers for each time sample buffer (the pointer, not what it points to).
  3. On destruction, the attribute will call the user-supplied function, passing the user-supplied context to it; see below for example use cases.
  4. The sample buffers and user-supplied context (they can be different) must remain valid for the lifetime of the FnAttribute which has taken ownership of the data.

Example Usage:

// -----------------------------------------------------------------------------
// Create an IntAttribute with a single time sample of 10 values.

int64_t tupleSize = 1;
int64_t valueCount = 10;

// Create a buffer to store our data -- this must live on the heap, as the
// attribute we create stores pointers into this memory block.
unique_ptr<int[]> buf(new int[valueCount]);

// Fill `buf` with data.
// ...

FnAttribute::IntAttribute myIntAttr(buf.get(), valueCount, tupleSize, buf.get(),
                                    [](void* context) {
                                      delete[] static_cast<int*>(context);
                                    });
// myIntAttr now owns the buffer.
buf.release();

// -----------------------------------------------------------------------------
// Create a FloatAttribute with three time samples of 64 values.

int64_t tupleSize = 4;
int64_t valueCount = 64;

int64_t sampleCount = 3;

// Create a buffer to store our data -- this must live on the heap, as the
// attribute we create stores pointers into this memory block.
unique_ptr<float[]> buf(new float[valueCount * sampleCount]);

// Fill `buf` with data.
// ...

// FnAttribute copies these buffers into internal storage; they need not live on
// the heap.
const float* samples[] = {&buf[valueCount * 0],
                          &buf[valueCount * 1],
                          &buf[valueCount * 2]};
float sampleTimes[] = {-0.5f, 0.0f, 0.5f};

FnAttribute::FloatAttribute myFloatAttr(sampleTimes, sampleCount, samples,
                                        valueCount, tupleSize, buf.get(),
                                        [](void* context) {
                                          delete[] static_cast<float*>(context);
                                        });
// myFloatAttr now owns the buffer.
buf.release();

It is now possible to create StringAttributes that take ownership of user supplied memory. As a result of this, StringAttribute hash calculations have also now changed for regular and zero copy hashes.

The following example demonstrates creating a multi-sampled StringAttribute that takes ownership of the string buffers supplied to it.

const char* kString1 = "We hope";
const char* kString2 = "you enjoy";
const char* kString3 = "Katana 3.0v1.";
    
// Prepare the string data.
char** valueArrayForSample1 = new char*[1];
char* dataForSample1 = new char[strlen(kString1) + 1];
memcpy(dataForSample1, kString1, strlen(kString1) + 1);
valueArrayForSample1[0] = dataForSample1;

char** valueArrayForSample2 = new char*[1];
char* dataForSample2 = new char[strlen(kString2) + 1];
memcpy(dataForSample2, kString2, strlen(kString2) + 1);
valueArrayForSample2[0] = dataForSample2;

char** valueArrayForSample3 = new char*[1];
char* dataForSample3 = new char[strlen(kString3) + 1];
memcpy(dataForSample3, kString3, strlen(kString3) + 1);
valueArrayForSample3[0] = dataForSample3;

// Prepare the value pointers and sample time array.
char** valuePtrs[3] = {valueArrayForSample1, valueArrayForSample2, valueArrayForSample3};
float sampleTimes[3] = {-0.5f, 0.0f, 0.5f};

// This vector will be passed to StringAttribute's constructor so we can
// clean up the data when the attribute is to be destroyed.
auto* context = new std::vector<char**>{
           valueArrayForSample1, valueArrayForSample2, valueArrayForSample3};

FnAttribute::StringAttribute zeroCopyStringAttribute(sampleTimes, 3,
                                                     (const char***)valuePtrs, 1,
                                                     1,
                                                     context,
                                                     [](void* context){
    auto* memoryToFree = static_cast<std::vector<char**>*>(context);
    for(char** dataForSample : *memoryToFree)
    {
        delete[] dataForSample[0];
        delete[] dataForSample;
    }
    delete memoryToFree;
});

// Access the attribute data as normal.
auto firstSample = zeroCopyStringAttribute.getNearestSample(-0.5f);
auto secondSample = zeroCopyStringAttribute.getNearestSample(0.0f);
auto thirdSample = zeroCopyStringAttribute.getNearestSample(0.5f);
std::cout << "[-0.5] = " << firstSample[0] << std::endl;
std::cout << "[0.0] = " << secondSample[0] << std::endl;
std::cout << "[0.5] = " << thirdSample[0] << std::endl;

Usage of Katana's FnAttribute library from external applications (via the FnAttribute::Bootstrap() function) now consistently requires a katana_r license.

OpScript

Support for the OpScript legacy API has been removed. This API was available by default in some alpha releases of Katana 2.0v1, but has been disabled by default since then.

OpScript nodes have been upgraded to use LuaJIT 2.1.0 as their Lua interpreter, which should result in general performance improvements as well as laying the foundation for future improvements. The change should be generally transparent, however the previous OpScript implementation based on Lua 5.1 is still available should you encounter issues with the LuaJIT-based one. This implementation can be selected by setting the environment variable KATANA_OPSCRIPT_INTERPRETER to Lua_5_1.

LuaJIT follows the Lua 5.1 language standard more strictly, and some Lua 5.0 constructs that were grandfathered into Lua 5.1 are not available in LuaJIT. These are:

  • the math.mod() function; use math.fmod() or the % operator instead.
  • the implicit arg parameter for old-style vararg functions; use Lua 5.1 vararg expressions (...) instead
  • the string.gfind() function; use string.gmatch() instead

Note that we build LuaJIT 2.1 in "GC64" mode so that OpScript is not subject to the address-space limitations of earlier LuaJIT releases.

For further information, see:

Renderer Plug-ins

The DiskRenderOutputProcess::setRenderAction() method's renderAction parameter's type has changed from std::auto_ptr<RenderAction> to FnPlatform::internal::UniquePtr (a.k.a. std::unique_ptr). As std::unique_ptr is not copyable, FnPlatform::internal::UniquePtr::move (a.k.a. std::unique_ptr::move) needs to be invoked explicitly in order to move the internal pointer from one container to another.

Display Drivers

It is now possible to configure pixel data sent to Katana via the Display Driver API as either RGBA or ARGB. RGBA is the internal pixel format used by Katana and using this pixel format ensures Katana does not have to do a channel swizzle to re-order the alpha channel. The following example demonstrates how enable this channel pixel format:

NewChannelMessage_v2Ptr newChannelMessage_v2(
    new FnKat::NewChannelMessage_v2(*newFrameMessage));
// Setup NewChannelMessage_v2
newChannelMessage_v2->setChannelID(/* Channel ID */);
newChannelMessage_v2->setChannelOrigin(0, 0);
newChannelMessage_v2->setChannelDimensions(1000, 1000);
newChannelMessage_v2->setSampleRate(sampleRate);
newChannelMessage_v2->setDataSize(4 * sizeof(float));
newChannelMessage_v2->setPixelLayout(FnKat::NewChannelMessagev2::PixelLayout::RGBA);
newChannelMessage_v2->setChannelName(legacyChannelName);

Improved performance of Katana's display driver API, including reduced memory allocation and message transmission overhead.

A new 'zero copy' option for DataMessage types has been introduced. This allows display driver authors to transfer ownership of a pixel buffer to the display driver API, which significantly reduces pixel copy and memory allocation overhead. For example:

// Pixel buffer clean-up callback.
void ImageDataDeleter(void* data)
{
    char* dataArray = reinterpret_cast<char*>(data);
    delete[] dataArray;
}

// Sending pixel buffers back to Katana...
FnKat::DataMessage dataMsg(/* ... */);
// Configure coordinates/dimensions unchanged...

// Allocate pixel buffer...
char *dataArray = new char[blockSize];

// NEW: Pass pixel data clean-up callback to setData() and the Display Driver 
// API will take ownership of the pixel buffer.
dataMsg.setData(dataArray, blockSize, ImageDataDeleter);

// Send to Katana via Katana Pipe interface
pipe->send(dataMsg);

FnLogging

The C++ class FnLogging::FnLogQueue has been deprecated and will be removed in a future release. This class has never been functional, and logged warnings if you attempted to instantiated it.

Removed APIs

API headers and sources that were deprecated in Katana 2.5 have been removed. These headers are:

  • FnGeolib: util/Mutex.h, util/Platform.h, util/PowerNap.h, util/Semaphore.h, util/ThreadException.h, util/Timer.h

  • FnRender: plugin/SocketConnection.h, plugin/CommandLineTools.h

Common Plug-ins Compatibility

USD

USD binaries compiled against Katana 2.6 are compatible with Katana 3.0.

The USD source code can be compiled against Katana 3.0 as of USD version 0.8.2.

Support for USD/Katana plug-ins on Windows remains unavailable.

RenderMan

RenderMan for Katana binaries compiled against Katana 2.6 are compatible with Katana 3.0.

Arnold

Arnold for Katana binaries compiled against Katana 2.6 are compatible with Katana 3.0.