Readers and Writers are an intrinsic part of NUKE. These 2D operators allow us to overcome the problem of reading and writing to and from the ever growing existing file/image formats.
Knowing about the readers and writers architecture is therefore an important part of thirdparty NUKE development, which many times may involve custom file formats.
This section will go through the in’s and out’s of creating, understanding and using Readers and Writers.
Before we delve into the coding part of creating your own readers and writers it is important to understand how NUKE handles the reading and writing of files:
To identify which reader or writer version is needed, NUKE relies on parsing the file extension and finding a matching reader, so NUKE will always try to match up a file with a .mov extension to a movReader.<OS specific extension> reader.
There is still however a problem that can arise from a single format having multiple extensions, an example of this is JPG which can be both represented as .jpg and .jpeg, with just the previous system in place it would mean creating or duplicating the jpgReader extension to satisfy both jpg and jpeg. NUKE provides two systems to help you get around this problem one is through a TCL script file that it tries to invoke just before it attempts to find the reader in question and can be used to redirect a reader. This TCL script must have the same name as the source reader, in the case of .jpg to .jpeg the script would be called jpegReader.tcl , as for its contents it simply needs to redirect to the correct reader as following:
# jpegReader.tcl
load jpgReader
This script is invoked instead of jpegReader.<OS specific extension> and ensures the correctly named jpgReader.<OS specific extension> is loaded for the read.
The other system, is by extending the supported extension types for the reader itself ( will be explained below ).
A Reader in the context of NUKE Thirdparty development provides a base class and interfaces to which NUKE can plug into in order to read a custom piece of data from disk. So in order to create a custom Reader the first step will be to create a new Reader which the base class is a DD::Image::Reader, there is a specialization of this class that can be used in its stead DD::Image::FileRead this class is a specialization of DD::Image::Reader and provides extend file management functionality, this will be explained in the next sub-section section link.
In order to create a custom reader using the DD::Image::Reader base class the derived class for the new reader needs to follow some specific steps and implement the basic interfaces.
The DD::Image::Reader::Description interface must be populated with the relevant information, this allows NUKE to identify and manage the readers and their supported extensions. This static member can be declared as follows:
class MyReader : DD::Image::Reader
{
static const Description d;
...
};
static Reader* build( Read* r, int fd, const unsigned char* b, int n )
{
return new MyReader( r, fd, b, n );
}
static bool test( int fd, const unsigned char* block, int n )
{
return ( block[0] == MY_MAGIC_BYTE );
}
const Reader::Description MyReader::d( "myFile\0", "my file format", build, test );
In this example a class MyReader is derived from DD::Image::Reader, the derived class then declares a static Description member this description is what will control the construction and evaluate whether or not the MyReader is suitable to manipulate the current file, through the declaration of build and test in this case.
The previous implementation will look up for files with the extension .myFile, if this specific format a test is performed to ensure the format matches up to the extension which in this case will be to identify the MY_MAGIC_BYTE as the first element of the buffer. In the case of a misnamed format, NUKE will still test against the tester function. It falls then on the user to ensure the testing is sufficient to determine the file type is readable and avoid collisions with similar file formats.
As well as using the TCL solution suggested in the previous section, multiple extensions can be specified on a reader’s descriptions by constructing it with a NULL separated list for it’s first argument.:
const Reader::Description MyReader::d( "myFile\0someOtherFile", "my file format", build, test );
In this example not only will the reader be used for files with a .myFile extension but also for .someOtherFile.
The constructor should use the DD::Image::Reader::set_info() to setup the area to be accessed from the file IO. This should ( if possible ) be done by a quick read operation. Complex reading, decoding and buffering can be done via DD::Image::Reader::open().
The last function that needs to be overriden in order to create a basic working custom reader is DD::Image::Reader::engine(), this function will implement the actual reading of the file data and perform the user conversion into the NUKE native image format.:
void MyReader::engine( int y, int x, int r, ChannelMask mask, Row& row )
{
row.range( 0, width() );
for( int z = 0; z < 4; z++ ) {
// A user implemented read function
custom_read_channel( y, z, row.writable( z ) );
}
}
This code represents a basic DD::Image::Reader::engine() implementation, starts by setting the range of the row to the full width, and then reads all of the channels into the data row.
These are optiomal methods that may be implemented to provide extra functionality and information to your Reader.
| Function | Description |
|
This should be overriden to have frame specific metadata |
|
A function that can be used to fetch metadata |
|
Flag indicating whether it supports stereo or not |
|
Flag indicating whether this file is stereo or not |
|
Flag indicating whether or not this file is a video sequence |
FileReader is a class derived from Reader, it performs very much like its base class and a basic FileReader implementation must also implement the same basic interfaces ( see The Reader Base Class ). What makes DD::Image::FileReader interesting however is the added functionality it can provide, while creating your reader directly from DD::Image::Reader can entail having to also implement file manipulation functions ( this can ofcourse be desirable for specific systems ), the FileReader is a suitable solution for most implementation by providing functions to access and manipulate file data directly.
So in order to use this additional functionality we simply need to inherit from our custom reader from DD::Image::FileReader.:
class MyReader : public DD::Image::FileReader
{
...
};
Now our reader can use the associated file handling helper functions associated with FileReader.
The extended methods that are provided by the FileReader for extra functionality.
| Function | Description |
|
Locks the data between the given parameters |
|
Locks the data between the given parameters |
|
Locks the data between the given parameters |
|
Accesses the byte at the given offset |
|
Accesses the byte at the given offset |
|
Unlocks a previous locked buffer |
|
Reads directly from the file into memory |
|
Reads directly from the file into memory |
|
Reads directly from the file into memory |
Adding Knobs to readers requires an extra step, because Readers do not behave like normal Ops there is a special object that can be initialized and registered with the Reader to give it the same capabilities as a regular op to create knobs. The two main steps to achieve this is to derive a custom object of type DD::Image::ReaderFormat the ReaderFormat is then capable of implementing the knobs() and knob_changed() interfaces, the second step is creating a new object of the right type within your custom reader and making sure the correct reader format is initialized.
The following code does just that
class MyFileFormat : public DD::Image::ReaderFormat
{
// implementation of the knobs call
void knobs(Knob_Callback c)
}
class MyReader : public DD::Image::FileReader
{
static const Description d;
}
static Reader* build( Read* r, int fd, const unsigned char* b, int n )
{
return new MyReader( r, fd, b, n );
}
static bool test( int fd, const unsigned char* block, int n )
{
return ( block[0] == MY_MAGIC_BYTE );
}
static ReaderFormat* buildformat(Read* iop)
{
return new MyFileFormat();
}
const Reader::Description MyReader::d( "myFile\0", "my file format", build, test, buildFormat );
The previous code now extends our reader to provide a ReaderFormat that can implement and extend the knobs collection for the MyReader object.
Metadata is accessed using the same pattern as the decoding, the DD::Image::fetchMetaData( const char* key ) can be implemented to return an object of type DD::Image::MetaData::Bundle.
class MyReader : public DD::Image::FileReader
{
...
DD::Image::MetaData::Bundle& fetchMetaData( const char* key )
{
return _metaData;
}
DD::Image::MetaData::Bundle _metaData;
}
In this example the _metaData object is returned when requested, this meta data is already pre-built somewhere else during object/reader construction.
As it was previously mentioned ( see The Reader Base Class ) when a file is read NUKE tries to identify which reader to use based on the registered valid extensions for each individual reader, a second mechanism also exists to safeguard against misnamed files, the DD::Image::Reader::(Tester*)( int fd, const unsigned char* block, int n ):
#define MY_MAGIC_BYTE 0x000000001
static bool test( int fd, const unsigned char* block, int n )
{
return ( block[0] == MY_MAGIC_BYTE );
}
The previous code is a simple test which will check the first byte of that and ensure it matches the defined MY_MAGIC_BYTE, this is ofcourse a very simple example of how to set it up, a truly usable implementation would check for a more complex bit pattern in order to ensure the format matches.
Metadata in NUKE is handled throught the DD::Image::MetaData::Bundle object, this abstract object provides all of the functionality needed to write and read data of integral types ( int, float, string... ).
There are two important things that need to be taken into account when handling MetaData, as these can be split into two categories, one is the NUKE recognized and defined metadata this is the case for time codes, where a specific key ( DD::Image::MetaData::TIMECODE ) should be used to store this value on a specific format ( you can learn more about keys here ). So when NUKE wants to read or write a time code this will be the key it will use.
The second type of metadata is the custom type, these will be recognized internally as simple key-value pairs and should be handled by on a case by case basis by the user.
For example if you decide to use a key named “somenamespace/timecode” instead of DD::Image::MetaData::TIMECODE to store your time codes, then you are responsible for maintaining and ensuring that data is accessed and written to the file where applicable.
Whenever possible it is always advisable to use the internal key descriptions so that the metadata can take advantage of any specific NUKE systems that use it.
A writing a custom writer is an omologous part of writing a Reader, most of the time when you spend the time to write a custom Reader you will probably also want to have a Writer capable and bringing your new/modified work back onto your custom file format. To achieve this in the same way that we have developed a Reader ( see The Reader Base Class ) we can develop a writer by inheriting from the DD::Image::Writer , this ofcourse does not mean that you need to have a writer for your custom reader and vice-versa.
In order to have a basic Writer much like the Reader there are a set of interfaces that need to be implemented that provide the basic functionality and description of this new writer so that it can be managed by NUKE.
The first step to achieve this is to inherit the class from our DD::Image::Writer and make sure we implement a DD::Image::Description for the Writer:
class MyWriter : public DD::Image::Writer
{
static const Description d;
...
};
static Writer* build()
{
return new MyWriter();
}
const Writer::Description MyWriter::d( "myFile\0", "my file type", build );
This code will setup a basic writer, we have created a new writer named MyWriter that derives from DD::Image::Writer within this writer we added a description, in our definition we can then declare and define a build function, this will be the factory method that will actually create the MyWriter object. The description is then populated with all of this relevant information, the writer will be triggered for files of extension .myFile unlike the reader there is no test function needed, since a writer should be able to write from NUKE’s internal format to any custom output data format.
In the same way as the Reader, we can extend the file extensions that will be associated with this writer by simply extending the file types ensure each element is separated by a NULL terminator. The writer does not suffer from misnaming problems since we can just choose another writer regardless of the extension we set for the file, the auto detect is simply for convenience.
In order to actually do any writing there are a few more intricate interfaces that we must implement, the main one being DD::Image::Writer::execute() this function should request the data from input0() into the chosen file, the file name can be retrieved using DD::Image::Writer::filename() and the frame using DD::Image::Writer::frame():
void MyWriter::execute()
{
// example implementation of execute
}
This code will request the data from Input0() and attempt to write the data into the my file type format into filename(). Any errors can be reported on to DD::Image::Writer::error().
This is a list of more advanced interfaces that you may want to implement for your custom writer and their description.
| Function | Description |
|
Flag indicating whether it is a movie or not |
|
Should be overriden if the writer needs to perform any operations at finish |
|
Should be overriden to create writer knobs |
|
Should be overriden to handle knob changed events |
|
Specifies the default LUT for this writer |
|
Flag indicating whether the LUT is default or not |
|
Flag indicating whether split inputs are present |
|
Lets the writer override the output context |
The FileWriter is a writer base class which extends the current DD::Image::Writer functionality by providing basic file handling functionality, in any case where the basic file handling functionality is needed this should be the class to derive for the implementation of the new writer as follows:
class MyWriter : public DD::Image::FileWriter
{
static const Description d;
...
};
By implementing the writer this way we are able to take advantage of the extended interface and file manipulation functions, unless there is a compeling reason or you are using some third party file manipulation, the FileWriter should be used as the base class.
| Function | Description |
|
Opens the output file |
|
Closes the output file |
|
Writes n bytes to the file. |
|
Writes n bytes given the offset. |
|
Seek the file |
|
Returns where the pointer currently is on the file |
|
Returns the current temporary file name |
In order to access metadata to be written into your file, the Writer itself must evaluate the metadata from an input Iop, and call the corresponding fetchMetaData() , this data can then be used to modify or append to the file itself.
In order to effectively create Readers and Writers for use with NUKE, it is also essential to have an understanding of how NUKE handles colourspace, this section will describe how NUKE handles colourspacing and LUTs in the context of Reads, Writes and during compositing.
While the intent of this section is not to explain the functionality associated with the viewer and compositing, it is essential to understanding NUKE’s handling of colourspace. NUKE assumes that any images to be displayed in the viewer are linearly colourspaced, this not only facilitates handling and calculations but also ensures a standard write format accross all supported file types, the viewer itself will apply a desired LUT to ensure that a preview is reliabily similar to the real output.
As mentioned previously, NUKE expects all of its formats to be read into linear space, what that means is that the user is responsible for ensuring any colourspace conversions happen at the time of reading, once the reading is finished, NUKE will assume the colourspace is linear and will apply any operations under that assumption:
void MyReader::engine( int y, int x, int r, ChannelMask mask, Row& row )
{
row.range( 0, width() );
for( int z = 0; z < 4; z++ ) {
// A user implemented read function
custom_read_channel( y, z, row.writable( z ) );
}
}
In this example the custom_read_channel when writing the channel z would have to also perform the appropriate calculations to ensure the data is in linear colourspace.
Writting data as an omologous operation to a read also has to make certain assumptions, all data is assumed to be in linear space when the reader is called, and any conversions need to be made on write, the difference here is that the colourspace output can be implicit or explicit, to achieve this there is a particular interface ( DD::Image::Writer::defaultLUT() ) that can be implemented for the Write that allows each write to specify a default ( implicit ) colour space. It is important to note that all other registered colourspaces are also available through the NUKE UI.
LUT* MyWriter::defaultLUT() const
{
return LUT::getLut(LUT::FLOAT);
}
This code will set the default LUT for the MyWriter writer to be FLOAT, as to when a file with a supported extension is detected and the default colourspace setting is selected this will be the output colourspace for the write.