Preview Socket¶

‘’Preview Socket’’ is a protocol for extracting real-time images from ‘’modo’’s’ preview renderer. Preview Socket clients are separate applications that communicate with modo over a TCP/IP network connection or through pipes and implements the Preview Socket protocol.

While the name implies a TCP (socket) connection, a named pipe variant is also supported. For the purposes of this article, “Preview Socket” refers to both sockets and pipes unless otherwise indicated (usually by through the word “TCP” or “pipe’).

Note that Preview Socket is only used for obtaining preview renders and setting up said renderer. Changes to the actual scene must be done through commands, either through modo itself or through Telnet.

Server Commands ¶

High-level control of the Preview Socket server is done through commands.

previewSocket.listen is used to start or stop listening for TCP connections on the given port. In other words, this starts and stops the Preview Socket server. Any number of clients can connect to the server, with each getting their own preview renderer.
preview.suspend pauses all preview instances, including sockets and pipes. When used with its identifier argument, this can be used to maintain sync with between applications while other commands are being executed.
previewPipe.start opens the named pipe. Unlike the TCP based server, only one client can be connected to the pipe at a time.
previewPipe.stop closes the named pipe.

 previewSocket.listen port:12378 open:true

Once the port is open, clients can connect to setup a preview renderer and receive renderings.

Protocol ¶

These codes are used to identify any data sent to or from Preview Socket, and are used to prefix the packet.

Preview Socket uses a request/response protocol to communicate with clients. For the most part, this means that the client sends request packets to the Preview Socket server. Preview Socket then sends back acknowledgements or errors. Every message sent by the client will have a matching response from Preview Socket.

        #define LXfPREVIEWMSG_REQ                               0x10000000
        #define LXfPREVIEWMSG_ACK                               0x20000000

        #define LXiPREVIEWMSG_ACK_ERROR                         (LXfPREVIEWMSG_ACK | 0x01000000)

        // Initial Setup
        #define LXiPREVIEWMSG_REQ_SET_PROTOCOL                  (LXfPREVIEWMSG_REQ | 0x00001000)
        #define LXiPREVIEWMSG_ACK_PROTOCOL_SET                  (LXfPREVIEWMSG_ACK | 0x00001000)

        // Image Property Setup
        #define LXiPREVIEWMSG_REQ_SET_RES                       (LXfPREVIEWMSG_REQ | 0x00002000)
        #define LXiPREVIEWMSG_ACK_RES_SET                       (LXfPREVIEWMSG_ACK | 0x00002000)

        #define LXiPREVIEWMSG_REQ_SET_FORMAT                    (LXfPREVIEWMSG_REQ | 0x00002001)
        #define LXiPREVIEWMSG_ACK_FORMAT_SET                    (LXfPREVIEWMSG_ACK | 0x00002001)

        #define LXiPREVIEWMSG_REQ_SET_STEREO                    (LXfPREVIEWMSG_REQ | 0x00002002)
        #define LXiPREVIEWMSG_ACK_STEREO_SET                    (LXfPREVIEWMSG_ACK | 0x00002002)

        #define LXiPREVIEWMSG_REQ_SET_SAMPLES                   (LXfPREVIEWMSG_REQ | 0x00002003)
        #define LXiPREVIEWMSG_ACK_SAMPLES_SET                   (LXfPREVIEWMSG_ACK | 0x00002003)

        #define LXiPREVIEWMSG_REQ_SET_DRAFT                     (LXfPREVIEWMSG_REQ | 0x00002004)
        #define LXiPREVIEWMSG_ACK_DRAFT_SET                     (LXfPREVIEWMSG_ACK | 0x00002004)

        #define LXiPREVIEWMSG_REQ_SET_FINAL                     (LXfPREVIEWMSG_REQ | 0x00002005)
        #define LXiPREVIEWMSG_ACK_FINAL_SET                     (LXfPREVIEWMSG_ACK | 0x00002005)

        #define LXiPREVIEWMSG_REQ_SET_EXTENDED                  (LXfPREVIEWMSG_REQ | 0x00002006)
        #define LXiPREVIEWMSG_ACK_EXTENDED_SET                  (LXfPREVIEWMSG_ACK | 0x00002006)

        #define LXiPREVIEWMSG_REQ_SET_EFFECT                    (LXfPREVIEWMSG_REQ | 0x00002007)
        #define LXiPREVIEWMSG_ACK_EFFECT_SET                    (LXfPREVIEWMSG_ACK | 0x00002007)

        #define LXiPREVIEWMSG_REQ_SET_MOUSEPOS                  (LXfPREVIEWMSG_REQ | 0x00002008)
        #define LXiPREVIEWMSG_ACK_MOUSEPOS_SET                  (LXfPREVIEWMSG_ACK | 0x00002008)

        #define LXiPREVIEWMSG_REQ_SET_CAMERA                    (LXfPREVIEWMSG_REQ | 0x00002009)
        #define LXiPREVIEWMSG_ACK_CAMERA_SET                    (LXfPREVIEWMSG_ACK | 0x00002009)

        // Frame Requests
        #define LXiPREVIEWMSG_REQ_SEND_FULL_FRAME               (LXfPREVIEWMSG_REQ | 0x00003000)
        #define LXiPREVIEWMSG_ACK_SEND_FULL_FRAME               (LXfPREVIEWMSG_ACK | 0x00003000)

        #define LXiPREVIEWMSG_PREVIEW_UNAVAILABLE                                    0x00000001
        #define LXiPREVIEWMSG_PREVIEW_RGBAFP                                         0x00000002
        #define LXiPREVIEWMSG_PREVIEW_COMPLETE                                       0x00000100

        #define LXiPREVIEWMSG_ACK_SEND_FULL_FRAME_RGB24         (LXiPREVIEWMSG_ACK_SEND_FULL_FRAME)
        #define LXiPREVIEWMSG_ACK_SEND_FULL_FRAME_RGBAFP        (LXiPREVIEWMSG_ACK_SEND_FULL_FRAME | LXiPREVIEWMSG_PREVIEW_RGBAFP)
        #define LXiPREVIEWMSG_ACK_FULL_FRAME_UNAVAILABLE        (LXiPREVIEWMSG_ACK_SEND_FULL_FRAME | LXiPREVIEWMSG_PREVIEW_UNAVAILABLE)
        #define LXiPREVIEWMSG_ACK_FULL_FRAME_DONE               (LXiPREVIEWMSG_ACK_FULL_FRAME_UNAVAILABLE | LXiPREVIEWMSG_PREVIEW_COMPLETE)

        // Preview image ID
        #define LXiPREVIEWMSG_REQ_GET_ID                        (LXfPREVIEWMSG_REQ | 0x00004000)
        #define LXiPREVIEWMSG_ACK_ID_GET                        (LXfPREVIEWMSG_ACK | 0x00004000)

In general, REQ_ packets are sent by the client to the server, ACK_ packets are sent from the server to the client in response.

The datatypes used for the packet are as follows. Note that when packing multi-byte datatypes, Preview Socket uses Intel byte order (little endian), as opposed to the Motorola byte order (big endian) more commonly used more with networking.

{| class=”wikitable” {{Template:Center Table}} ! Shorthand ! Description ! C Datatype |- |U1 |One byte unsigned integer |unsigned char |- |U1[] |Array of one byte unsigned integers |unsigned char [] |- |U4 |Four byte unsigned integer |unsigned int |- |Float |Four byte floating point number |float |- |Float[] |Array of four byte floating point numbers |float [] |- |S |Array of bytes representing a string |char [] |}

Errors ¶

ACK_ERROR ¶

‘’’Server to Client’’’: The last request resulted in an error. The request was not completed. This may be sent as a response to any request made by the client in place of an acknowledgement. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_ACK_ERROR |- |U4 |LxResult code describing the error |- |U4 |Length of an optional string describing the error (may be a human-readable or internal string); may be 0 if no string is sent. |- |S |If the previous U4 is non-zero, this is an array of bytes represnting a C string describing the error message. Note that the the string is NOT ‘0’ terminated. |}

Initialization and Setup ¶

Before doing anything else after connection to the server, the client must indicate which version of the protocol to use. After that, the other the other setup requests can be made at any time to change the render, although some must be called before obtaining the first frame.

REQ_SET_PROTOCOL ¶

‘’’Client to Server’’’: Request to set the protocol used to communicate with Preview Socekt. Currently only the default protocol 0 is supported, and attempts to set any other protocol will result in an ACK_ERROR. This message must be sent before any other messages are sent. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_REQ_SET_PROTOCOL |- |U1 |Protocol used for communication. Currently only protocol 0 is supported. |}

ACK_PROTOCOL_SET ¶

‘’’Server to Client’’’: Acknowledgement to REQ_SET_PROTOCOL stating that the requested protocol will be used for communication between Preview Socekt and the client. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_ACK_PROTOCOL_SET |}

REQ_SET_RES ¶

‘’’Client to Server’’’: Request to the Preview Socket to set the frame resolution. The server will begin rendering when the resolution changes. The resolution must be set before frames can be sent. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_REQ_SET_RES |- |U4 |width in pixels |- |U4 |height in pixels |}

ACK_RES_SET ¶

‘’’Server to Client’’’: Acknowledgement to a REQ_SET_RES event stating that the resolution has been set. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_ACK_RES_SET |}

REQ_SET_FORMAT ¶

‘’’Client to Server’’’: Request to the Preview Socket to set the frame’s pixel format. The server will begin rendering when the format changes. If the format is not set, LXiIMD_BYTE is assumed. Note that the only options are byte RGB and float RGBA. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_REQ_SET_FORMAT |- |U4 |Format; LXiIMD_BYTE for 24 bit RGB; LXiIMD_FLOAT for floating point RGBA. These defines can be found in the image (lx_image.hpp)#Pixel Formats. |}

ACK_FORMAT_SET ¶

‘’’Server to Client’’’: Acknowledgement to a REQ_SET_FORMAT event stating that the format has been set. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_ACK_FORMAT_SET |}

REQ_SET_STEREO ¶

‘’’Client to Server’’’: Request to the Preview Socket to set the stereoscopic mode, which causes it to render frames for the left eye, the right eye or monocularly. By simultaneously connecting to two Preview Sockets set to different eyes, you can render a stereo pair. Preview Socket will begin rendering when the mode changes. If no stereoscopic mode is set, monocular mode is assumed. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_REQ_SET_STEREO |- |U4 |Mode; 0 for monocular, 1 for the left eye and 2 for the right eye |}

ACK_STEREO_SET ¶

‘’’Server to Client’’’: Acknowledgement to a REQ_SET_STEREO event stating that the stereo mode has been set. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_ACK_STEREO_SET |}

REQ_SET_SAMPLES ¶

‘’’Client to Server’’’: Request to Preview Socekt to set the maximum number of samples. Only applies when preview quality is DRAFT or EXTENDED. If the quality is FINAL, the settings from the scene are used. Preview Socket will begin rendering when the number of samples changes. If not explicitly set, the current scene values are used. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_REQ_SET_SAMPLES |- |U4 |Samples; maximum number of samples to use |}

ACK_SAMPLES_SET ¶

‘’’Server to Client’’’: Acknowledgement to a REQ_SET_SAMPLES event stating that the number of samples have been set. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_ACK_SAMPLES_SET |}

REQ_SET_DRAFT ¶

‘’’Client to Server’’’: Request that Preview Socket use draft quality with an optional number of antialiasing samples and quality level (fraction of final quality). Preview Socket will begin rendering when the quailty changes. If the samples or quality level are set to 0, the current preferences are used. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_REQ_SET_DRAFT |- |U4 |Samples; maximum number of samples to use |- |Float |Quality; fraction of final render (0.0 to 1.0) |}

ACK_DRAFT_SET ¶

‘’’Server to Client’’’: Acknowledgement to a REQ_SET_DRAFT event stating that the number of samples have been set. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_ACK_DRAFT_SET |}

REQ_SET_FINAL ¶

‘’’Client to Server’’’: Request final render quality Preview. Preview will begin rendering when the quailty changes. The current scene and rendering preferences are used. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_REQ_SET_FINAL |}

ACK_FINAL_SET ¶

‘’’Server to Client’’’: Acknowledgement to a REQ_SET_FINAL event stating that the number of samples have been set. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_ACK_FINAL_SET |}

REQ_SET_EXTENDED ¶

‘’’Client to Server’’’: Request extended quality for the Preview Socket render, using an optional number of antialiasing samples. Preview Socket will begin rendering when the quality changes. Rendering settings are the same as used for final rendering quality, except using a potentially greater number of antialiasing samples. If the samples are set to 0, the current preferences are used. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_REQ_SET_EXTENDED |- |U4 |Samples; maximum number of samples to use |}

ACK_EXTENDED_SET ¶

‘’’Server to Client’’’: Acknowledgement to a REQ_SET_EXTENDED event stating that the number of samples have been set. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_ACK_EXTENDED_SET |}

REQ_SET_EFFECT ¶

‘’’Client to Server’’’: Request to Preview Socket to set the output channel or effect. Preview will begin rendering when the effect changes. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_REQ_SET_EXTENDED |- |U4 |length of effect string (‘’’without’’’ a ‘0’) |- |S |Effect string (‘’’not’’’ terminated with ‘0’) |}

ACK_EFFECT_SET ¶

‘’’Server to Client’’’: Acknowledgement to a REQ_SET_EFFECT event stating that the effect has been set. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_ACK_EFFECT_SET |}

REQ_SET_MOUSEPOS ¶

‘’’Client to Server’’’: Request to Preview Socket to focus the preview on the mouse position – or rather, the given X/Y position within the rendering frame as defined by a previous call to ‘’REQ_SET_RES’’. Preview will continue rendering, but reorder the buckets to concentrate on the position specified for the next update only, You must send continue to send repeated mouse position requests for continued refinement. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_REQ_SET_MOUSEPOS |- |U4 |X position of mouse |- |U4 |Y position of mouse |}

ACK_MOUSEPOS_SET ¶

‘’’Server to Client’’’: Acknowledgement to a REQ_SET_MOUSEPOS event stating that the mouse position has been set. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_ACK_MOUSEPOS_SET |}

REQ_SET_CAMERA ¶

‘’’Client to Server’’’: Request to set camera position, direction & zoom for preview. Preview Socket will restart rendering with the new camera state. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_REQ_SET_CAMERA |- |Float[] |XYZ position of camera |- |Float[] |XYZ direction of camera |- |Float |Zoom factor |}

ACK_CAMERA_SET ¶

‘’’Server to Client’’’: Acknowledgement to a REQ_SET_CAMERA event stating that the mouse position has been set. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_ACK_CAMERA_SET |}

Requesting Rendered Frames ¶

To request a rendered frame, you send the LXiPREVIEWMSG_REQ_SEND_FULL_FRAME message. The client will receive the currently waiting RGB or RGBAFP if one is available. Otherwise it will get ACK_FULL_FRAME_UNAVAILABLE if there is not currently a frame ready to send, or LXiPREVIEWMSG_ACK_FULL_FRAME_DONE if rendering has finished, thus indicating that the most recently received frame represents the final render.

REQ_SEND_FULL_FRAME ¶

‘’’Client to Server’’’: Request the most recently rendered image as single large packet. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_REQ_SEND_FULL_FRAME |}

ACK_SEND_FULL_FRAME_RGB24 ¶

‘’’Server to Client’’’: Send the most recently rendered frame to the client as a response to REQ_SEND_FULL_FRAME. The frame is sent as a single large packet containing a byte-format RGB image. The RGB part of the packet is ‘’’width * height * 3’’’ bytes long. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_ACK_SEND_FULL_FRAME_RGB24 (synonym: LXiPREVIEWMSG_ACK_SEND_FULL_FRAME) |U4 |Image width |U4 |Image height |U1[] |Byte array of RGB color triples |}

ACK_SEND_FULL_FRAME_RGBAFP ¶

‘’’Server to Client’’’: Send the most recently rendered floating point frame to the client as a response to REQ_SEND_FULL_FRAME, when the format is LXiIMD_FLOAT. The frame is sent as a single large packet. The RBGA part of the packet is ‘’’width * height * 16 bytes long’’’ (4 floats per pixel). {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_ACK_SEND_FULL_FRAME_RGBAFP |U4 |Image width |U4 |Image height |Float[] |Float array of RGBA colors |}

ACK_FULL_FRAME_UNAVAILABLE ¶

‘’’Server to Client’’’: Sent as an alternate response to REQ_SEND_FULL_FRAME_RGB24 and REQ_SEND_FULL_FRAME_RGBAFP when no frame is available to send. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_ACK_FULL_FRAME_UNAVAILABLE |}

ACK_FULL_FRAME_DONE ¶

‘’’Client to Server’’’: Sent as an alternate request to REQ_SEND_FULL_FRAME_RGB24 and REQ_SEND_FULL_FRAME_RGBAFP when the render is complete and no more rendering is required. The last frame sent to the client represents the most recent render, and no further renders will be sent until the scene or Preview Socket state are changed. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 | LXiPREVIEWMSG_ACK_FULL_FRAME_DONE |}

Render ID ¶

The render ID is a unique identifier assigned to each specific requested frame. This can be used to Synchronizing

REQ_GET_ID ¶

‘’’Client to Server’’’: Request the server ID assigned to the last completed render. This message is only valid after REQ_SEND_FULL_FRAME has returned a state, including FULL_FRAME_DONE. This ID can be used for Synchronizing. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_REQ_GET_ID |}

ACK_ID_GET ¶

‘’’Server to Client’’’: Acknowledgement to a REQ_GET_ID event, returning application ID assigned to the last completed render. If the last REQ_SEND_FULL_FRAME did not return a state (which may include FULL_FRAME_DONE), this value will be 0. {| class=”wikitable” {{Template:Center Table}} width=”90%” ! Data ! Description |- |U4 |LXiPREVIEWMSG_ACK_ID_GET |- |U4 |Render ID |}

Basic Loop ¶

A client implements the protocol by first connecting to the server, and then sending the LXiPREVIEWMSG_REQ_SET_PROTOCOL packet to indicate which version of the protocol it wants to communicate over. The current protocol version is 0. An acknowledgment will be returned indicating that the protocol was set.

Once the protocol version has been established, the client can send other setup-oriented requests to change the resolution, pixel format, samples, quality, stereo eye, camera position, and other options. These can be set at any time, although some must be set at least once before rendering can begin. After sending any request, the client should wait for an acknowledgment to ensure that the request was accepted and that there were no errors.

Once setup is complete, the client can begin asking for frames. Preview Socket must be polled for the latest frame; it will not simply send frames to the client without being asked first. Sending LXiPREVIEWMSG_REQ_SEND_FULL_FRAME to the server will return either LXiPREVIEWMSG_ACK_SEND_FULL_FRAME_RGB24 or LXiPREVIEWMSG_ACK_SEND_FULL_FRAME_RGBAFP depending on the pixel format previously defined along with the most recent render. If there is no render, the response will be LXiPREVIEWMSG_ACK_FULL_FRAME_UNAVAILABLE to indicate that no newer frame is available, or LXiPREVIEWMSG_ACK_FULL_FRAME_DONE to indicate that the last render received by the client represents the final render.

Note that if a packet header cannot be identified by Preview Socket, the connection will be closed. This is because there is no way for Preview Socket to know how far to seek ahead to find the next packet, so there’s really nothing else it can do.

Synchronizing ¶

Often a Preview Socket client will also use Telnet command over telnet to suspend all Preview Sockets.

 preview.suspend true

At this point there may already be a pending image waiting to be sent to the client. The client isn’t interested in this image, and wants to be sure that it gets only images after it has finished executing its commands. This can be done by using LXiPREVIEWMSG_REQ_GET_ID to request the unique identifier of the last image sent to the client. When all other commands have been sent over telnet, the client can resume rendering with preivew.suspend, but this time setting the suspend argument to false and providing the previously-obtained unique identifier.

 preview.suspend false identifier

The identifier is optional, in which case the client’s next request for an image may actually be a pending image from before Preview Socket was suspended. This behavior is useful if you just wanted to pause rendering to free up some CPU time, and then want it to continue afterward.

Headless ¶

When using Preview Socket with Headless’’ switch and a port number to start the telnet server. Chances are that you’ll be doing this anyway, since you’ll need be using telnet to communicate with modo to control manipulate the scene. If you do not launch the application with ‘’-telnet:’’, Preview Socket will fail to process input and appear to hang. This happens because ‘’modo’’’s main loop is blocked while it waits for the user to enter a new command in normal headless mode. Using the -telnet: switch causes the application to use an alternate input loop for processing network traffic, and thus allows Preview Socket to work properly.