A cluster configuration for DirectViz is composed of a master node where the Open Inventor application is started, and several slave nodes running solely a ray-tracing module called OpenRTClient, which computes parts of the complete image displayed by the master.
Typical configurations are based on commodity parts using off-the-shelf dual processor PCs linked by standard Ethernet to a central switch.
The master node controls the display and runs the Open Inventor application that specifies the scene to be rendered. DirectViz then transfers the scene to the slave machines through OpenRTRT as a set of geometry objects, instances, textures, shaders information, etc. suitable for the ray-tracing engine. If the DirectViz scene cache file is used, the cache file content is read directly on each slave from disk, instead of being sent through the network.
The master subdivides the image to be rendered in the Open Inventor viewer into tiles that are assigned to slaves for rendering. From the OpenRTRT architecture standpoint, the master displaying the rendered image is actually considered as a server answering requests from the clients (so-called OpenRTClient) asking what are the next tiles they have to compute on the slave machines. A dynamic load balancing scheme is used, more tiles are used than available processors and tiles are assigned to clients on demand.
![[Important]](../../images/important.jpg) | |
Because of mechanisms needed in cluster mode to optimize use of CPU and network resources, there is a latency of one frame on the master display when running in cluster mode, i.e., the application is displaying frame t-2 when requesting image t. |
Once tiles are computed by slave/clients, the resulting pixels are then sent back to the master/server, which generally requires a Gigabit-Ethernet connection (see below about the network).
Thanks to the sophisticated load balancing and optimizations of the OpenRTRT ray-tracing engine, the performance is quite linear according to the number of rays shot. Therefore you can expand the hardware as needed proportionally to the desired frame rate and/or resolution.
Note that the master doesn’t compute any portion of the image, it simply displays the computed image. Therefore setting a cluster configuration with only two machines makes sense only if the slave is significantly more powerful than the master.
The slaves do not need any graphics card as the ray-tracing engine only uses the CPU. On the master, OpenGL support is, of course, required for the Open Inventor application and used by DirectViz to display the computed image.
In order to use DirectViz in cluster configuration, the SoPreferences SoPreferences SoPreferences DIRECTVIZ_CLUSTER parameter must be set to the desired number of slaves. If set to 0 (the default), DirectViz will run in stand-alone mode.
Though it is possible to run DirectViz on a heterogeneous cluster setup, it is recommended to use a homogeneous setup. It is also recommended using the same file system and directory structure on the master and all slaves (typically NFS-mounted or shared directory).
![[Warning]](../../images/warning.jpg) | |
|
To use DirectViz in a cluster configuration, one should follow these steps:
Start the master Open Inventor application – when DirectViz initializes with cluster mode set in SoPreferences SoPreferences SoPreferences parameters, it stops and waits for the specified number of the clients to connect before continuing.
Start each client with the command line:
OpenRTClient.exe <master IP address> <port number>(Windows)or
OpenRTClient <master IP address> <port number>(Linux)where the connection port must be 9001.
Instead of starting each client separately, the OpenRTRT configuration file can be used, if all clients are accessible with a remote command like ssh.
The environment variable OPENRT_CONFIG_FILE must point to the configuration file on disk.
The auto start clients mode is turned on with the line :
AutoStartClients = true
For each client, the following information must be provided :
hostName or IP adress
Path to OpenRTClient executable on client machine
number of threads Specifies the number of threads to use for client rendering
number of rendering processes Specifies the number of processes launched on the client machine. If more than one process is specified, OpenRTRT will launch the number of specified threads for each process.
auto start mode (true or false) Specifies if this client is used for auto starting
connection command Command used for the connection from the master machine to the client machine
The syntax of the provided information must follow the rules :
ClientInfo = <hostname>, <OpenRTClient path>, <number of threads>, < number of processes>, autostart mode>, <connection command>
When a multi processor machine is used as a slave in a cluster configuration, the number of threads of the machine is set via the SoPreferences SoPreferences SoPreferences DIRECTVIZ_NUM_THREADS_SLAVE on the master node. This number must follow the same rules given above for stand-alone mode. Note that the number of threads applies to every slave – it is not possible to configure independently the number of threads for each slave.
All slaves must communicate with the master via the network. A Gigabit connection or better is recommended to provide the bandwidth required for high resolutions and high frame rates. For example, dedicated 100Mbit Ethernet can support roughly 3Mpixels/s (10fps@640x480), Gigabit roughly 12Mpixels/s.
For best performance, a fully switched network ensuring appropriate bandwidth is recommended for connecting all the slaves (e.g., 100Mbits) and the master (high-bandwidth uplink e.g., Gigabit). High latencies (e.g., because of several routers between slaves and master) would also decrease performance.
If you use many slaves, and the master uplink is starting to become a bottleneck, it may be very helpful to equip the server machine with multiple network cards, and use trunking to combine them to one virtual interface with higher bandwidth. Trunking is a technique supported by newer Linux kernels in which multiple physical network adapters can be configured to appear as one virtual adapter with the combined bandwidth of the individual physical cards. Ask your system administrator for assistance in setting this up.