Small Distributed Application
- Basic Steps
- Components
- Using the Cluster Daemon
- Using UDP
The first distributed application from this tutorial needs three components:
- A render server
- A scrtipt server
- The master component
The render server is a slightly modified glutCyberle VRML-browser from the first tutorial. The script server is a VRML-browser without any render capabilites and runs only the simulation of the world model. The OAWMaster component is the component that triggers the whole distributed application and intialize the other components. All these components are very general and therfore can also be found in the OAW-tools and OAW-projects folder. The whole sourcefile needed are also find in the OAW-tutorials/simple_pds folder.
To run this tutorial example:
To run the examples simply compile the sourcefiles to the needed executeables. Then execute from the commandline first "glutCyberleSrv" and "scriptSrv" and second "OAWMaster master.ini".
Here are the steps needed to transform the glutCyberle demo into a small renderserver:
First: Instanciate the SockEventBroker instead of the LocalEventBroker.
OAW::SockEventBrokerC* pEB = 0;
..
..
pEB = new OAW::SockEventBrokerC(params.ebname,params.port,params.netAddr.c_str());
..
Second: Register extern instanciated interfaces to the event broker.
pEB->AssignInterface("default_pointingdevice",pPointer);
pEB->AssignInterface("default_viewpointsensor",pViewpointSensor);
pEB->AssignInterface("sound_manager",&soundMgr);
All scripts from the VRML-File are assigned to the event broker as interfaces with their DEF name automatically.
Third: Load the VRML file from the url given by the event broker.
OAW::StringC initData = pEB->GetInitData();
world.LoadFromFile(params.wrlRoot+initData);
It is recommended to broadcast only the relative path of the VRML file by the master, and add it in the component to a wrl root path. Especially in heterogeneous cluster you may use different folders for the wrl files on each host.
Fourth:Give the event broker feedback about that the initialization is finished before starting the mainLoop.
pEB->InitializationReady();
glutMainLoop();
The ScriptSrv is similar to the glutCyberleSrv, but it does not contain any instatiation of objects that have to do with rendering or mouse event handling. But two points need to be noticed:
First:
Also there is no special font manager needed, we need to specify the global variable of the path to the fonts files. This is nesessary to provide the font geometry for intersection.
Second:
To make this script server also suitable for environments using headtracking, the script server instanciates also an interface for a headtracker, which is assigned to the user and is assigned to the EB. This is important to assure that also in this script server component billboards are rotated appropriatly, so that again the intersections can be calculated correctly.
The master component drives the distributed simulation loop. If you look into the source code below, you'll see that the functionality is already incapsulated into a class. Therefore the source of this component is very short.
int main(int argc, char** argv){
OAW::SockMasterC master;
try{
master.InitByFile(argv[1]);
master.BroadcastInitData();
}catch(OAW::ESockMasterC err){
OAW::msg("WERROR: %d",err.m_errorCode);
}
while(master.ProcessEvents(1000)){
}
return 0;
}
The ProcessEvents function drives the simulation loop until one component quits the simulation.
The master component is initialized by a file. This file has the following format:
<VRML file to be loaded>
--
<protocol = TCP,UDP>
<local ip address = default, aaa.bbb.ccc.ddd.> <port>
<broadcast address> <broadcast port base>
--
<host name> <host IP>
..
--
<component general name> <component instance name> <host name> <port> <arguments>
..
--
<interface name> <component name>
..
There are five blocks in the ini file that are separeted by "--". The first block is only one line and contains a relative path to the VRML file to be loaded for the distributed simulation. The second block are three lines. First you can specify by the key words TCP or UDP which transport protocol shall be used. In the case of UDP there are some things you need to pay attenion of. For more details see the last section of this tutorial. The second line contains the IP address of the ethernet card through which the master communicates. If instead of an IP address the keyword default is given the default ethernet card also used for default services of the operating system is used. In case of using UDP as a communication protocoll it is a must to use a dedicated network card on each host that are only connected among each other. The third line of the first block contains the broadcast address for a communication possibly through UDP and a port number, that is used as a base address. You have to be aware that in case of UDP communication the simulation will need the ports from port-n and port+n to be free, if n is the number of components of the distributed simulation.
The second block is a table of host names and its corresponding IPs. These IPs can be dedicated ethernet cards of this host or simply the name of the host.
The third block is a list of components participating the simulation. The meaning of the columns is as follows:
First column: Indicates the general name of the component used by the cluster daemon.
Second column: Contains name of the instance of the particular component.
Third column: Host names on which it shall run.
Fourth column: Port for the communication.
Fifth column: Optional parameters that are handed over to the component at startup through a cluster daemon. For details on using a cluster daemon see the next section.
The master.ini file of this example looks like this:
OAW-tutorials/maltepointer/pointerTest.wrl
--
TCP
default 4450
128.7.90.255 4523
--
myhost myhost
--
GSrv GSrv1 myhost 3331 "-name GSrv1 -port 3331"
ScriptSrv ScriptSrv myhost 2999 "-name ScriptSrv -port 2999"
--
system_time GSrv1
default_headtracker GSrv1
default_viewpointsensor GSrv1
default_pointingdevice GSrv1
WIN_EVENTS_SCRIPT GSrv1
PROTO::MaltePointer::POINTER::POINTER_SCRIPT ScriptSrv
All script nodes are defined as interfaces under their DEF names of the VRML-file, like you see in the case of the WIN_EVENTS_SCRIPT. The WIN_EVENTS_SCRIPT must be active on the GSrv1 component, because the ScriptSrv component has no window, and therefore can not contribute mouse events to the simulation. But the script node of the pointer is active in the ScriptSrv component. From this example you see the syntax of specifying a script in an instance of an PROTO. The syntax is:
PROTO::<proto name>::<DEF name of proto instance>::<DEF name of the script in the proto>
In practice it is very unconfortable to start all components of a distributed simulation on all hosts by hand. To facilitate the initialization of a distributed simulation there are the OAWClusterDaemon and the OAWCompStarter in the OAW-tools section. If you compiled these projects with the makefiles of the distribution, the executeables of these files should be already in the tutorial section. The OAWClusterDaemon is initialized by a file containing a table with the general name of the component, the name of the executable an the path to the executable. For the components of this tutorial the oawrepository.txt file therfore looks like:
GSrv myGSrv.exe ./
ScriptSrv myScriptSrv.exe ./
Now you may simply start the OAWClusterDaemon from the commandline. It will display what components it has read from the repository. Now you simply need to start "OAWCompStarter master.ini". The CompStarter contains also a masterobject. First it will read the master.ini file and start the components on all hosts by connecting to the cluster daemon. After this the simulation loop will be initialized an be driven by the integrated master object as usal. But to start the single components and to use the OAWMaster component is still useful for debugging.
The default example uses TCP for the communcation between the components. TCP uses point to point connection between the components. Therefore, if there are many components holding a whole world model, the data one component produces has to be send from the component successivly to all other components. Which means redundancy in the data transmitted throu the network and unnecessary latency from the higher number of system function calls. The usage of the TCP protocoll in OAW is optimized in respect to the number of system calls per frame in the way, that the master first collects all data from all components, pack them in dedicated data packages for each component (so that any component receives only data of the passive interfaces it holds) and send it back to all components. In this way TCP is sufficient for setups from three to four components holding a world model and some sensor server models.
But all this cannot compete by using UDP broadcast messages. The problem by using UDP is, that the delivery of each package is not reliable. The OAW middleware comes over this by require a dedicated network through which only the OAW components of one simulation exchange broadcast messages. This way collisions can be avoided by using some kind of token. For details see winkelholz02. Anyhow to use UDP first you need to have an additional network card in each computer of your cluster that are linked together by a isolated hub. (Using a switch we experienced problems. We think that a switch buffers the broadcast messages and therefore it may come to collisions. But this may depend on the switch.) Anyhow to use the UDP as a transport protocol you simply need to replace in the first line of the ini file TCP by UDP. Further you should use the dedicated network card and not the default card the operating system is using for its network services.