This document describes the stereo mode capability of OpenDX.
The necessary functionality for stereo operation is divided into
two areas: manipulation of the windowing system to create stereo image
buffers, and manipulation of the camera to create left and right eye views.
Several versions of each are built in to OpenDX - see hwStereoSys.c and
hwStereoCams.c in src/exec/hwrender to see the code implementing the built-in
modes. In addition, you can implement your own modes by implementing several
routines described below, and then cause DX to load object modules containing
the implementations of these modes at run-time.
dx -edit TestStereo
When DX comes up, put it in exec-on-change, and you'll see a simple image of my favorite sample data set. In this example, interaction modes are specified by keystrokes in the view window. If you type 'z' you'll enter zoom mode; button down motion up and down the imge zoom in and out. 'r' gets you rotate mode, and 'p' gets you pan mode.
Similarly, stereo modes are selected with keystrokes in the image window. If you hit the '2' key you will select a simple side-by-side stereo mode; you might have to zoom the image out to see it. I don't recall what '3' gets you, but '4' gets you a squashed over/under stereo mode.
There's more discussion of all this below, including how to set it up
appropriately for your own hardware environment.
In order to support the widest possible range of applications, we
have implemented a structure whereby any number of different stereo modes
can be incorporated simultaneously, and selected at run-time. Thus, whether
your hardware requires separate left and right windows placed side-by-side
onto the display, or (as is true on our SGI) separate viewports in a single
window placed over-and-under, you can very simply install a very small
amount of custom code to tell DX exactly what to do. Furthermore, this
code also allows the user to specify the means whereby the hardware itself
is placed in stereo mode. This code can be either linked into the DX visualization
executive or loaded at run-time and consists of a table of functions that
specify different stereo modes. This table is accessed by index, where
the index corresponds to the particular stereo mode desired. When the selected
mode is -1 (the default), mono mode is chosen.
A default set of two stereo modes should, however, suit most needs. The first of these (index 0) implements stereo in side-by-side viewports in a single window (which is either passed into the DX Display modules from the SuperviseWindow module, or specified by the camera passed into Display, or which defaults to 640x480) and does not affect the hardware at all. This mode is quite useful for cross-eyed stereo. The second mode (index 1) implements what appears to be a standard SGI mode, in which a single window (which must be a full-screen window with no 'decorations' - eg. window borders and menus) is divided into separate over-and-under viewports, one filling the top 492 rows of pixels and the other, the bottom 492 rows. This mode allows you to specify the command used to place the hardware into stereo mode as environment variables. Thus, if you place the system into stereo mode by entering the command '/usr/gfx/setmon -n STR_RECT' at a command-line, by placing this command string into an environment variable named DX_INIT_STEREO_COMMAND and the corresponding command to revert to mono mode (on our SGI, '/usr/gfx/setmon -n 60HZ') into an environment variable named DX_EXIT_STEREO_COMMAND, DX will make the necessary calls via system() when the stereo mode 1 is entered or exitted.
I will include a sample a run-time loadable set of functions that illustrate
how you can supercede the default modes. If you have to do so, however,
I'd very much like to hear about it so that we can augment the built-in
modes prior to a formal release of this technology.
In stereo mode DX will use precisely the same camera model as it
does in mono mode - a single viewpoint and projection. However, immediately
prior to rendering, this information is handed to a stereo camera model
that returns left and right viewing information, including separate look-to
and look-from points and projection matrices. Much like the implementation
of stereo system modes above, we would like to support any camera models
as the user desires, and further, any number concurrently to be selected
among. We have therefore implemented this as a table of functions that
can be linked into DX, be run-time loaded, or allowed to default to a built-in
stereo camera model. In addition to the current viewing information, the
selected camera model is passed user-specified arguments that allow the
model to be parameterized. For example, the simplest mode I can think of
is to separate the left and right viewpoints by a fixed distance; this
distance can be passed in as a parameter.
The default stereo camera model implements stereo by providing look-to
and look-from separation (eg. parallel displaced viewing directions) and
asymmetric projection matrices. These are parameterized by three values:
the separation of the viewpoints as distance in model space (the 'ocular
separation'), the degree of asymmetry of the projection frustums - the
'frustum overlap', and the aspect ratio to be used. While the ocular separation
and aspect ratio are fairly obvious, the parameterization of frustum overlap
requires explanation. It is given as a floating point number, varying from
0 to 1, in which a value of 0.0 corresponds to zero overlap in the divergent
direction, and 1.0, to zero overlap in the convergent direction.
I have included a sample network showing the use of stereo in DX.
This example should show you how to create a full-screen image window,
how to implement key-stroke commands within that window, and how to control
stereo mode.
To run the sample, install the new files, then set an environment variable named 'DXHWMOD' to 'DXhwddOGL.o' to cause DX to use OpenG. Then set the environment vaiables named DX_INIT_STEREO_COMMAND and DX_EXIT_STEREO_COMMAND environment variables to the appropriate command-line strings that will set your hardware into stereo mode - for our SGIs, it would be "/usr/gfx/setmon -n STR_RECT" and "/usr/gfx/setmon -n 60HZ", INCLUDING thr double quotes (I think that if you use csh, it would be something like "setenv DX_INIT_STEREO_COMMAND "\"string\"") and run the network by typing 'dx -edit stereo'. You will see the network editor open. Hit Execute On Change from the Execute pulldown and
a 800x400 window will appear containing an image of two isosurfaces of a test dataset. You'll see that the window is in rotate mode; click and drag to rotate the image. Now with the mouse in the image window, hit the '2' key. The image will go into side-by-side stereo mode, showing two 400x400 images; if you have the knack of it, you can see this in stereo by crossing your eyes. Hit '1' to return to mono mode. In either mode, pressing 'r', 'p' or 'z' will select rotation, panning or zooming from the default user interactors.
In model space, the geometry you are looking at fills a cube of side 2, centered at the origin. Default values are used for the separation and overlap of the default camera model. By pressing the 'w' key in the image window you can widen the separation, and by pressing the 'n' key you can narrow it. (Note: currently there is a negation problem here - the default separation is -1 and 'w' and 'n' are reversed). By pressing the '-' key you can increase frustum overlap, while by pressing the '=' key you lessen it.
Now, using the Windows / Open All Control Panels pulldown selection, open the control panel. You will see interactors for the default settings of the separation and overlap; you can revert to the default values by selecting 'reset separation' or 'reset overlap'. The amount that keypress events either increase or decrease these values is adjustable using the 'Separation stepsize' and
'Overlap stepsize' interactors, and you can revert to the defaults by pressing the 'Reset separation' and 'Reset overlap' interactors. You can also revert to the default camera by pressing the 'Reset camera' interactor.
Now change the 'Window size' interactor to 'full-screen', clear the 'window decoration' toggle in the control panel, reset the server (using the Connections / Reset Server command from the Execute pulldown on the visual program editor - this is required in the alpha version but will not be required in the next version) and re-enter Execute On Change mode. You should get a full screen undecorated image of the object. Now press '3' (rather than 2) to enable stereo, to use the over/under system mode rather than the side by side mode. If you installed the correct environment variable commands to cause your system to go into stereo mode, it will.
In this mode, you will find that the key-press still work - hitting
'1' will revert the image to mono mode. However, the window will overlay
your control panels. To retrieve them, you can hit 'c' in the image window
to close it, and later pop it back up using the "Pop window up" interactor
that you will find in the control panel.
The sample network is divided into six pages. Perhaps the most interesting
is the one named 'main', in which the standard SuperviseWindow/SuperviseState/Display
sequence is augmented to add stereo parameterization. This occurs in the
lower left of the page. In addition to the standard "rendering mode" option,
causing the Display module to function in hardware rendering mode, two
options are added to the renderable object: one named 'stereo system mode',
which contains a simple integer specifying which stereo system mode to
use - eg. mono, side by side, over-under etc., and one named 'stereo camera
mode', which consists of a group containing two named members: 'mode',
hard-wired to 0 to select the default stereo camera mode, and 'args', a
three-vector containing the current values of the separation, overlap and
aspect ratio parameters to the default stereo camera mode. Here the stereo
system mode and the separation and overlap parameters come in from from
other pages where they are derived from current values and keypress events.
The set of available modes (or camera models) is specified as a table, where each table entry consists of pointers to functions implementing the particular mode. For stereo system modes, table entries consist of instances of the structure typedeffed as 'StereoSystemMode' in hwStereo.h. In this structure, the entry points are:
int InitializeStereoSystemMode(Display *dpy, Window frame)
InitializeStereoSystemMode is used to place the system into stereo mode. The X display and image window are passed in as parameters.
int CreateStereoWindows(Display *dpy, Window frame,
Window *leftWindow, WindowInfo *leftWindowInfo,
Window *rightWindow, WindowInfo *rightWindowInfo);
Given an X display and a frame window, CreateStereoWindows specifies windows to contain the left and right views and viewports into those windows into which the views should be rendered. The leftWindow and rightWindow may either be independent windows, created (using standard X calls) as child windows of the frame, or may be set to the frame window itself. leftWindowInfo and rightWindowInfo are pointers to structures that define viewports into the leftWindow and rightWindow. Thus, two separate windows may be created and returned to DX, with viewports specifying zero offsets and full sizes in each, or the frame window may be used by passing its window ID back as leftWindow and rightWindow, and using leftWindowInfo and rightWindowInfo to specify separate viewports.
int ExitStereoMode(Display *dpy, Window frame, Window leftWindow, Window rightWindow)
ExitStereoMode is used to exit stereo mode. It should delete the left and right windows if they were created in InitStereoMode, and reset the hardware into mono mode.
Specifying your own set of default stereo system modes is accomplished in one of two ways - either by linking in an implementation of a function that installs the system mode table, or by loading in a function which will be called at runtime to load in a table. The first method involves implementing the function:
DefaultStereoSystemModes(int *nModes, StereoSystemModes **modeTable)
which sets *nModes to the number of modes being installed and **modeTable to point to a table of modes that is either allocated or statically present in the file. Alternatively, you can implement a file which implements the equivalent function:
DXEntry(int *nModes, StereoSystemModes **modeTable)
You can then compile and link this into an object file. Then you cause DX to load it at runtime by adding the directory containing the run-time loadable file to the list held in the DXMODULES environment and set the DX_STEREO_SYSTEM_FILE environment variable to the actual name of the loadable file. Both of these methods are illustrated in the sysmodes.c, makefile.link and makefile.runtime that I've included in this package. NOTE: in each case, you must edit the makefile and set BASE to the directory point where you've installed DX.
Similarly, the set of available stereo camera models is specified by a table of structures of type StereoCameraModel, also defined in hwStereo.h. The entry points therein are:
void *InitializeStereoCameraMode(void *block, dxObject)
InitializeStereoCameraMode is used to manage a parameter block for the stereo mode. Whenever a window is placed into stereo mode, this routine is called and passed the stereo camera mode argument (the 'args' member of the group passed in as the 'stereo camera model' attribute on the renderable object). InitializeStereoCameraMode can then allocate a parameter block and process the argument object to derive parameter values to place into the block. This block is then passed to subsequent calls to the remaining functions of the mode. If the argument object to the stereo mode changes without actually changing the mode itself, the current argument block is passed in so that the parameter values it contains can be updated.
int ExitStereoCameraMode(void *block)
Implementations of ExitStereoCameraMode should free up any space allocated in the parameter block.
int CreateStereoCameras(void *block,
int projection, float fov, float width,
float *from, float *to, float *up, float near, float far,
float *lFrom, float *lTo, float *lUp, float **lProjection,
float *rFrom, float *rTo, float *rUp, float **rProjection);
CreateStereoCameras is passed the parameter block along with camera information, and should returns the left and right camera parameters and a pointer to projection matrices for each. The
camera information includes the projection type (1 implies a perspective transformation and 0 an orthographic transformation), the field of view (for perspective transformation) and width (for orthographic tranlformation), the current camera look-from, look-to and up vectors and the distance to the near and far clipping planes. The projection model used is very similar to that discussed in the OpenGL manuals, with the exception that a left-hand coordinate system is used.
int MapStereoXY(void *block, Window frame, Window view, WindowInfo viewInfo, int vx, int vy, int *fx, int *fy)
Given an event location (eg. button- or key-press event) in the view window passed in as the view window and/or viewport passed in as the view and viewInfo parameters, transform the point (vx, vy) to the corresponding point in the single-window model. This allows DX components external to the rendering process to handle event locations transparently to stereo rendering.
Again similar to the handling of stereo system modes, user-defined stereo camera models can be installed either at link- or run-time by implementing:
DefaultStereoCameraModes(int *nModes, StereoCameraModes **modeTable)
or
DXEntry(int *nModes, StereoCameraModes **modeTable)
and is illustrated in mymodes.c and makefile that I've included. To
install runtime loadable stereo camera modes you must add the directory
containing the loadable file to the list of directories in the DXMODULES
environment variable and set environment variable DX_STEREO_CAMERA_FILE
to the actual name of the loadable file.
Picking is provided via the StereoPick module and is demonstrated
in TestStereo.net. Run it again with the lt-in stereo modes (that is, unset
the DX_STEREO_CAMERA_FILE and DX_STEREO_SYSTEM_FILE environment variables),
and place the cursor in the display window. Initially, the example is not
in stereo mode. Using the center button, click on the object; a sphere
glyph will appear at the pick point. Pick again elsewhere and it'll move.
Now pick a third time near the center.
Now enter side-by-side stereo by hitting the '2' key. You'll see the
side-by-side images, and the pick sphere will appear in both. Zoom out.
Now, instead of picking on the object itself, you pick on the virtual 3D
object which, in this case, is centered in between the two images (to actually
see the virtual 3-D image you'll need to look at it cross-eyed, or use
a stereo-opticon of some sort). You therefore can right button in the center
and you'll see a sphere glyph appear in both eye views.
Lots. The code at this stage is inadequately tested and should be
considered alpha code. Any bugs found or comments on inadequacies encountered
will be gratefully received.