Reflection Scene rendered by our combined renderer

I’ve been studying computer sciences for three years and recently I finished writing my bachelor thesis. The topic was implementing reflections and refractions in the context of mixed reality. Basically, mixed reality means adding virtual objects into a video stream (of real environment). One of the goals is to make the resulting images look as realistic as possible. Naturally this includes reflection and refraction effects.

In my work I utilise the framework implemented in the RESHADE project. There was already a mixed reality renderer based on a Direct3D rasterizer. My task was to work on a ray tracer using OptiX and merge the resulting image with the rasterizer’s image. OptiX provides a ray tracing framework for the graphics card.

Glass Bunny Scene

Generally a ray tracer is slower than a rasterizer, but it has better quality with reflections and refractions. Because of that, we used a combined method. The ray tracer is working only on transparent and reflecting surfaces. This is achieved by a mask, which is created by the rasterizer in a separate render pass. This mask indicates, where reflecting or refracting objects are.

Especially with virtual flat mirrors there are very strong artefacts (see reflection scene, right mirror). This is due to lack of information. We are using the video frame, the environment map and the textured model of the real scene to gather information for the reflected ray.

Other problems we encountered were artefacts due to different shading algorithms in the rasterizer and ray tracer, and various issues with implementation due to the tone mapper.

When I was more or less finished with my work, I discovered that somebody else in the project was working on the implementation of reflections and refractions using the rasterizer. This was cool, because we were able to compare the results. Usually rasterizing is much faster than ray tracing. Surprisingly this was not the case with all scenarios. The reflection scene (first image) was faster with the combined method, the transparent bunny was slower. Especially with the bunny scene, I believe that the main drawback in speed was accuracy. The ray tracer also computes inner reflections, which causes a lot of additional work. I didn’t manage to benchmark this (rendering without inner reflections), because I was running out of time, but maybe I will do it yet..

Thanks to Michael Wimmer for making this work possible and special thanks to Martin Knecht for being an excellent supervisor and helping with all the questions .

Finally, if you are interested into more details, here you can find the thesis..

Well, we added loads of new features, mainly graphical ones:

• lens  flares and sun dazzle effect
• moving space dust (white circles), which are moving randomly, this is an simple particle system.
• explosion and fire stream effects (more complicated particle systems)
• environment mapping (on the ships and station)
• bloom
• music and sound effects
• some power ups (armor, rockets, boost)
• …

Here is a video, we made for the presentation. Unfortunately it’s low quality, I hope to get a better one tomorrow, but it’s probably gone..

Update: We found a better HD quality Video

I’ve made a short video, where you can see bullet physics in action (at the end of the video you can see bullet debug mode enabled).

Music by Bézèd’h, music and video are CC licensed.

This term i’m attending the second computer graphics course in my university. The goal is to program a 3d game in OpenGL. We (Felix and me) chose to program a split screen flight duel game, as this is a simple game concept, but it’s still very extensible graphics and game logic wise.

The first step was to setup OpenGL etc. (GLEW, GLFW), which produced this first screen shot in begin of April.

We have worked hard and now we have implemented model loading (Assimp), shading, texturing, controls and split screen. Looks good for our first deadline

Next steps will be to implement some more game logic, in game display of hull state, frags and menu and then physics. We will use CEGUI and Bullet for these tasks..