Posts Tagged ‘CG’

Merging Ray Tracing and Rasterization in Mixed Reality

Saturday, January 5th, 2013
Reflection Scene rendered by our combined renderer

Reflection Scene rendered by our combined renderer

I’ve been studying computer sciences for three years and recently I finished writing my bachelor thesis. The topic was implementing reflections and refractions in the context of mixed reality. Basically, mixed reality means adding virtual objects into a video stream (of real environment). One of the goals is to make the resulting images look as realistic as possible. Naturally this includes reflection and refraction effects.

In my work I utilise the framework implemented in the RESHADE project. There was already a mixed reality renderer based on a Direct3D rasterizer. My task was to work on a ray tracer using OptiX and merge the resulting image with the rasterizer’s image. OptiX provides a ray tracing framework for the graphics card.

Glass Bunny Scene

Glass Bunny Scene

Generally a ray tracer is slower than a rasterizer, but it has better quality with reflections and refractions. Because of that, we used a combined method. The ray tracer is working only on transparent and reflecting surfaces. This is achieved by a mask, which is created by the rasterizer in a separate render pass. This mask indicates, where reflecting or refracting objects are.

Especially with virtual flat mirrors there are very strong artefacts (see reflection scene, right mirror). This is due to lack of information. We are using the video frame, the environment map and the textured model of the real scene to gather information for the reflected ray.

Other problems we encountered were artefacts due to different shading algorithms in the rasterizer and ray tracer, and various issues with implementation due to the tone mapper.

When I was more or less finished with my work, I discovered that somebody else in the project was working on the implementation of reflections and refractions using the rasterizer. This was cool, because we were able to compare the results. Usually rasterizing is much faster than ray tracing. Surprisingly this was not the case with all scenarios. The reflection scene (first image) was faster with the combined method, the transparent bunny was slower. Especially with the bunny scene, I believe that the main drawback in speed was accuracy. The ray tracer also computes inner reflections, which causes a lot of additional work. I didn’t manage to benchmark this (rendering without inner reflections), because I was running out of time, but maybe I will do it yet..

Thanks to Michael Wimmer for making this work possible and special thanks to Martin Knecht for being an excellent supervisor and helping with all the questions :) .

Finally, if you are interested into more details, here you can find the thesis..

Lindenmayer brush for Krita

Tuesday, January 24th, 2012

This term i had a course about fractals on the university. We also had to write code for a submission and the lecturer made the technique and topic free to choose. So I chose to make a lindenmayer brush for krita, because I planed to do so since the first university year, where I heard of lindenmayer systems the first time.

I quickly coded something together, so that I was able to prove, that my idea would work.

Lindenmayer systems are some kind of a grammar, they work like this: You have an alphabet of letters, in this case these letters are simple short lines, then you have productions and a rule: Always produce all letters. These productions eat one letter and produce ether no, one or several new letters. There are several classes of such productions, one of the simplest would be “A -> AA”. More sophisticated productions, like in the brush, can contain parameters, if clauses, calculations, random values etc. Basically it would be possible to have several different productions, but I chose to have only one and in turn make it more powerful.

Here is an example of a production, it is used in one of the brushes:

if{letter[branched] == false} {
newLetter = newLetter();
newLetter[position] = variant(letter[endPosition]);
newLetter[angle] = variant(newLetter[angleToSun]);
newLetter[length] = 5;
letter[branched] = bool(true);
}

if{letter[age] > 1} {
letter.delete();
}

It actually creates just a line.

At first the productions were plain C++ code, but that is very cumbersome, as you have to build, install and start krita for testing. The next thing I tried, was to use QtScript and write the productions in ECMAScript (also called JavaScript). It took about 5 to 7 hours until I proved, that it was way to slow.

So I decided to implement my own little scripting language. The interpreter is about 700 lines of code, including some tests. It’s certainly not as powerful, as ECMAScript, but it’s fast and okayish for this use case.

There is an if clause, which supports &&, there is a rand() function, a function for mixing angles, some very basic math, bool and float data types, you can create new letters and delete old ones. It’s possible to write new parameters (“branched” in the code above), edit some default ones (position, angle, length..), which will be used for drawing and access some computed ones (endPosition, angleToSun, distanceToSun, age..).

I have to write some documentation, expand the possibilities of the language and work on the configuration interface. But for now you can already test the current code, it’s in the krita-lbrush-adamc branch in the calligra repository.

Here is a final picture..

The presets for this brushes are included in the repository :)

Chawah, Final update a little latish..

Tuesday, January 24th, 2012

I had lots of things to do, no time etc etc. You know the excuses. But now I want to write another blog about a new Lindenmayer brush for Krita, and though have the motivation to catch up with old posts..

Well, we added loads of new features, mainly graphical ones:

  • lens  flares and sun dazzle effect
  • moving space dust (white circles), which are moving randomly, this is an simple particle system.
  • explosion and fire stream effects (more complicated particle systems)
  • environment mapping (on the ships and station)
  • bloom
  • music and sound effects
  • some power ups (armor, rockets, boost)

Here is a video, we made for the presentation. Unfortunately it’s low quality, I hope to get a better one tomorrow, but it’s probably gone..

Update: We found a better HD quality Video :)

Chawah Progress

Saturday, June 4th, 2011

More than one month since my last post and we (Felix and me) are quite satisfied with our progress. There is some basic game logic (you can destroy the other ship and then the game restarts), health display and physics. We got all points at our first submission and currently we are starting implementing the effects. Environment mapping is already in place.

I’ve made a short video, where you can see bullet physics in action (at the end of the video you can see bullet debug mode enabled).

Music by Bézèd’h, music and video are CC licensed.

First chawah screenshots

Thursday, April 28th, 2011

Unfortunatly I hadn’t time for Krita any time recently. I had to work and do stuff for university. I will focus more on Krita in summer again..

This term i’m attending the second computer graphics course in my university. The goal is to program a 3d game in OpenGL. We (Felix and me) chose to program a split screen flight duel game, as this is a simple game concept, but it’s still very extensible graphics and game logic wise.

The first step was to setup OpenGL etc. (GLEW, GLFW), which produced this first screen shot in begin of April.
First Chawah screen shot

We have worked hard and now we have implemented model loading (Assimp), shading, texturing, controls and split screen. Looks good for our first deadline :)
First Chawah screen shot

Next steps will be to implement some more game logic, in game display of hull state, frags and menu and then physics. We will use CEGUI and Bullet for these tasks..