Screen Effects

Contents:

1. Intro

Screen effects allow you to create nice effects by processing the rendered image. Demos:

• Try the View -> Screen Effects menu in view3dscene, after loading any 3D scene. Note that you can activate many effects at the same time.

• Try the X3D files screen_effects.x3dv, screen_effect_blood_in_the_eyes.x3dv, screen_effect_film_grain.x3dv, screen_effect_movie_texture.x3dv. You should download our complete VRML/X3D demo models and open files in screen_effects subdirectory there, to see the complete working demos with an example castle model underneath.

• Compile and run Castle Game Engine demo in examples/screen_effects_demo.

2. Definition

You can define your own screen effects by using the ScreenEffect node in your VRML/X3D files. Inside the ScreenEffect node you provide your own shader code to process the screen, given the current color and depth buffer contents. With the power of GLSL shading language, your possibilities are endless :). You can warp the view, apply textures in screen-space, do edge detection, color operations and so on.

ScreenEffect : X3DChildNode {
  SFNode     [in,out]      metadata    NULL      # [X3DMetadataObject]
  SFBool     [in,out]      enabled     TRUE    
  SFBool     [in,out]      needsDepth  FALSE   
  MFNode     [in,out]      shaders     []        # [X3DShaderNode]
}

A ScreenEffect is active if it's a part of normal VRML/X3D transformation hierarchy (in normal words: it's not inside a disabled child of the Switch node or such) and when the "enabled" field is TRUE. In the simple cases, you usually just add ScreenEffect node anywhere at the top level of your VRML/X3D file. If you use many ScreenEffect nodes, then their order matters: they process the rendered screen in the given order.

You have to specify a shader to process the rendered screen by the "shaders" field. This works exactly like the standard X3D "Appearance.shaders", by selecting a first supported shader. Right now our engine supports only GLSL (OpenGL shading language) shaders inside ComposedShader nodes. To learn more about GLSL and X3D, see

• The overview of shaders support in our engine.
• The GLSL function reference.
• GLSL description at Wikipedia and GLSL description at Khronos wiki.

3. Shader language (GLSL) variables and functions

The GLSL shader code inside ScreenEffect can use some special functions and uniform variables.

// Screen size in pixels.
uniform int screen_width;
uniform int screen_height;
 
// Position of the current pixel,
// in range [0..screen_width - 1, 0..screen_height - 1].
// This pixel's color will be set by our <code>gl_FragColor</code> value at exit.
ivec2 screen_position();
int screen_x(); // Same as screen_position().x
int screen_y(); // Same as screen_position().y
 
// Previous color at this pixel.
vec4 screen_get_color(ivec2 position);
 
// Depth buffer value at this pixel.
// Only available needsDepth = TRUE at ScreenEffect node.
// The version "_fast" is faster, but less precise,
// in case full-screen multi-sampling is used.
float screen_get_depth(ivec2 position);
float screen_get_depth_fast(ivec2 position);
 
// In Castle Game Engine >= 6.5:
// Float-based versions of the above functions.
// They take/return float,vec2 instead of int,ivec2.
// The screen positions are in range [0..screen_width,0..screen_height].
vec2 screenf_position();
float screenf_x();
float screenf_y();
vec4 screenf_get_color(vec2 position);
float screenf_get_depth(vec2 position);
float screenf_get_depth_fast(vec2 position);

Note: do not redeclare these uniform variables or functions in your own GLSL shader code. Instead, just use them. If you try to declare them, you will get "repeated declaration" GLSL errors, in case of uniforms. Internallly, we always "glue" our standard GLSL code (dealing with screen effects) with your GLSL code, to make these variables and functions available without the need to declare them.

4. Examples

A simplest example:

ScreenEffect {
  shaders ComposedShader {
    language "GLSL"
    parts ShaderPart {
      type "FRAGMENT"
      url "data:text/plain,
      void main (void)
      {
        gl_FragColor = screen_get_color(screen_position());
      }
      "
    }
  }
}

The above example processes the screen without making any changes. You now have the full power of GLSL to modify it to make any changes to colors, sampled positions and such. For example make colors two times smaller (darker) by just dividing by 2.0:

void main (void)
{
  gl_FragColor = screen_get_color(screen_position()) / 2.0;
}

Or turn the screen upside-down by changing the 2nd texture coordinate:

void main (void)
{
  gl_FragColor = screen_get_color(
    ivec2(screen_x(), screen_height - screen_y()));
}

5. Details

Details about special functions available in the ScreenEffect shader:

• Internally, we pass the screen contents (color and, optionally, depth buffer) as a texture (normal non-power-of-two texture) or a multi-sample texture. You should always use the functions screen_get_xxx to read previous screen contents, this way your screen effects will work for all multi-sampling (anti-aliasing) configurations.

• The texture coordinates for screen_get_xxx are integers, in range [0..screen_width - 1, 0..screen_height - 1]. This is usually comfortable when writing screen effects shaders, for example you know that (screen_x() - 1) is "one pixel to the left".

  You can of course sample the previous screen however you like. The screen_position() (or, equivalent, ivec2(screen_x(), screen_y())) is the position of current pixel, you can use it e.g. to query previous color at this point, or query some other colors around this point (e.g. to blur the image).

  Since Castle Game Engine 6.5 you can also use float-based screen coordinates with functions like screenf_get_xxx (note the extra "f" letter in the name). The float coordinates are in the range [0..screen_width, 0..screen_height].

• If you set "needsDepth" to TRUE then we also pass depth buffer contents to the shader. You can query it using screen_get_depth function..

  You can query depth information at any pixel for various effects. Remember that you are not limited to querying the depth of the current pixel, you can also query the pixels around (for example, for Screen Space Ambient Occlusion). The "Flashlight" effect in view3dscene queries a couple of pixels in the middle of the screen to estimate the distance to an object in front of the camera, which in turn determines the flashlight circle size.

• Remember that you can pass other uniform values to the shader, just like with any other ComposedShader nodes. For example you can pass an additional helper texture (e.g. a headlight mask) to the shader. Or you can route the current time (from TimeSensor) to the shader, to make your effect based on time.

6. Todos

ScreenEffect under a dynamic Switch doesn't react properly — changing "Switch.whichChoice" doesn't deactivate the old effect, and doesn't activate the new effect. For now, do not place ScreenEffect under Switch that can change during the world life. If you want to (de)activate the shader dynamically (based on some events in your world), you can send events to the exposed "enabled" field.