Simple Shadow Volumes with Geometry Shaders in Unity
Video games are interactive, real-time and simulating. They can simulate entire cities within themselves sometimes, and when making games, sky is the limit with how you want your world to be. I wanted to try an idea, to see a world where a shadow can be seen, as a dark obscuring volume, throughout the entire stretch of its path. This also connects subjects to their shadows, which is more a more subtle and philosophical effect. A usual player might not even notice that the world has such a feature, but it definitely adds to their experience of perceiving the world as a darker place than real life (or maybe even the shadows are coloured with unique colours for each character!).
The GPU Rendering Pipeline: A Quick Overview
When a mesh is drawn on screen, it passes through several programmable stages on the GPU. The world and all the 3D models are stored and loaded as a list of points or vertices. These vertices are then transferred to the GPU and a "Shader Program" runs on the GPU doing all the math necessary to color the correct pixels appropriately.
The Vertex Shader runs first, processing each vertex individually, transforming positions, passing along normals and texture coordinates for each vertex and calculating the pixel these vertices will land up on the screen.
The GPU groups these vertices together to make quadrilaterals or triangles (called shape/primitive). Next comes the Geometry Shader, which takes the input of one primitive at a time, and can any number of primitives. This means Geometry Shaders can modify geometry and vertex definitions at runtime. Finally, the Fragment (Pixel) Shader runs once per pixel covered by the primitives, determining the final color written to the screen on those pixels which are covered by the primitive.
Most shaders only use the vertex and fragment stages. We will use the Geometry Shader in this article to do make the shadow volumes by generating new triangles on the fly based on per-triangle logic.
The Goal: Volumetric Shadow Regions
Traditional shadow mapping answers a binary question: "is this point in shadow?" You can ask this question for each pixel is drawn of any given shape, and thus determine to draw it brightly (as if it is lit by a candle or the light), or color it darkly to demonstrate shadows. However, if you look through the distance of a shadow, the Pixel Shader has to know whether a ray through the given pixel passes through a shadow of any other object.
Our idea is to construct an actual 3D volume in space that represents the region occluded from the light. Any geometry that falls inside this volume is in shadow. This shader implements the extrusion step. It takes an occluding mesh and stretches it away from the light source to form the walls of the shadow volume. The volume is rendered as a semi-transparent overlay, making the shadowed region visually apparent.
How the Geometry Shader Builds the Volume
The Vertex Shader
The vertex shader in this implementation is deliberately minimal. It passes object-space positions and normals straight through to the geometry shader without any transformation The heavy lifting happens in the next stage.
ZWrite Off // Prevent this shadow volume's write to depth buffer
Cull Back // Cull faces which are pointing away from the camera
Blend SrcAlpha OneMinusSrcAlpha // Enable Alpha
// Declare vertex shader HLSL Function for Unity
#pragma vertex vert
// vertex shader inputs
struct Attributes
{
float4 position : POSITION;
float4 normal : NORMAL;
float2 uv : TEXCOORD0;
};
// vertex shader outputs to geometry shader
struct v2g
{
float4 vertex : POSITION;
float3 normal : NORMAL;
float2 uv : TEXCOORD0;
};
// Vertex Shader Code
v2g vert(Attributes IN)
{
v2g OUT;
OUT.vertex = IN.position;
OUT.uv = IN.uv;
OUT.normal = IN.normal;
return OUT;
}The Geometry Shader
The geometry shader receives one triangle at a time (three vertices). Its first task is back-face determination: it computes the triangle's world-space normal and takes a dot product with the main directional light's direction. If the triangle faces away from the light (`dot(world_normal, lightDir) < 0`), it is a candidate for extrusion — it sits on the shadow-casting silhouette of the mesh. Triangles facing the light are discarded entirely (no vertices are emitted for them).
For each shadow-facing triangle, the shader computes an extrusion vector by projecting the light's direction back into object space and scaling it by a fixed magnitude. Three new vertices are created by displacing the original triangle's vertices along this vector, producing an "extruded" copy of the triangle pushed away from the light.
For connecting the original and extruded triangles to form the walls of the volume, we have written a utility function make_traingle_from_verts, which takes 3 vertices, constructs and outputs a valid triangle from them. Each of the three edges of the original triangle becomes a quad (two triangles) bridging the gap between the near face and the far face. make_traingle_from_verts also computes a face normal from the cross product of the edges, transforms vertices by the MVP matrix, and appends them to the (output) triangle stream. With three edges producing two triangles each, the geometry shader emits up to 18 vertices per input triangle (hence the [maxvertexcount(30)] declaration, which includes headroom).
The result is a closed shell extending from the shadow-facing surfaces of the mesh away along the light direction, forming the shadow volume walls.
// geometry shader outputs to fragment shader
struct g2f
{
float4 vertex : SV_POSITION;
float3 normal : NORMAL;
};
// Utility
void make_traingle_from_verts(
float4 v0, float4 v1, float4 v2,
inout TriangleStream<g2f> triStream
) {
float3 normal = normalize(cross(v1 - v0, v2 - v0));
g2f o;
o.vertex = mul(UNITY_MATRIX_MVP, v0);
o.normal = mul(UNITY_MATRIX_MVP, normal);
triStream.Append(o);
o.vertex = mul(UNITY_MATRIX_MVP, v1);
o.normal = mul(UNITY_MATRIX_MVP, normal);
triStream.Append(o);
o.vertex = mul(UNITY_MATRIX_MVP, v2);
o.normal = mul(UNITY_MATRIX_MVP, normal);
triStream.Append(o);
triStream.RestartStrip();
}
// Geometry shader code
int _InvertNormals;
// Max vertices output by the GS (e.g., 2 quads + original edge)
[maxvertexcount(30)]
void geom(triangle v2g input[3], inout TriangleStream<g2f> triStream)
{
float3 normal = normalize(cross(
input[1].vertex - input[0].vertex,
input[2].vertex - input[0].vertex)
);
float3 world_normal = mul(UNITY_MATRIX_M, normal);
Light mainLight = GetMainLight();
float extrude_mag = 10.0;
float4 extrusion = extrude_mag * float4(mul(
UNITY_MATRIX_I_M, mainLight.direction
));
world_normal = _InvertNormals == 0 ? world_normal : -world_normal;
if (dot(world_normal, mainLight.direction) < 0)
{
// Extrude this triangle
// Each edge of this traingles gets extruded to a quad
// A quad is 2 new triangles
float4 extruded_v0 = input[0].vertex - extrusion;
float4 extruded_v1 = input[1].vertex - extrusion;
float4 extruded_v2 = input[2].vertex - extrusion;
make_traingle_from_verts(extruded_v0, input[0].vertex, extruded_v1, triStream);
make_traingle_from_verts(extruded_v0, extruded_v2, input[0].vertex, triStream);
make_traingle_from_verts(extruded_v1, input[1].vertex, extruded_v2, triStream);
make_traingle_from_verts(extruded_v1, input[0].vertex, input[1].vertex, triStream);
make_traingle_from_verts(extruded_v2, input[2].vertex, input[0].vertex, triStream);
make_traingle_from_verts(extruded_v2, input[1].vertex, input[2].vertex, triStream);
}
else
{
// This triangle gets skipped
}
}Rendering the Volume
The fragment shader is intentionally simple: it outputs the base color at a low alpha controlled by the `_Intensity` property. The pass uses alpha blending (`Blend SrcAlpha OneMinusSrcAlpha`) with depth writes disabled (`ZWrite Off`), so the volume renders as a translucent darkening overlay.
// Fragment Shader Code
float4 _BaseColor;
float _Intensity;
half4 frag(g2f IN) : SV_Target
{
float4 col = _BaseColor * float4(1, 1, 1, _Intensity);
return col;
}
Summary
Demonstration of this approach
This shader demonstrates a simple approach to model shadow volumes. We use the geometry shaders to identify light-facing silhouette triangles, extrude them along the light direction to form volume walls, and render the resulting geometry. This implementation uses a direct alpha-blended visualization, and creates geometry on the fly. This shadow volume will change direction with the light direction at runtime, and some material properties like color and intensity can be changed per object to use this effect for artistic purposes.
Member discussion