Datamoshing in COPs
A full motion-based pixel advection pipeline in Houdini COPs featuring three effect nodes and two motion extraction methods
There is something about seeing a video fail, revealing the system underneath.
Datamoshing originates from encoding errors, which we don’t have access to in COPs, so this implementation tries to replicate the effect as closely as possible.
What even is datamoshing?
Traditional datamoshing relies on video encoding artifacts. Video streams are split into I-frames (Intra-coded), which display full images without any outside data, and P-frames (Predicted) and B-frames (Bi-predictive).
Instead of storing full frames, P- and B-frames only contain data describing changes from the preceding frame, like Motion Vectors for pixel movement or color transforms.
The classic effect happens when you remove the I-frames, forcing the P- and B-frame motion data to transform the wrong pixels.
Datamoshing + Pixelsorting
Pixelmosher mask-driven mosh region test
Block Matching motion vector extraction
Faking the codec
Optical Flow or Block Matching is used to extract Motion Vectors from any input video. Block matching is the actual algorithm used during compression to measure motion.
These Motion Vectors are then used to advect the frozen frame’s pixels, while masks based on frame differences and motion magnitude mix the new and old frames together, causing the characteristic distortion.
Optical Flow Lite motion advection detail
Control
One of my main goals was maximum user control, since the original effect is not very controllable.
Rendered motion vectors can distort any video, or just noise. Custom masks allow only certain regions to be moshed, or mosh based on depth. Switch to the custom optical flow for smoother motion.
At last free from the whimsical nature of the codec!
Walkthrough of all the nodes in action
Included nodes
Pixelmosher
Automatic datamosh emulation with mask-driven mosh regions, colorshift residuals, and feedback control.
Pixel Advection
Direct pixel displacement along motion vectors.
Pixel Pusher
Iterative pixel pushing with built-in loop control.
Block Matching
Fast GPU motion estimator, the algorithm behind real encoding artifacts.
Optical Flow Lite
Smoother motion extraction for finer detail.
Butterfly moshing
Jellyfish, notice the color distortion
