Pixlane / Blog / What Is Optical Flow?
What Is Optical Flow? A Visual Introduction You Can Try in Your Browser
Published July 2026 · 5 min read
Optical flow is the apparent motion of pixels between two consecutive frames of a video — or any two images taken a moment apart. An optical flow algorithm compares the frames and outputs a motion vector for every point: a small arrow that says where that pixel moved and how far. That deceptively simple idea powers video stabilization, motion effects, frame interpolation, and object tracking — and you can compute it on your own images in seconds, directly in your browser.
Optical Flow in One Picture
Imagine overlaying two frames of a cyclist riding past a building. The building's pixels barely move; the cyclist's pixels shift noticeably to the right. Optical flow turns that intuition into data: a field of vectors, one per pixel or per feature point, each describing the direction and speed of local motion.
There are two flavors:
- Dense optical flow computes a vector for every pixel in the image. You get a complete motion map of the scene — ideal for visualization, segmentation cues, and motion analysis.
- Sparse optical flow tracks only a set of distinctive feature points, such as corners. It is much cheaper to compute and is the classic choice for point tracking and camera-motion estimation.
Dense flow tells you what the whole scene is doing; sparse flow tells you precisely what a few well-chosen points are doing.
How Computers Estimate Motion
Nearly every classical method starts from the same assumption, called brightness constancy: a point in the scene keeps roughly the same brightness as it moves from one frame to the next. If a dark pixel in frame A shows up slightly to the right in frame B, the algorithm infers motion to the right.
One assumption is not enough, though. A single pixel's brightness change gives you one equation with two unknowns (horizontal and vertical motion) — the classic aperture problem. Looking at motion through a small window, you can only tell how an edge moves perpendicular to itself. Algorithms resolve the ambiguity by adding a second assumption: neighboring pixels tend to move together.
| Approach | Output | Typical use |
|---|---|---|
| Lucas–Kanade | Sparse (feature points) | Point tracking, camera-motion estimation |
| Farnebäck | Dense (every pixel) | Motion visualization and analysis |
| Deep learning (e.g., RAFT) | Dense | High-accuracy offline processing |
The Farnebäck method — the one behind Pixlane's optical flow tool — approximates the neighborhood around each pixel with a local polynomial, then finds the displacement that best explains how those polynomials change between frames. Modern deep-learning models can be more accurate on hard scenes, but classical methods remain fast enough to run in real time, even inside a browser tab.
All of these methods share the same failure modes, which follow directly from the assumptions: sudden lighting changes, reflections, occlusions (a pixel that simply disappears behind something), textureless regions, and very large jumps between frames all degrade the estimate.
How to Read a Flow Visualization
Raw flow data is just two numbers per pixel — horizontal and vertical displacement — so tools render it in ways your eye can parse:
- Vector arrows — arrows drawn on a sampled grid. Direction shows where pixels are heading; length shows how far they moved. Best for reading the overall motion pattern.
- Heatmap — displacement magnitude mapped to color intensity: bright, saturated regions are moving fast, while dim or dark areas are nearly static. Best for spotting where motion happens.
- Color wheel (HSV) — hue encodes direction and intensity encodes speed, packing the full field into a single image. Best for dense, per-pixel detail.
If you load two frames into the optical flow tool, you can switch between these views on the same result — a fast way to build intuition for what each encoding shows and hides.
Where Optical Flow Shows Up in Real Tools
Once you can measure motion per pixel, a lot of practical video processing falls out of it:
- Video stabilization. Estimate the global camera motion between frames, smooth that trajectory, and warp each frame to compensate. That is essentially what a video stabilization tool does to remove handheld shake.
- Motion and time effects. Trails, ghosting, stutter, and dynamic-zoom looks in a motion effects editor all depend on reasoning about how content moves between frames.
- Video compression. Codecs like H.264 and AV1 use block-based motion estimation — a close cousin of optical flow — to encode only what changed between frames.
- Tracking and analysis. Following objects across frames, detecting activity in a scene, and estimating a drone's movement from its camera (visual odometry) all build on flow.
- Frame interpolation. Slow-motion and frame-rate conversion generate in-between frames by moving pixels along their flow vectors.
Try It Yourself in Your Browser
The fastest way to understand optical flow is to run it on your own images. Grab two frames of anything that moves — a pet, traffic, a hand waving — keeping the camera as still as you can so the motion belongs to the subject rather than the viewpoint.
Pixlane's optical flow tool computes dense Farnebäck flow between the two frames and renders vector, heatmap, and direction color-wheel views. Like every Pixlane tool, it is free and runs entirely in your browser through WebAssembly: your images are processed on your own device and never uploaded to a server. That makes it safe to experiment with private footage, and it means results appear in seconds with no account or install.
A few tips for a clean first result: keep the two frames close in time, avoid lighting changes between shots, and start with a subject that has visible texture. Then break the assumptions on purpose — change the lighting, or move the camera instead of the subject — and watch how the flow field responds. Seeing the algorithm fail is the quickest way to understand what it actually measures.