Automatic Lens Distortion in Nuke from a Single Frame

In VFX, every comp that puts CG into a plate has to match the lens the plate was shot through. Most of the time this boils down to two questions:

How strong is its distortion? Where is the distortion centered?

The usual answer is found with a lens grid shot on set and solved in 3DEqualizer, PFTrack or Nuke's LensDistortion. Without a grid, Nuke's CameraTracker and LensDistortion can also solve the distortion by tracking features through the shot, but only if the camera actually moves.

I explored a third option that avoids the need for a grid or a camera move (Nuke can also estimate distortion from a single frame by marking straight lines, but that still requires an artist to identify and trace suitable edges).

It's built on AnyCalib, an open-source network that predicts a camera ray for every pixel of an image. I fine-tuned it on real cine footage and on renders through Nuke's lens model, then converted those outputted rays to Nuke-compatible distortion coefficients for the LensDistortion node.

The basic workflow

Connect a plate to the AnyCalib-Nuke node, then press Analyze.

1. The node runs inference on the model via Nuke's bundled libtorch.
2. Fits each pixel's predicted ray to calculate the likely distortionCenter.
3. Calculates distortion1 (k1), distortion2 (k2) for the lens distortion model.
4. Builds LensDistortion2 nodes or writes the STMaps directly.

The viewer overlay draws the straight edges/lines the node found, green if the estimate straightens them and red if not.

Results

Tested against 341 plates of real cinema footage from 13 different locations.

The footage is InFlux (Liang et al., NeurIPS 2025): real footage from an ARRI Alexa Mini and two cine zooms, with the lens calibrated for every frame.

The fine-tuned model averaged a mean error of 5.36 px.

Most cinema lenses are mild, so predicting no distortion at all is a relatively strong baseline, at around 8px. Stock AnyCalib doesn't clear that baseline unfortunately. It reads real lenses with too much barrel distortion and scores around ~10px.

Estimate	Mean map error	Against doing nothing
No distortion	8.34 px	baseline
Stock AnyCalib	9.97 px	↑ 1.63 px worse
Straight edge refinement only	7.10 px	↓ 1.24 px better
Fine-tuned network	5.81 px	↓ 2.53 px better
Fine-tuned + straight edge refinement	5.36 px	↓ 2.98 px better

Undistortion map error: undistort a grid of points with the estimate and with the calibrated lens, and average how far apart they land from ground truth.

Here it is on one plate, each arrow the miss at one grid point:

Averaged over the whole test set:

The node is most useful on wide and mid lenses with real distortion. On long lenses and good primes, doing nothing still wins (see Limits).

How it works

Under the hood, AnyCalib-Nuke does three things:

1. Predicts a ray for every pixel. Imagine drawing a line from the camera through each pixel, extending it into the scene.

   • A distorted camera lens bends those directions, largely at the edges of the frame.
   • AnyCalib (Tirado-Garín and Civera, 2025) predicts these directions for each pixel in an image.
   • These predicted rays aren't tied to any one lens model, so Nuke's can be fitted to them afterwards.

2. Fits Nuke's lens model to the rays. Nuke can't understand these predicted rays natively.

   • From the thousands of predicted rays, we need to figure out distortion coefficients used by Nuke.

   • Nuke's Classic model is one formula, where $r$ is a pixel's distance from distortionCenter:

     $$\text{undistorted} = \frac{\text{distorted}}{1 + k_1 r^2 + k_2 r^4}$$

   • The node solves for the distortion values whose rays line up best with the network's.

3. Straightens the edges. Lines that are straight in the scene should also come out straight once undistorted.

   • The node finds long lines the lens should have bent and nudges distortion1 and distortion2 until they're straight, while staying close to the network's answer.

Lens distortion basics: barrel, pincushion and STMaps

• Barrel bulges the image outward and is common on wide lenses. Undistorting stretches the plate past the format, so CG needs overscan.
• Pincushion pinches it inward and is common on long lenses. Undistorting leaves the format's corners empty.

The formula.

$$\text{undistorted} = \frac{\text{distorted}}{1 + k_1 r^2 + k_2 r^4}$$

Every pixel gets pushed toward or away from the center, and $r$, its distance from the center, decides by how much. The middle barely moves and the corners move most.

• distortion1 ($k_1$) sets the strength: negative is barrel, positive is pincushion.
• distortion2 ($k_2$) only matters out in the corners, where $r^4$ grows fast, and fine-tunes how far they bend.

Measured in pixels. A distortion1 value is hard to picture, and it means something different on every format: −0.03 moves an HD corner about 45 px, and an ALEXA Mini LF corner nearly twice that. So this post measures distortion by how far the corner moves.

Converting rays to distortion coefficients

The unknowns are focal length $f$, radial distortion coefficients $k_1$ and $k_2$ (Nuke's distortion1 and distortion2), and the distortion center $(c_x, c_y)$.

First, find the distortion center.

Radial distortion moves points toward or away from the distortion center, without changing their direction relative to it.

A pixel at $(x, y)$ sits at offset $(x - c_x,\ y - c_y)$ from the center. Undistorting only scales that offset, and the pixel's ray is the undistorted offset plus a step of $f$ along the optical axis. So the ray's sideways part, $(r_x, r_y)$, is parallel to the pixel's offset from the center.

Parallel vectors have equal slopes:

$$\frac{y - c_y}{x - c_x} = \frac{r_y}{r_x}$$

Cross-multiplying gives one equation per pixel, linear in $c_x$ and $c_y$:

$$(x - c_x)\,r_y = (y - c_y)\,r_x$$

Every pixel draws a line along its ray, and least squares finds where the lines meet.

Then, solve for $f$, $k_1$ and $k_2$.

With the center known, each pixel has two distances from it:

• $r$, where the pixel actually is on the plate.
• Where it would be without distortion, which its ray tells us.

The second distance comes from how steeply the ray tilts away from the optical axis.

A ray $(r_x, r_y, r_z)$ goes $r_z$ forward, along the axis, and $r_x$ across and $r_y$ up.

Those two sideways parts are at right angles, so the ray goes $\sqrt{r_x^2 + r_y^2}$ sideways in total. Call the tilt $s$, sideways over forward:

$$s = \frac{\sqrt{r_x^2 + r_y^2}}{r_z}$$

An undistorted lens works like a pinhole, with the sensor $f$ behind the hole. Light comes through the hole at tilt $s$ and carries on in a straight line, moving $f \cdot s$ sideways by the time it reaches the sensor, so it lands $f \cdot s$ from the center.

Nuke's formula compares the two. Divide where the pixel is by where its ray says it should be:

$$\frac{r}{f\,s} = 1 + k_1 r^2 + k_2 r^4$$

The left side is how much the lens squeezed that pixel toward the center:

• With no distortion, nothing is squeezed, and it's 1 everywhere.
• With barrel distortion, it's 1 at the center and drops below 1 toward the edges. $k_1$ sets how fast it drops, and $k_2$ adjusts the far corners.
• Pincushion is the opposite, rising above 1 toward the edges.

Every pixel gives one of these equations, and least squares finds the $f$, $k_1$ and $k_2$ that fit them all best. $f$ is pinned down near the center, where nothing is squeezed and $f = r / s$. $k_1$ and $k_2$ then describe how the squeeze changes toward the edges.

LoRA Training

Why fine-tune. Stock AnyCalib learned from renders too, just not ones that look like actual everyday footage.

Half its training images were fisheyes, all of them barrel, and none had a field of view under 20°. Actual VFX plates tend to be a bit more mild with distortion, so it read them with too much barrel distortion, by ~39px at the corner on average.

Process:

Generate more synthetic frames. Ground truth is easy: pick a camera, render what it would see of a 360° panorama, and every pixel's ray is known exactly. The fine-tune rendered 65k+ new views through Nuke, from 82 panoramas outside AnyCalib's original training set.

• Add additional sharpness falloff + vignetting to training set to simulate real lens behaviors around corners.
• Additional grain and chromatic abberation artifacts were also added.

Incorporate real footage. Include real footage from cinema cameras with calibrated camera intrinsics. Exposes model to grain, visual defects, and other imperfections.

Fine-tuning. AnyCalib is a large DINOv2 backbone that reads the image, plus a small decoder and head that turn it into rays. The backbone stays frozen, with LoRA adapters on its attention and MLP layers to nudge how it reads a plate.

• The decoder and head train in full. That's 22M trainable weights on a RTX 3090, and the adapters merge back in, so each model ships as a single TorchScript file.

Limits

Mild lenses. Predicting no distortion still beats the node on long lenses and quality primes. The network struggles when lenses have minimal distortion, often a few px at the corners.

Fisheyes. Undistorting a fisheye to a flat image throws the corners thousands of pixels out, and the node doesn't flag it reliably yet.

Anamorphic. The model + LoRA are trained for radial distortion. It doesn't estimate squeeze or target LensDistortion2's anamorphic presets yet.

What's next

• Anamorphic output, through LensDistortion2's anamorphic presets.
• A clearer flag for plates it can't handle, fisheyes first.
• Finding more real footage with calibrated camera intrinsics for training.

Thanks to Javier Tirado-Garín and Javier Civera for AnyCalib, to Erich Liang, Jia Deng and the InFlux team for real footage with calibrated lenses, and to Poly Haven for the panoramas.