In this article
In my anterior-segment OCT experiments, I train models by giving them two different kinds of targets. A masked autoencoder predicts hidden pixels. My global–local JEPA recipe brings representations of several views of an image into agreement. Both learn from the images themselves, and the choice of target shapes what the encoder learns.
Anterior-segment optical coherence tomography, or AS-OCT, produces cross-sectional images of the front of the eye. I implement these objectives to study how models represent corneal structure. Their differences help explain the feature geometry I measured across five training runs.
This research supports the next stage of OphtaFlow AI. I am prioritizing JEPA as the first self-supervised foundation for broader diagnostic coverage and, ultimately, fusion of corneal and retinal information from several devices.
To follow the learning signal, we can start with a drawing. Cover part of an arc with a sheet of paper and predict the missing lines. The answer is already underneath the cover. Self-supervised learning turns that idea into a training task: construct an input and target from the data, then adjust a model to make its prediction closer to the target.
The examples below use three synthetic drawings—a shallow arc, a steep arc and an asymmetric arc—to make the calculation visible.
Follow an image into the encoder
An image is an array of numbers. An encoder transforms those numbers into a learned representation. Training adjusts its parameters to reduce the error defined by the task.
An embedding is one such representation: a vector like [0.4, -1.2, 0.7]. Image models commonly produce a larger vector for each patch of an image. The coordinates encode distinctions learned through training; inspecting the vectors and using them in another task reveals which distinctions they carry.
Both masked autoencoders and joint-embedding methods train an encoder. The useful place to begin their comparison is the target: what is each prediction compared with?
MAE: predict the hidden pixels
In the masked autoencoder introduced by He and colleagues, the image is divided into patches and a subset is hidden. The encoder processes the visible patches. A decoder combines their representations with position information to predict the pixels in the hidden patches. Training error is calculated on that hidden content. MAE paper
Suppose a hidden part of our synthetic arc contains [0.2, 0.8], and the decoder predicts [0.3, 0.6]. The errors are 0.1 and -0.2. Squaring makes both contribute positively, and averaging gives:
mean squared error = ((0.3 − 0.2)² + (0.6 − 0.8)²) / 2
= (0.01 + 0.04) / 2
= 0.025
The optimizer uses this error to update the model. The same calculation extends to many hidden pixels. In recipes that normalize each target patch, the loss compares normalized pixel values; that preprocessing defines the quantity being minimized. The original MAE implementation includes this option. MAE implementation
Predicting the missing arc benefits from information about the visible contour, its position and recurring shapes. The reconstruction task supplies the encoder’s learning signal. In my full-field AS-OCT recipe, it covers both the cornea and its surrounding image context. Each region contributes according to the pixel targets and their weighting, which makes target preparation an important part of the implementation.
JEPA: predict in representation space
Now give the model two views of the same shallow arc: a full drawing with a brightness change and a local crop. A joint-embedding objective relates their numerical representations. The target comes from an encoder, so its construction is part of the method.
I-JEPA, introduced by Assran and colleagues, predicts representations of target regions from a context region. It maintains a target encoder through an exponential moving average: each update blends the previous target parameters with the current online encoder parameters. I-JEPA paper
LeJEPA, by Balestriero and LeCun, combines a predictive objective with SIGReg, an explicit regularizer that shapes the distribution of representations. This supplies a different way to train the representation. LeJEPA paper
For my September 2026 AS-OCT experiments, I implemented a shared-encoder recipe with global and local views and VISReg regularization. Here is the actual calculation.
Ten views, one encoder
Each source image produces two global views and eight local views. Every view passes through the same encoder, followed by a projector: a small network that maps encoder features into the space used by the objective.
For each source, I average the two global projections. All ten view projections are encouraged to approach that average. In a two-coordinate example, global projections [1, 3] and [3, 1] give a target of [2, 2]. A local projection [2, 3] then has a mean squared difference of 0.5 from the target.
The real objective averages across coordinates, views and sources. Gradients flow through both global projections as well as the predictions. The same trainable encoder constructs both sides of the comparison.
This agreement term has an exact shortcut: give every view of every drawing the vector [2, 2]. The global average is [2, 2], every local vector matches it, and the agreement error is zero. The shallow, steep and asymmetric arcs have become indistinguishable.
That is representation collapse. Any constant vector gives the same result. A classifier receiving that vector has lost the information needed to distinguish the drawings.
Paired-view error: 0. Relative spread: 1.00.
Each number represents a different source with two identical views. Move the slider to see the sources converge while paired-view error stays at zero.
The control moves the constructed source vectors apart while keeping their paired views equal. Agreement error stays at zero across the whole range. This interactive example isolates the agreement calculation; the regularizer adds the next part of the training objective.
Preserve variation across sources
My recipe combines agreement within a source with regularization across sources. VISReg, introduced by Wu, Balestriero and Levine, controls representation scale and distributional shape separately, using one-dimensional projections to compare distributions. VISReg paper
The implementation exposes three components:
- Center: penalize a nonzero average representation across sources.
- Scale: penalize coordinate standard deviations that depart from unit scale.
- Shape: center and standardize the representations, project them along sampled directions, and compare the sorted projections with Gaussian reference quantiles.
The shape calculation reduces a multidimensional distribution to several ordered lists of values. Comparing those lists provides a training signal about how the representations are distributed beyond their average and scale.
The AS-OCT configuration uses:
total loss = 0.3 × agreement + 0.7 × regularization
The coefficients belong to this experiment. Agreement relates the ten views of each source; regularization shapes variation across sources. I monitor encoder features before the projector to follow how that training signal changes the representation used by later tasks.
Choose what the views should preserve
The input views determine which information the model can use. Brightness changes encourage agreement across intensity variations. Cropping connects a local region to the full image. Reducing resolution changes the detail available to the encoder.
An invariance is a learned tolerance to a transformation. For the arc drawings, brightness tolerance may preserve the distinction between shapes. Removing a small notch has a different effect if the task is to detect that notch. Choosing transformations therefore makes the intended distinctions concrete.
In my AS-OCT recipe, I combine full-image context with native local crops. Six local views are guided by the corneal mask, and two are sampled independently of it. This lets me study the effect of anatomical sampling while retaining additional image context. The five-recipe article gives the resolutions, masking rules and patch budgets.
I examine the resulting training at three levels: gradients and reproducible updates show how optimization is behaving; representation measurements show how variation changes; patient-grouped task evaluation tests which distinctions the features make accessible. Each loss is read in the units of its own objective—pixel reconstruction error for MAE, representation agreement and regularization for this JEPA recipe.
Test the mechanism
Suppose every image produces [7, -3], and every augmented view matches its source perfectly. The agreement error is zero, and all images still have the same representation. The center, scale and shape terms give the model a reason to change it: useful agreement needs variation between sources as well as consistency between views.
That calculation is the reason I implemented and inspect the regularizer alongside the agreement term. In the recorded AS-OCT runs, these objective and input choices produced different feature spectra. The next article follows the five configurations that generated those results.