iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Supervised contrastive learning (SupCon) in Keras is a two-stage image-classification workflow: first use labels to train an encoder to place same-class images near one another in an embedding space, then freeze that encoder and train a classifier on its features. Keras’s CIFAR-10 example demonstrates the approach with ResNet50V2 and a custom contrastive loss; its reported accuracy is specific to that tutorial run, not a general performance guarantee.
How supervised contrastive learning differs from a single classifier fit
Ordinary image classification commonly trains an encoder and classification head together using cross-entropy. In the SupCon workflow shown by Keras, the model instead learns useful representations first, then fits a classifier on top of the frozen encoder.
- Contrastive pretraining: pass labeled images through an encoder and a projection head. A supervised contrastive loss uses the labels to treat other examples of the same class as positives and examples of different classes as negatives.
- Classifier training: discard the projection head for the classifier shown, freeze the pretrained encoder, and train classification layers on the encoder’s features.
The intended effect is to pull same-class representations together and push representations from different classes apart. The paper by Prannay Khosla and coauthors compares two supervised contrastive loss formulations and reports image-classification benchmarks; it is separate from the Keras tutorial’s CIFAR-10 demonstration. Read the paper, “Supervised Contrastive Learning”.
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallCrashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteSet up the Keras example and model
The official Keras supervised contrastive learning example is a runnable notebook demonstration. It was created on 2020-11-30 and marked last modified on 2026-08-19. The example sets KERAS_BACKEND to jax and notes that TensorFlow and Torch are alternatives. It uses the Keras API, including keras.layers and keras.ops.
#1 Best Overall
- Use scikit-learn to track an example ML project end to end
- Explore several models, including support vector machines, decision trees, random forests, and ensemble methods
- Exploit unsupervised learning techniques such as dimensionality reduction, clustering, and anomaly detection
- Dive into neural net architectures, including convolutional nets, recurrent nets, generative adversarial networks, autoencoders, diffusion models, and transformers
- Use TensorFlow and Keras to build and train neural nets for computer vision, natural language processing, generative models, and deep reinforcement learning
The tutorial loads CIFAR-10: 50,000 training images and 10,000 test images, each 32 × 32 × 3. Its encoder is ResNet50V2 followed by global average pooling, producing a 2,048-dimensional feature vector. A dense projection head with 128 units and ReLU activation maps that vector for contrastive pretraining.
The projection head belongs to the pretraining model: it supplies the vectors used by the contrastive objective. For the subsequent classifier, the example uses the encoder output rather than the projection-head output.
What the custom loss computes
The example subclasses keras.losses.Loss and implements call(), the standard Keras extension point for a custom loss. In outline, it normalizes the projected feature vectors, calculates pairwise dot products, scales the similarities by a temperature, and builds a label-based mask.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →- Positive pairs: two different examples with equal labels are treated as a positive pair.
- Self-pairs: the diagonal of the pair mask is excluded, so an example is not counted as its own positive.
- Loss: a numerically stabilized log-probability calculation is used to average the positive-pair log probabilities.
The label mask makes batch composition operationally important. A batch needs useful same-class examples for positive pairs; when a class is represented only once in a batch, that example has no other same-label item there to serve as a positive. Check how labels are distributed across batches when adapting the example.
Pretrain the encoder with labeled batches
Compile the contrastive model with the custom loss and train it with both images and labels. Labels are needed by the loss to construct the positive-pair mask; they are not merely targets for a conventional softmax classifier in this phase.
The Keras tutorial lists batch size 265, temperature 0.05, learning rate 0.001, and 10 epochs. Treat these as the example’s settings, not recommended defaults: batch construction, encoder, data, and compute constraints can all change what is practical. The example’s relatively large batch is an implementation choice, not a hardware requirement for every SupCon application.
Keras’s operations in the loss include normalization, matrix multiplication, equality masks, and reductions. If using a different backend, confirm that the Keras version and operations in your environment support the chosen backend and that the model and training pipeline run there consistently.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitchesFreeze the encoder and fit the classifier
Once contrastive pretraining is complete, use the trained encoder to produce image features. Freeze the encoder, attach the classification layers used for the downstream task, and train those layers with the class labels. Evaluate on a held-out test set using the same metric and protocol as any baseline. The projection head used to train the embedding is not part of the classifier path shown in the Keras example.
Best Value
- THE FASTEST WAY TO PHONICS MASTERY - Teach and Learn Phonics with Audio Sounds, learners get to see the spelling pattern and hear the related phonetic sounds. The audio reinforcement demonstrates the content and solidifies the learning quicker than flash cards and workbooks.
- PHONICS SYSTEM QUIZZES THEM IN 13 STEPS - The electronic phonics workbook starts with single letter sounds like a, b and c. This progresses through short and long vowel sounds, consonant digraphs, trigraphs, diphthongs, bossy R, silent letters and irregular phonics.
- TEST AND BUILD PHONEMIC AWARENESS - Our Educational Learn to Read Machine challenges them to find words which contain a particular phonetic sound or pick out phonetic sounds from the given vocabulary. All created with American English Audio.
- LEARNING THAT CHILDREN ENJOY - The Screenless Educational Tablet With Talking Flash Cards tests and quizzes children on their reading and phonics knowledge while correcting errors and compounding knowledge, all the while putting a smile on their face.
- UNLOCK YOUR CHILD'S POTENTIAL WITH BAMBINO TREE! - From numbers and pictures bingo to letter flashcards and phonics games, we offer a variety of learning materials and games for children with effective tested teaching strategies.
For a fair comparison against ordinary cross-entropy training, keep the dataset split, encoder architecture, augmentation, training budget, and evaluation metric aligned. SupCon also introduces choices such as batch composition, temperature, and projection-head design, so a difference in accuracy should not be attributed to the loss alone if other conditions differ.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What the Keras CIFAR-10 numbers mean
The Keras example reports 64.2% test accuracy for its conventional CIFAR-10 baseline and 72.98% for the classifier trained after supervised-contrastive pretraining. Keras says the comparison uses the same number of epochs. These are results reported in that maintained tutorial, marked last modified on 2026-08-19—not an independent reproduction, a universal expected gain, or a guarantee for a different dataset or architecture.
The same Keras example says large batches and multi-layer projection heads improve effectiveness and suggests SupCon may pay off for complex encoders and multi-class tasks with many labels. Those are the example’s guidance, not proof that SupCon always outperforms cross-entropy or that its listed hyperparameters are optimal. The authors’ paper reports 81.4% top-1 ImageNet accuracy for ResNet-200, 0.8 percentage points above the best number then reported for that architecture; that paper benchmark should not be combined with the tutorial’s CIFAR-10 results.
Recommended Free Tools
Quick Recap
Practical constraints and evaluation
- Labels: SupCon requires labels to define positive pairs. If labels are unreliable or batches rarely contain multiple examples of a class, the contrastive signal may be weak or misleading.
- Memory and batch size: pairwise similarities scale with the number of examples in a batch, so memory use can constrain batch size. Choose a batch size that fits the available compute and still gives useful positive pairs.
- Backend and execution: the example notes JAX, TensorFlow, and Torch backend options. Its notebook can be run locally or in a hosted notebook environment; Keras notes that Google Colab offers cloud runtimes with GPU and TPU options, but no particular provider or paid service is required.
- Validation: use a validation set for choices such as temperature, batch strategy, and training duration, and reserve the test set for final evaluation. Do not assume the tutorial’s test accuracy predicts performance on your own task.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
