No. Full retraining is a common way to make substantial changes to an AI model, but it is not required for every update. Fine-tuning, replaying older examples, targeted edits, or retrieval from an external knowledge store can handle some changes without rebuilding the model from scratch. Each option has limits: updates can weaken old capabilities, require access to old data, or change only a narrow part of what the model knows.
Why do teams retrain models from scratch?
A model’s parameters are shared across the tasks and examples it has learned. Training it on new data changes those parameters; changes that improve performance on the new data can interfere with behavior learned earlier. This is one reason teams often combine old and new training data and train a replacement model, rather than repeatedly updating the existing one.
In its 2024 paper Loss of plasticity in deep continual learning, published in Nature, the authors describe discarding the old network and training a new one on both old and new data as the most common strategy for incorporating substantial new data. This approach can preserve a broad training mix when that data is available, but it requires another training run and access to the historical examples or an adequate substitute.
What goes wrong when a model keeps learning?
Catastrophic forgetting weakens earlier skills
Catastrophic forgetting occurs when training on later tasks or data harms performance on earlier ones. A model may improve on its latest examples while becoming less reliable on capabilities it previously had. The problem is especially difficult when the original training examples cannot be retained or reused: the model then has less direct evidence with which to preserve its earlier behavior.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
#1 Best Overall
- High-Performance AI Processor：The MS-02 Ultra features an Intel Core Ultra 9 285HX (24C/24T, up to 5.5 GHz, 13 TOPS NPU), delivering fast and efficient performance for AI inference, algorithm development, and media workloads. A PCIe x16 expansion slot supports desktop-class GPU upgrades for advanced model training and accelerated computing tasks. It's ideal for creators, engineers, and teams handling intensive parallel workloads.
- 4 × M.2 PCIe 4.0 + 4 × DDR5 SODIMM slots:Four DDR5 SODIMM slots support up to 256 GB of memory, while ECC helps maintain data integrity in mission-critical environments. Four PCIe 4.0 M.2 slots support up to 24 TB of storage, supporting RAID 0/1/5/10, combining high-speed performance with data protection. It allows for the creation of independent scratch disks, media libraries, and project drives, providing high-throughput for production workflows.
- PCIe & USB 4.0 v2: Up to three PCIe slots can be equipped, including a dual-slot x16 GPU. The main slot supports PCIe 5.0, meeting the needs of high-bandwidth creative and computing workloads. USB 4.0 v2 (80Gbps) supports high-bandwidth external storage and displays.
- Ultra-fast Networking: Wi-Fi 7 further enhances wireless performance with next-generation speeds and low-latency stability. Intelligent bandwidth switching optimizes throughput in different network environments, ensuring optimal performance for enterprise or local networks. Dual 25GbE ports (providing up to approximately 3.125 GB/s bandwidth, about 25 times faster than traditional 1GbE), enabling seamless large-scale file transfers and parallel computing. 10GbE and 2.5GbE ports, with support for Intel vPro technology, ensure enterprise-grade remote management and deployment flexibility.
- Server-grade thermal architecture: Utilizing a dedicated CPU/GPU airflow design, equipped with a 6-pipe dual-fan cooler, it maintains stable performance even under sustained loads, delivering up to 140W Turbo power while maintaining a 100W TDP, and operating with noise levels as low as 36 dB. An integrated 350W power supply ensures stable and reliable output for demanding computing tasks and fully loaded extended configurations.
Loss of plasticity makes future learning harder
Loss of plasticity is a different failure. It means a network becomes less able to learn as training continues, rather than simply performing worse on old examples. The Nature paper studies this in continual-learning experiments using ImageNet and CIFAR-100 settings, where new classes arrive over time. The distinction matters: preventing forgetting does not necessarily ensure that the model will remain easy to train on still newer tasks.
What are the alternatives to a full retrain?
There is no universal replacement for retraining. Continual-learning surveys group methods according to the compromises they make in retained performance, access to past data, compute, and system complexity. The options below address different kinds of updates; they are not interchangeable guarantees.
| Approach | What changes | Retention and new capability | Main trade-offs |
|---|---|---|---|
| Fine-tuning | Continues training from an existing checkpoint. | Can adapt the model to new data or tasks, but updates to shared parameters can interfere with earlier capabilities. | Can avoid starting from random initialization, but retention depends on the training setup and data. It is not a guarantee against forgetting. |
| Replay | Mixes examples from earlier tasks or data with new examples during training. | Gives the model direct reminders of prior behavior while learning the new material. | Requires access to suitable historical examples and adds data-handling and training costs. Keeping or reusing old data may also be constrained by privacy or governance requirements. |
| Regularization or consolidation | Constrains changes to parameters considered important to earlier tasks. | Aims to protect previous capabilities while allowing some learning on new data. | Must balance protecting old behavior against adapting to the new task; it does not make every update risk-free. |
| Knowledge distillation | Trains an updated model to reproduce behavior from an earlier model, often alongside new learning. | Can help preserve earlier behavior while adding a task. Amazon Science’s 2021 work proposes a distillation-based approach for continual learning of new natural-language tasks. | Requires an earlier model and a suitable way to capture its behavior; the approach still involves training and does not eliminate trade-offs. |
| Targeted model editing | Changes a narrow fact or response rather than broadly retraining the model. | Can be appropriate for a specific correction, but a targeted edit is not the same as teaching a general capability. | Its scope is narrow. Microsoft Research describes caching and selectively retrieving new transformations between model layers as one editing approach. |
| Retrieval or external memory | Updates information available to the model at answer time, rather than encoding every change in its base weights. | Can supply changing or newly added information without retraining the underlying model for each item. | Refreshes accessible information, not the model’s underlying reasoning. The answer still depends on whether the system can retrieve and use relevant material. |
| Modular approaches | Add or update components for particular tasks rather than changing one shared model for everything. | Can isolate some changes to a particular function or task. | May introduce integration and maintenance complexity; whether it preserves other capabilities depends on the system design. |
| Full retraining | Trains a new model using a broad training mix, potentially including old and new data. | Can address broad changes to data or objectives in one training process, rather than relying on a narrow patch. | Requires substantial compute, evaluation, and access to the training data. It is a broad reset, not proof that every previous behavior will be retained. |
Why can’t ChatGPT simply learn every new fact?
“Learning” can mean different things. A system can be given access to new information through retrieval or external memory without changing its base model weights. That can make changing facts available more quickly, but it does not mean the model has incorporated them into its underlying parameters or acquired a general new skill. Updating the model itself is a training or editing problem, with the retention trade-offs described above.
For a narrow correction, targeted editing may be more appropriate than broad retraining. For frequently changing information, retrieval can avoid a training run for every update. For a substantial new capability or a major shift in the data or objectives, broader training may be more suitable. The right choice depends on what must change and what old behavior must remain dependable.
Rank #3
- Professional AI & Creator Workstation: AMD Radeon AI PRO R9700 GPU with 32GB GDDR6 is engineered for AI development, professional content creation, and compute-intensive workloads.
- Massive 32GB Memory Capacity: 32GB of GDDR6 memory on a 256-bit bus provides ample bandwidth for large AI models, 8K video editing, and complex 3D rendering.
- Advanced RDNA 4 with AI Accelerators: 64 Compute Units with 3rd Gen Ray Tracing and dedicated 2nd Gen AI Accelerators for groundbreaking AI performance and visual computing.
- Professional Blower Cooling: Efficient single blower design exhausts heat directly out of the chassis, ideal for multi-GPU workstation and server configurations.
- Enterprise-Grade Thermal Solution: Vapor chamber heatsink with industrial Honeywell PTM7950 thermal interface material ensures reliable cooling under sustained professional loads.
How much does retraining a large model cost?
There is no universal price for retraining a commercial model in the sources cited here. Cost depends on factors including model size, the volume of training tokens, hardware, run duration, energy use, evaluation, and engineering overhead.
The authors of the 2024 Nature paper write: “When the network is a large language model and the data are a substantial portion of the internet, then each retraining may cost millions of dollars in computation.” This is an order-of-magnitude warning about a particular scale of retraining, not a price quote for every model or update.
Rank #4
- FAST RUNS IN THE FAMILY — The 16-inch MacBook Pro with the M5 Pro or M5 Max chip brings next-generation speed and powerful on-device AI to personal, professional, and creative tasks. With all-day battery life, double the starting storage,* and a breathtaking Liquid Retina XDR display, it’s pro in every way.*
- BUCKLE UP — Along with a next-generation CPU, faster unified memory, and up to 2x faster SSD storage,* M5 Pro and M5 Max feature a more powerful GPU with a Neural Accelerator built into each core, delivering faster AI performance and on-device training capabilities. So you can blaze through demanding workloads at mind-bending speeds.
- BUILT FOR AI — Apple silicon, and every major component that powers it, is designed to run demanding on-device AI workloads like LLM inference and training. And Apple Intelligence helps you write, express yourself, and get things done effortlessly with groundbreaking privacy protections at every step.*
- ALL-DAY BATTERY LIFE — MacBook Pro delivers the same exceptional performance whether it’s running on battery or plugged in.*
- MACOS RUNS APPS FAST — All your go-to apps run lightning fast in macOS, including built-in apps like FaceTime and Messages. Plus, built-in virus protection and free software updates help keep your Mac running smoothly and securely.
Does making a model larger solve the update problem?
Google Research reports that larger pretrained ResNets and Transformers are more resistant to catastrophic forgetting than randomly initialized models trained from scratch, and that resistance improves with model and pretraining-data scale. That result suggests that pretraining and scale can help retention; it does not establish that larger models avoid forgetting, remain plastic indefinitely, or can absorb every update without further training.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How should a team choose an update method?
The decision starts with the scope of the change. A new fact, a new task, and a major change in the model’s data distribution are different update problems. Before choosing a method, teams should establish what must remain reliable and what data and compute are available.
Free tools Windows power users keep installed
One-click scans. No signup required.
Best Value
- 【High-Performance APU】The MS-S1 MAX features an AMD Ryzen AI Max+ 395 APU, integrating a Zen 5 architecture CPU (up to 5.1GHz, 16C/32T, 64M L3 Cache), an RDNA 3.5 GPU, and an NPU (50 TOPS). The total system output is 126 TOPS. It provides powerful parallel computing capabilities for demanding AI workflows. It is ideal for running local LLMs, multimodal models, and computationally intensive tasks
- 【128GB UMA Memory】Equipped with up to 128GB of LPDDR5x-8000MT/s unified memory, it enables the CPU and GPU to access a shared, high-bandwidth memory pool with extremely low latency. Ideal for large-scale AI inference, 3D workloads, and complex timelines in video editing. It eliminates traditional VRAM bottlenecks, ensuring smoother data transfer during high-intensity computations. The UMA design maximizes performance stability under high loads
- 【Flexible Expansion】The MS-S1 MAX features USB4 V2 (up to 80Gbps), dual 10GbE LAN, HDMI 2.1 (up to 8K60), a full-length PCIe x16 expansion slot, and dual M.2 slots supporting up to 16TB RAID 0/1. Wi-Fi 7 provides stronger signal coverage and a more stable wireless experience. The slide-out design facilitates upgrades and maintenance. It easily adapts to personal, studio, or rack-mount enterprise environments
- 【High-Efficiency Cooling System】Utilizing an aerospace-grade aluminum alloy chassis, copper base plate, six heat pipes, dual turbine fans, and advanced PCM thermal conductive material, it maintains stable cooling performance even under continuous load. This system supports 130W continuous power and 160W peak power operation, with a built-in 320W power supply. It boasts multiple global certifications including CCC, FCC, UL, CE, and UKCA, ensuring stable and reliable operation in various environments
- 【Cluster Design】Two MS-S1 MAX units can be configured as a dual-unit cluster to run a large 235B Q4 model locally, achieving an output speed of 10.87 tok/s. Supporting 2U rack deployment, multiple MS-S1 MAX units can be cascaded into a distributed cluster to create a high-efficiency AI computing center. A cluster of four MS-S1 MAX units successfully ran a DeepSeek-R1 671B Q4 large model. A reserved cluster power-on interface allows for unified start-up and shutdown
- For changing reference information: consider retrieval or external memory if the goal is to make updated material available at answer time.
- For a narrow factual correction: consider targeted editing, while checking that the edit has not caused unintended changes elsewhere.
- For a new task with prior examples available: replay or distillation can help preserve earlier behavior while training on the new task.
- For a broad change in data, architecture, or safety objectives: a new training run may be justified when narrower updates cannot meet the required scope.
Whichever route is used, evaluate both the new behavior and the capabilities the model is meant to retain. Also assess compute and memory needs, whether historical data can legally and safely be reused, how quickly the update can be deployed, and how the change can be audited, rolled back, or handled if it must later be removed. These are separate requirements: a method that is fast to apply may be narrow, while a method that better preserves past performance may require more data and training.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
