Background
Recently I was browsing the Reddit/LLMDevs and came across the following post. This is very valid and timely question that is begging for an answer.
Hi, I am wondering if knowledge cutoffs for LLMs could be something that would be able to be updated to the day/ month eventually?
The reason I’m asking is that right now , in order to have updated data, we call bing or whatever else to get snippets of web searches to supplement our data.
What if, LLMs can be trained to the day? Wouldn’t this eliminate the web search tooling ?
The short answer to this question is Yes, but let’s dive a bit deeper to understand it better.
Problem to solve
Imagine a high school student who was dropped in a remote isolated and uninhabited island in year 2000. He survived on the natural resources available on the island. Due to their isolation, their knowledge of the world would be frozen in time. Any question about events that occurred after the year 2000 would be impossible for them to answer.
LLMs are similar to the high school student rescued from the isolated island.
Just as the isolated high school student’s knowledge is limited to what they learned before their isolation, an LLM’s understanding is confined to the information it was trained on. While the student might be able to reason based on their existing knowledge, they’d be unable to answer questions about events or developments that occurred after their isolation. Similarly, an LLM can only provide responses that are grounded in the data it was trained on. If you ask it about a recent event or a piece of information that it hasn’t been exposed to, it’s likely to generate an incorrect or nonsensical response.
An LLM’s knowledge cutoff refers to the last date on which the data used to train the model was collected. Since pre-training of LLMs is not a trivial task (high cost, and time needed), it cannot be carried out frequently to address this challenge. Depending on the use case RAG may or may not provide the
Solution
How do we humans stay up to date? Well, answer is simple, we get a continuous stream of information that we process and memorize. We are then able to use the latest and greatest information to answer questions and engage with world effectively.
Similar to humans, LLMs can learn from a continuous stream of information through a process known as continuous training. This approach aims to equip LLMs with the ability to adapt to evolving knowledge over time.
This is an evolving area of research that has started to show very promising results recently. Research in this area falls into three categories:
Continuous pre-training
Involves updating the LLM’s foundational knowledge with the latest information. This could include daily or hourly updates with current events or domain-specific information, such as the latest medical procedures for a medical LLM.
Continuous instruction-tuning
Focuses on refining the LLM’s ability to perform specific tasks. By regularly incorporating new data, fine-tuned models can stay current and effective. For instance, a fraud detection model can be improved by continuously ingesting the latest transaction data to adapt to emerging fraud patterns.
Continuous alignment
LLMs must constantly adapt to changing societal values, norms, and ethics. People have diverse and evolving preferences, so LLMs need to adapt to these changes. This requires continual alignment, including updates to reflect new values and the inclusion of diverse perspectives.
Challenges
Catastrophic forgetting
A major challenge with continuous learning is that models often forget what they’ve learned when they’re trained on new data. This is called catastrophic forgetting. For example, a model might become very good at recognizing widget-X, but when it’s trained to recognize widget-Y, it might start to forget how to recognize widget-X.
Resource needs
The computational cost of training and updating LLMs is a major barrier. While advancements in hardware and software are helping to address this issue, it remains a significant challenge, especially for large-scale models.
Update efficiency
The time required for training and updating LLMs can limit the frequency of updates. Researchers are working on more efficient training algorithms and hardware optimizations to reduce the time-to-update.
Summary
By enabling LLMs to stay up-to-date with the latest information, we can improve their accuracy, relevance, and overall usefulness in a wide range of applications.
Continuous training a.k.a. continuous learning is an active field of research. while significant progress has been made, there’s still a gap between the current state of technology and the goal of real-time LLM updates.
As research progresses, we can expect to see advancements in techniques that address catastrophic forgetting, reduce computational costs, and improve update efficiency.
