Search results:

Back

Beyond Prompting: What CL-Bench Reveals About Context Learning

The shift from better prompts to better information environments exposes a deeper constraint: today's models still struggle to learn and apply genuinely new knowledge from context.

AI Summary

The shift from better prompts to better information environments exposes a deeper constraint: today's models still struggle to learn and apply genuinely new knowledge from context. Over the past two years,prompt engineeringhas become one of the industry's most overused terms. Teams have studied how to write prompts, design few-shot examples, and guide models through multi-step reasoning. These methods formed the shared foundation for working with language models.

Over the past two years,prompt engineeringhas become one of the industry's most overused terms.


Teams have studied how to write prompts, design few-shot examples, and guide models through multi-step reasoning.


These methods formed the shared foundation for working with language models.

We debugged natural language as if it were code, refining the question in an effort to move the model closer to the correct answer.


More recently, another term has moved to the center of the discussion:

context engineering.


The term makes a practical problem explicit.

When models enter complex systems, connect to real data, and become embedded in business processes, many failures result not from weak model capability but from a disorganized information environment.


Even when information is prepared carefully, however, the model may still fail to absorb and apply it correctly.


The central question is shifting fromhow to express the problemtohow to construct the environment.


01

From writing prompts to designing context



Prompt engineering addresses alignment at the language layer.


  • How should a question be phrased so the model understands the expected result?
  • How should steps be decomposed to reduce skipped reasoning?
  • How can examples make the output style more consistent?


At its core, prompt engineering optimizes theconversation interface


between a person and a model. Context engineering asks a different question: what information environment should surround the model before it begins generating an answer?


Consider a simple example.

Suppose you ask AI to reply to an email:

'Do you have time for a quick meeting tomorrow?'


A system that relies only on the prompt sees nothing beyond that sentence.


It may return a reasonable response:

'Thanks for your message. I am available tomorrow. What time works for you?'


A system supported by context engineering performs several invisible steps first.

It checks your live calendar to identify open time slots.

It reviews previous communication with the sender to understand the relationship and preferred tone.

It determines whether the sender is a colleague or a customer.

It may even call a meeting tool and create an invitation with a link.


The final reply could be:

'I am free at 3:00 p.m. tomorrow. I have sent a meeting link; shall we talk then?'


The resulting response has a fundamentally different level of information density and immediate utility.

The difference comes from the dimensions of information available to the model.


That is the core of context engineering:

building a dynamic, structured, and actionable information environment around the model.


02

More information ≠ More learning


A common misconception is that

context engineering simply means connecting more data and feeding more history into the model.


A recent paper introducing CL-Bench, published jointly by Tencent's Hunyuan team and Fudan University,

directly evaluates a model's ability to learn new knowledge from context.


The benchmark contains500 complex contexts, 1,899 tasks, and 31,607 evaluation criteria, all developed with experienced domain experts.


It divides the tasks into four categories.



Domain knowledge reasoning

The context provides specialized knowledge in a particular domain, such as a fictional legal system, a newly designed financial instrument, or niche professional material.

The model must learn that domain knowledge from the context and apply it to a concrete task such as a legal judgment, financial analysis, or professional recommendation.



Rule-system application

The context defines an entirely new and clearly structured rule system, such as a new game mechanic, formal mathematical system, programming syntax, or technical standard.

The model must understand and apply the rules correctly to complete game simulations, mathematical proofs, code implementations, or regulatory interpretations.



Procedural task execution

The context provides a complex process or operating guide, such as a product manual, software documentation, or meeting-coordination procedure.

The model must learn the required steps and follow the process to troubleshoot a problem, provide operating instructions, or coordinate a workflow.



Empirical discovery and simulation

The context provides experimental data, observations, or a simulated complex system.

The model must infer patterns from the data or analyze and reason within the simulation. These tasks emphasize inductive reasoning and form the benchmark's most difficult category.



CL-Bench context categories


Every test task satisfies three strict conditions:

  • the required knowledge is absent from the model's pretrained parameters,
  • it must be acquired from the context provided in the current task,
  • and the model must apply that knowledge correctly.

The results are revealing.


The researchers evaluated ten leading language models.

The average task success rate was only 17.2%.


Even the strongest result, from GPT-5.1 High, reached just 23.7%.



All models were evaluated in reasoning mode.

Results are reported as the mean plus or minus standard deviation across three runs.


What does this tell us?

Every task included all the information required for a solution, yet the models still failed on the large majority of assignments.


Providing information is not the same as absorbing it. Supplying the right context does not guarantee a correct result.


CL-Bench exposes a fact that has received too little attention:


current language models behave more likeretrievers of parametric memorythanadaptive learners within an environment.


They are highly capable at using static knowledge established during pretraining,

but far less mature at learning immediately from new information in the current input.


03

The Inductive Reasoning Gap


One observation in the paper deserves particular attention.

When the context defines explicit rules and the model can reason deductively from them, performance is relatively acceptable.


Performance falls sharply when the model must

discover patterns in experimental data, extract recurring behavior from logs, or construct abstract rules inside a simulation.


Success rates decline substantially.


Inductive reasoning requires the model to construct a new structure rather than match an existing pattern.


That requirement exposes the current boundary of large language models.


They are effective at generation and recombination,

but remain inconsistent when reorganizing rules or constructing new conceptual structures.



04

Updating Knowledge Through Learning



Another overlooked problem isknowledge conflict.


When the context introduces a new rule that resembles, but differs from, something learned during pretraining,the model often gives priority to knowledge already stored in its parameters.


Even when the current context provides an explicit updated definition, thisparametric inertiacan still dominate the reasoning path.


The issue is not insufficient information. It reflects the influence of the model's internal weight distribution.


Training establishes a relatively stable knowledge structure and set of priorities.

At inference time, contextual information must override or adjust that structure.

That is inherently difficult.


Several implications follow:


  • a model's internal knowledge system has persistent, fixed characteristics,
  • contextual information must compete with parametric memory for control of the reasoning path,


and context engineering must schedule and reprioritize information rather than merely increase the amount of input.


To some extent, it changes the path of reasoning, not just the size of the prompt.



Distribution of error types across models


Current contextual learning is also temporary.

Even when a model successfully absorbs a new rule during one conversation, that knowledge disappears when the context window is cleared.


The model cannot truly accumulate experience, transfer skills across assignments, or continue developing over multiple conversations.


In-context learning currently functions astemporary working memory,notdurable knowledge consolidation.


That helps explain whymemory systemshave become a major research focus in 2026.


Without a way to preserve what is learned from context, the model remains a short-term reasoning system.


05

Conclusion



Prompt engineering improves control over expression.

Context engineering improves the organization of the information environment.

Context-learning capability determines whether a model can operate continuously inside a complex system.


Together, these three layers define a central challenge for the next phase of AI.


As AI moves from laboratories into real business systems, it no longer faces static, standardized questions. It encounters changing rules, live operational state, and tasks that evolve through repeated execution.


If context learning improves substantially, the human role in AI systems will change as well.


Competition will move fromwho can train the strongest modeltowardwho can supply the richest and most relevant context for the task.


A more distant challenge remains: even better in-context learning is still temporary. When the context window is cleared, the learning disappears. How can knowledge acquired from context become persistent?


Only when models evolve from memory retrieval systems into learners that adapt to their environments

will AI enter a genuinely new phase.


Talk About Digital Transformation?

Talk About Digital Transformation?

Discuss digital transformation with GeekOnUp

一同向上生长

Growing Upward Together

Let's Talk About Your Ideas

  • Custom APP Development
  • Custom Mini Program Development
  • Custom Web Development
  • AI Agent Development
  • Enterprise Digital Transformation
  • Other