At the 2026 Lujiazui Forum, Agricultural Bank of China Limited (ABC) Chairman Gu Shu addressed the key risks associated with large-scale AI models in financial applications during a session on technological innovation empowering high-quality financial development.
He identified several primary challenges: the "black box" nature of models, the phenomenon of "model hallucination," and the uncertainties stemming from models' capacity for autonomous thought and decision-making.
Key Risk Categories
Gu Shu elaborated on three major categories of risk. The first involves the massive number of parameters, which creates a significant challenge in explaining how models function. With mainstream models now containing hundreds of billions or even trillions of parameters, the complex matrix operations and deep non-linear layers render the decision-making process opaque and difficult to interpret.
The second risk stems from the probabilistic nature of model generation, which tests output accuracy. Unlike human linear reasoning, large models operate by statistically analyzing token probabilities from vast training datasets. This process is fundamentally based on probability, not factual deduction, making models prone to generating coherent but incorrect "hallucinations" when data or factual grounding is insufficient.
The third category involves new risks arising from models' ability to think and decide autonomously. As AI models evolve and intelligent agents are deeply integrated, they move beyond the traditional "fixed input-output" software paradigm. This autonomy can amplify risks related to uncontrollable processes and unpredictable outcomes.
Managing the Inevitable
Gu Shu emphasized that, given the operational mechanics of these models, such risks are an objective reality. The goal, therefore, is not to eliminate risk but to manage it. This requires a dual approach: fully leveraging the significant utility of large models while simultaneously acknowledging their limitations and establishing a governance system that coexists with and controls these risks.
ABC's Practical Approach
Discussing Agricultural Bank of China Limited's practical measures, Gu Shu outlined a four-pronged strategy. The first is to implement tailored, scenario-specific adaptations. For financial applications, different scenarios require different approaches. The bank establishes a tiered control mechanism for the model "black box," matching diverse technical pathways and explainability requirements to specific use cases.
For heavily regulated scenarios like credit decisioning, ABC employs model distillation technology. This allows the large model to assist with data synthesis and attribution analysis, transferring its capabilities to smaller, more interpretable models. For high-cognition scenarios like investment research, the focus is on enhancing chain-of-thought design. For product marketing, the approach is more flexible to fully harness the model's creative potential.
The second measure involves setting constraints and control benchmarks. To manage instances where models "hallucinate" or over-extend, ABC establishes parameters and benchmarks for different applications to control their operational pathways. This is combined with a human-in-the-loop approach, where human judgment remains the final decision-making authority.
For example, when using AI to assist in generating credit investigation reports, the bank sets business benchmarks. It employs model cross-checking, model reflection, and business data calibration for automatic content verification. Crucially, key output content is reviewed by business personnel to ensure "controllable results."
The third strategy is using AI tools to combat AI application risks. Gu Shu advocated for "fighting AI with AI" by building a multi-layered defense system. This includes deploying specialized security monitoring models to enhance AI defense capabilities, establishing a full-chain trusted verification and access mechanism to strengthen vulnerability scanning and operational status monitoring for supply chain security, and implementing defenses like prompt injection protection and output content filtering to prevent sensitive data leakage during model inference.
The fourth pillar is strengthening the bank's internal AI governance framework. Gu Shu stressed the need to健全 a clear, responsibility-based governance system that balances innovation with risk management. The aim is to preserve space for innovation while minimizing the limitations posed by model "black boxes" and "hallucinations."
In conclusion, Gu Shu stated that for Agricultural Bank of China Limited, the imperative is both to utilize large models effectively and to robustly guard against the risks inherent in their application.
Comments