At the recent closed-door seminar titled "Intelligence Reshaping Banking: Exploration and Challenges," Sun Li, General Manager of the Testing Center at BANKCOMM, shared her insights on the challenges of AI testing and introduced the bank's practical experiences in this area. She noted that the layering of AI applications across multiple stages could potentially amplify errors, suggesting that the future requires not only consideration of "human-machine collaboration" but also exploration of "machine-to-machine collaboration."
First, bank software testing faces dual challenges. On one hand, the deepening of digital transformation and the explosive growth in software system scale have significantly raised the bar for software quality, expanding the scope of testing from functional tests of single systems to end-to-end complete process testing, as well as encompassing performance, data, user experience, and security domains. Concurrently, the demand for faster iteration speeds and lower error tolerance constitutes the current practical challenges in testing. On the other hand, the financial industry demands high certainty in test results, whereas the accuracy of AI models is inherently probabilistic, creating a reliability gap between probabilistic models and deterministic requirements. The layering of AI applications across multiple stages could potentially amplify errors, necessitating not only a focus on "human-machine collaboration" but also future exploration into "machine-to-machine collaboration." Furthermore, it is essential to ensure AI applications meet requirements for fairness, security, and ethical alignment, addressing issues such as data bias, discrimination, and lack of explainability.
Second, the dual-drive approach of "using AI to test AI." One aspect is "AI for testing," which involves leveraging AI to empower software testing. Last year, BANKCOMM launched an artificial intelligence training camp, which has already implemented three assistants for test review, test outline generation, and test case generation. The average acceptance rate for these implemented assistants is 30%-40%, and can reach 70% for systems with highly structured and standardized requirements. Currently, efforts are also underway to develop a batch of specialized testing agents and establish human-machine collaborative workflows. The other aspect is "testing AI," which refers to the evaluation and verification of the "determinism" of AI systems. Specifically, this is advanced through three directions: building a system, setting objectives, and establishing a toolchain. First, by referencing peer experiences, relevant standards, and best practices to focus on AI characteristics for building a testing system and enhancing testability. Second, by implementing a layered, progressive verification strategy and exploring the establishment of an AI testing indicator system. Third, by gradually building and refining the toolchain for AI evaluation, establishing a closed-loop feedback and monitoring mechanism for model iteration.
The "Fintech 50 Forum" is celebrating its eighth anniversary. We extend special thanks to our enthusiastic readers for their support. We have established a "Fintech · Reader WeChat Group" to release research findings, foster communication and cooperation, and offer Fintech books as gifts. Your participation is welcome. To join the group: Add the group administrator as a WeChat friend (WeChat ID: serenazys). When adding, please provide your personal name (real-name verification) + organization + position + "join group" as a note. After review by the group administrator, you will be invited to join the group. We welcome our reader friends to leave comments and interact with us. For academic research, conference cooperation, and specialized training inquiries, please contact: Office Phone: 010-62561700 Email Address: songzeying@cft50.com.cn Fintech 50 Forum (CFT50): The Fintech 50 Forum (CFT50) was established on April 22, 2017. In recent years, the forum has closely focused on the frontiers of fintech theory, practice, and policy, actively promoting exchanges and cooperation among government, industry, academia, research, and application sectors, and has gradually developed into an influential independent academic platform in the field of financial technology. The forum has organized or participated in producing important academic works such as the "China Fintech Operation Report," "Research Report on Fintech Development of Small and Medium-sized Banks," "Data Elements: Open Lectures for Leading Cadres," "Green Finance: Assisting Carbon Peak and Carbon Neutrality," "15 Lectures on Fintech," and "20 Lectures on Banking Digital Transformation." It has also collaborated with the journal "Financial Theory Exploration" to open a digital finance academic column, which has been highly praised by policymakers, academics, and industry professionals. Since 2019, the forum has participated in establishing institutions such as the Beijing Dongcheng Liding Finance and Development Institute and the Xiamen Lujiang Fintech Research Institute.
For massive information and precise interpretation, all can be found in the Sina Finance APP.
Comments