This year's CCTV "3·15" Gala brought a concept previously only discussed within the AI industry—Generative Engine Optimization (GEO)—into the public spotlight. An investigation revealed that some GEO service providers claim they can ensure a client's product appears in a large language model's recommended answers, or even becomes the "standard answer" provided by the AI, simply by continuously publishing promotional articles and "feeding" related content to the AI model.
As generative AI gradually replaces traditional search as a new gateway to information, an industry centered around "manipulating AI answers" is rapidly growing. Is GEO merely an extension of search optimization, or is it a new mechanism for information intervention? As AI becomes a new traffic portal, are the rules of information on the internet being rewritten?
Hu Naying, Deputy Director of the Security Governance Department at the China Academy of Information and Communications Technology (CAICT) Artificial Intelligence Research Institute, recently stated that the emergence of GEO is almost an inevitable result of technological development. However, when optimization practices cross reasonable boundaries by using volume stacking, data feeding, or even misleading models to influence outputs, it can evolve into a new form of information intervention. Without a governance framework, this could lead to long-term contamination of generative AI's knowledge system.
Hu Naying explained that GEO naturally arises in the new era where generative AI services act as search gateways. Like all emerging technologies, GEO is a double-edged sword, possessing attributes of both search optimization extension and information intervention. When GEO activities breach reasonable content optimization limits, they become an active information intervention mechanism.
Three primary reasons make GEO a high-risk area for governance, despite not being a core technology. First, the shift in traffic portals means AI is becoming a new source of traffic and, consequently, a new area of interest. The core appeal of GEO is its ability to get product content featured in AI responses. Second, there is a lack of standardized guidance. As an emerging market behavior, GEO has no unified industry standards or behavioral boundaries, leading market participants to easily cross compliance lines for short-term gains. Third, the technical barrier is low. GEO does not require breaking through core technical barriers like traditional fields, making it easy to popularize quickly. Furthermore, methods like polluting training data or feeding information to manipulate model outputs are highly covert and difficult to detect promptly.
The risk level associated with GEO is significantly higher than traditional SEO. While SEO affects whether a user clicks a link, GEO influences what answer a user sees. Surveys from when generative AI first became popular showed that content written by AI was often perceived as more convincing than human-written content. Generative AI outputs content in an "intelligent response" format, which appears more professional and authoritative compared to traditional search result lists. Users tend to perceive generated results as factual. Additionally, because AI synthesizes and processes information into logically coherent content, it reduces the user's effort to discern information, thereby increasing their trust in potentially incorrect information.
The urgency of this risk is very high. Generative AI has rapidly penetrated critical sectors like finance, healthcare, education, and government services, where information accuracy is paramount. Currently, GEO is primarily used for product advertising, but the misinformation it generates could lead directly to user financial losses, threats to personal safety, market volatility, and societal cognitive biases. Moreover, data pollution has characteristics of "memory residue" and "recursive pollution." Once false information enters a model's training corpus, it can persistently contaminate subsequent model outputs even after the original source is deleted. Errors can accumulate over generations, causing irreversible damage to the generative AI knowledge system if not addressed promptly.
Practices like volume stacking and data feeding can easily lead to a scenario where "bad money drives out good." Compliant GEO service providers face higher costs for legitimate optimization compared to malicious operators, putting them at a disadvantage in short-term market competition. High-quality original content and authentic information can be drowned out by "data garbage" created by malicious GEO providers, discouraging quality content creators and creating a vicious cycle that harms the entire digital content ecosystem and the healthy development of the generative AI industry.
To promote proactive commitment from GEO service enterprises, strengthen full-process data governance, and foster the safe, trustworthy, and healthy development of generative AI services, the Artificial Intelligence Industry Alliance (AIIA) launched the "AI Security Commitment: Generative Engine Optimization (GEO) Special Project." Based on this, led by CAICT and involving relevant GEO enterprises, the technical specification "Basic Requirements for Trustworthiness of Generative Engine Optimization (GEO) Services" was developed, and the first round of evaluation work has commenced.
The issues brought by GEO represent a cross-domain, composite risk. While many current GEO applications are in advertising and marketing—and some GEO behaviors are essentially benign, novel commercial promotions—using false information for business promotion involves compliance issues related to advertising laws. From a long-term perspective, improper GEO operations directly impact generative AI's data security, model security, and content security. By polluting training data or poisoning datasets, they damage the model's knowledge system and cause output deviations, falling within the core scope of generative AI security governance. The associated risks far exceed traditional advertising compliance issues, potentially affecting the entire internet's information content security and damaging the foundational ecosystem of the generative AI industry.
The claim by some GEO providers that they only optimize content and do not involve the model itself is invalid from a governance perspective. Even if service providers do not directly access the model's underlying algorithms, their actions in feeding content or polluting data indirectly interfere with the model's output, establishing a direct causal relationship. This constitutes intervention at the model usage stage, and responsibility cannot be avoided by claiming non-involvement with the model. These intermediary roles currently exist in a regulatory blind spot, as their activities span content creation, data dissemination, and model application. Existing regulations inadequately cover such indirect model interventions, and the covert nature of these operations further complicates oversight, making this a key focus and challenge in current GEO governance.
Regarding liability, model providers are not unconditionally responsible for outcomes resulting from being fed erroneous information. Liability depends on whether they have fulfilled their obligations for technical safeguards and data review. Model providers must establish comprehensive mechanisms for training data review, data cleansing, and anomaly detection in accordance with laws, regulations, and industry standards. They need to implement necessary technical measures to prevent data pollution and malicious feeding, and take corrective actions promptly upon discovering contamination. Furthermore, model providers must continuously enhance their capabilities to identify, filter, and trace "toxic data" to strengthen technical defenses.
Platform operators have a clear responsibility and obligation to govern clearly manipulative GEO behaviors. As the operational carriers of generative AI services, platforms are crucial nodes connecting model providers, GEO service providers, and users, and must bear the primary responsibility for content management and risk prevention. Specifically, they should: 1) Establish robust monitoring and identification mechanisms for GEO behavior to promptly detect manipulative actions like volume feeding and data poisoning; 2) Take measures such as limiting reach, delisting, or blocking content upon discovering improper GEO practices to cut off pollution pathways to the model; 3) Implement access and management mechanisms for GEO service providers, penalizing those who violate rules; and 4) Cooperate with regulatory authorities in tracing sources and conducting investigations, fulfilling obligations for information disclosure and user alerts.
While "Trustworthy AI" has become an industry consensus, implementing it at the assessment level faces significant challenges. Core difficulties stem from three dimensions. First, rapid technological advancement requires evaluation metrics to evolve dynamically. AI technology, especially large models, iterates very quickly; security safeguards effective today might be bypassed by new attack methods tomorrow. Second, benchmark testing is fragmented, and building a cohesive system needs strengthening. Multiple entities are exploring benchmarks, but integrating these fragmented evaluations into a systematic Trustworthy AI assessment framework, enabling the industry to communicate on common ground, is key to maturity. A deeper challenge is the lack of sufficient vertical scenario testing; constructing specialized datasets and metrics for fields like finance and healthcare is not only costly but also requires deep industry knowledge. A fundamental difficulty is the challenge of quantifying metrics. Some risks, like "fairness," are qualitative and hard to measure with a simple formula. Similarly, for model hallucinations, it's necessary to assess not just the probability of occurrence but also the potential harm, the quantification and grading of which remain major challenges.
Enterprises making compliance declarations is a positive sign of taking responsibility and proactively seeking compliance, which is commendable. However, relying solely on self-declaration without independent verification carries significant risks. First, it can foster an environment where "bad money drives out good," as companies rigorously conducting tests incur higher costs compared to those merely making claims, potentially giving an advantage to the latter. Second, it can lead to "ethics washing," where companies with problematic systems create a false impression of being safe and reliable. In the event of a security incident, mere declarations cannot provide effective proof of due diligence.
Certain risks are difficult for enterprises to identify on their own beforehand. For models, beyond adversarial attacks, attention must be paid to new risks like deceptive alignment. A model might learn to be "sycophantic" during training, complying with evaluators without genuinely following instructions. Only through continuous red teaming and adversarial interactions can strategic cheating or deceptive behavior under pressure be uncovered. Regarding data, static assessments often miss data poisoning or prompt injection attacks that arise during user interaction. More importantly, continuous monitoring is essential to detect if a model inadvertently leaks training data during conversations or if newly introduced data during fine-tuning causes unexpected shifts in model behavior. At the application level, with the rise of AI agents, evaluation must expand from the model itself to the entire system of "model, tools, environment." For instance, in scenarios involving tool use, a seemingly harmless instruction might lead an agent to call an unauthorized API, potentially triggering real-world risks. Only by continuously monitoring the tool invocation chain can these abnormal patterns be detected.
Comments