The Evolution of xAI: From Chasing Anthropic to Providing it with Computing Power

Deep News06-05 23:52

This year has been one of significant turbulence for xAI, the artificial intelligence laboratory under SpaceX, marked by frequent changes in its executive and core personnel and delays in the development of its proprietary code-based large language model. The company has been forced to seek external assistance to narrow the technological gap with Anthropic.

Six industry insiders involved in related projects revealed that, largely unknown to the outside world, xAI has long been closely monitoring Anthropic, seeking ways to refine its own products by drawing on the competitor's technology. This competitive dynamic persisted even after Anthropic restricted access to its technology, ultimately leading to the current collaboration where xAI leases its scarce computing power resources to Anthropic.

Following a series of recent setbacks, the market has begun to question the future focus of the soon-to-be-listed SpaceX: will it be on deepening its in-house AI research, or on generating revenue by leasing out computing power? The substantial rental fees from Anthropic have effectively offset xAI's significant losses, which have been a major factor in the overall weakening of SpaceX's first-quarter performance.

However, with SpaceX preparing for what could be the largest IPO in history next week, the company is presenting itself to Wall Street as a high-growth AI investment. Its self-developed AI products for enterprise and consumer markets are positioned as a core growth segment for the group.

Elon Musk has stated multiple times on the X platform recently that the computing power cooperation with Anthropic is only a short-term arrangement. In a post on Tuesday, he explained that the short-term nature of the deal was due to "anticipating that at some point in the future we will need to reclaim the compute for our own use."

Musk founded xAI in 2023 with the goal of competing directly with leading AI firms like OpenAI. The project has raised approximately $42 billion in cumulative funding and amassed a vast pool of GPU computing resources. Yet, in the fiercely competitive arena of code-specific large models, xAI's shortcomings in catching up to OpenAI and Anthropic have been laid bare.

While xAI launched its conversational chatbot Grok in 2023, its first dedicated code model did not materialize until August of last year, by which time Anthropic and OpenAI had already launched commercially mature code tools built on their own proprietary foundations.

Around the same period, Anthropic began tightening access to its technology for competitors: in August of that year, it shut down OpenAI's API access, telling Wired magazine that OpenAI personnel had improperly used its code tools in violation of service agreements. OpenAI countered, arguing that using a competitor's API for benchmark performance testing was common industry practice.

Two individuals with knowledge of the projects disclosed that even after Anthropic blocked access for competitors like OpenAI, xAI continued to access output from the Claude model. It even spent months on a model distillation project, directly training its own code model on data generated by Claude.

In January of this year, xAI co-founder Tony Wu internally announced that Anthropic had formally banned corporate access to Claude. However, sources confirmed that some xAI engineers switched to using personal accounts to continue accessing Claude, after which Anthropic subsequently shut down a large number of associated personal accounts.

xAI also utilized the intermediary service Blackbox AI, accessing other AI models through its encrypted channels. Two industry sources stated that as recently as mid-May, xAI was still using this platform to access Anthropic's models for research and development work such as benchmarking. Blackbox AI did not respond to requests for comment.

Musk publicly admitted during the court proceedings with OpenAI that xAI did indeed "partially reference OpenAI models" when training Grok, calling the practice very common in the industry. However, neither Musk nor xAI have disclosed the extent to which the company used Anthropic's technology or the various workarounds employees employed to circumvent the bans.

Beyond facing increasingly blocked channels to access Anthropic's technology, xAI also suffered a major R&D incident in recent months. Project staff revealed that during a data migration, an employee accidentally deleted core training data for a code model, resulting in losses equivalent to two to three weeks of the entire team's R&D output.

xAI has not officially responded to requests for comment.

Bringing in External Resources

An individual involved in the project revealed that xAI originally planned to launch a new version of its general-purpose large model alongside a brand-new code model by the end of last year. When this plan failed, the company pivoted to integrating code capabilities into its developer platform, Grok Build, which was officially launched in May of this year.

With direct access to Anthropic's models becoming increasingly difficult, Musk began recruiting teams from outside to fill the gaps in code development. Two Tesla Motors engineers confirmed that earlier this year, the Tesla Motors technical team was among the first to receive an internal beta version of Grok Build to assist with product optimization and provide feedback.

The capital ties between Musk's affiliated companies also deepened: Tesla Motors announced a $2 billion investment in xAI at the end of January; just days later, SpaceX completed the acquisition of xAI at a $250 billion valuation. Following the acquisition, xAI experienced significant personnel upheaval, with the remaining eight co-founders collectively leaving, accompanied by layoffs and budget cuts.

In March, Musk publicly stated that Tesla Motors would collaborate with xAI on a smart office agent project called Macrohard, positioned as a product to replace white-collar labor. Three sources familiar with the matter said that in that month, technical executives from Tesla Motors and SpaceX frequently visited xAI headquarters to discuss cooperation. Teams from Tesla Motors' autonomous driving, AI infrastructure, and Dojo supercomputer divisions were subsequently stationed at xAI to support the Macrohard project and computing infrastructure development.

Musk later held an internal meeting, informing employees that he would bring in technical talent from Cursor and Mistral AI to fill R&D gaps and accelerate product delivery. He explicitly stated his apprehension about Claude's technological lead.

The meeting outlined a clear division of labor: Mistral, a French company specializing in customizable open-source models, would be responsible for model pre-training. The code assistant startup Cursor would handle post-training, refining the base model into a commercial product. xAI would provide the underlying computing infrastructure.

The meeting also disclosed that it was Anthropic that initiated discussions about computing power leasing. However, Musk emphasized to employees that xAI would not transform into a mere computing power leasing company.

But not long after, xAI's strategy shifted. In April, it officially announced a partnership, opening its own computing power to Cursor to support the latter's development of the Composer code model. The agreement included an option for SpaceX to fully acquire Cursor for $60 billion later this year. If SpaceX declines the acquisition, it would have to pay a $1 billion termination fee.

In May, xAI secured a major deal: Anthropic agreed to pay $1.25 billion per month to lease the majority of the computing power in xAI's data centers. The initial contract is locked for six months, after which it converts to a flexible monthly agreement that either party can terminate at any time.

On another front, preliminary discussions between xAI and Mistral did not materialize into a formal partnership. Sources said Mistral executives visited xAI headquarters for talks in mid-April, but negotiations stalled a few weeks later. Devendra Chaplot, a founding engineer from Mistral who had just joined xAI to lead pre-training efforts, left the company in May. Mistral, Anthropic, and Cursor all declined to comment.

Since the Cursor partnership began, Cursor CEO Michael Traurig has been regularly stationed at xAI's offices, with the two companies holding frequent meetings to synchronize R&D progress and exchange core technical details. Following the implementation of this partnership, xAI initiated a new round of engineering layoffs.

An internal source revealed that as of mid-May, xAI's pre-training team had fewer than five people remaining, compared to over 20 a year ago. Within a few short months, the Grok Code project cycled through four different leaders, with the latest, Beibei Li, having left in May.

The future path for SpaceX's AI business remains unclear. The company launched the Grok Build platform at the end of May, simultaneously iterating on video and voice generation capabilities. SpaceX stated in its IPO prospectus that its own data centers are still continuously training the next-generation flagship Grok large model but did not disclose a formal launch timeline.

Disclaimer: Investing carries risk. This is not financial advice. The above content should not be regarded as an offer, recommendation, or solicitation on acquiring or disposing of any financial products, any associated discussions, comments, or posts by author or other users should not be considered as such either. It is solely for general information purpose only, which does not consider your own investment objectives, financial situations or needs. TTM assumes no responsibility or warranty for the accuracy and completeness of the information, investors should do their own research and may seek professional advice before investing.

Comments

We need your insight to fill this gap
Leave a comment