To The Moon

In the early morning hours of July 23, it was revealed that Meta's Llama 3.1-405B review data had suffered a leak, and that the largest parametric model in the Llama 3 series may be released tomorrow, along with a Llama 3.1-70B version $Meta Platforms, Inc.(META)$

This is also a feature iteration on top of version 3.0, and even the base 70B model outperforms the GPT-4o.

Even the magnet link is streaming out, "AIGC Open Community" tried about 763.84G. originally there was one on huggingface, but then the library was deleted.

The download speed is not bad, about 14M per second. It seems that there are quite a lot of people downloading this model.

But this model of the general GPU is certainly not run up, such a large parameter in the deployment of individual developers can not afford (if you have some H100 is also no problem), it is estimated to be for the enterprise, government public sector with.

For Meta's soon-to-be-released model, there are netizens who throw cold water on it.Compared to OpenAI's latest GPT-4o mini version, Llama 3.1-70B inference costs 3 times more, but the performance of encoding is much worse.

In terms of price/performance ratio and features, Meta's new model is nothing to look forward to.

Someone else even saw the said released model on GitHub, but it was quickly taken down and presumably some people might have been able to use it.

Someone else also said that for this leak he thinks it's real because it's streaming from Microsoft's Azure Github.

However, this model has large parameters and is too demanding on the GPU, and is not as cost-effective as the GPT-4o mini.

Although the model is free, it's still quite a bit of work to try to run it, and you really can't use it without an enterprise-level arithmetic base.So this is good news for the enterprise.

It has been pointed out that even if the Llama 3.1-405B model is significantly optimized and quantized to 5 digits, it still won't work for consumer GPUs, and is really exceptionally demanding on hardware.

If this review data is true, it would be a godsend for most of the world.Because this is the top model of Meta's Llama 3 series and it's all open weighted. That means free AI models for everyone.

But if you want to develop generative AI applications, you also need a strong AI arithmetic base, high-quality data, and fine-tuning techniques.

Meta has been delaying the release of the 405B series of models due to regulators and various bills.So, was this leak purposely put out by Meta, as it's an old tradition of theirs, having done it once with the Llama model last year