To The Moon

$Advanced Micro Devices(AMD)$ just unveiled a new accelerator chip called MI300 at an event on 06 Dec (Wednesday), AMD CEO has give a projection which is twice to what they have projected in August.

There have been talks and discussions about the Chip battle race, all the major chip makers are all coming up with chips that could help to accelerate the A.I. journey, especially now we are into the phase of more sophisticated A.I. models.

As we know that building an AI systems that could one day overtake human intelligence, we need to feed a model with lots of data, processing these data require huge amount of processing power, so we are already seeing companies like $Microsoft(MSFT)$ coming up with their own chips.

But is it enough to support what Microsoft wants to do?

In this article, I would like to bring us to the process of A.I. or rather the foundation that A.I. start off, machine learning, I have been doing ML since 2010 when it is still not so popular.

What Is Machine Learning Inference?

Machine learning inference refers to the process of using a trained machine learning model to make predictions or decisions based on new, unseen data. In other words, after a machine learning model has been trained on a dataset to learn patterns and relationships, the model can be deployed to make predictions on real-world data.

Steps Involved In Machine Learning Inference

Training

During the training phase, a machine learning model is exposed to a labeled dataset, where the input data and corresponding output (or target) values are provided. The model learns patterns and relationships within the data, adjusting its internal parameters to minimize the difference between its predictions and the actual outcomes.

Model Deployment

Once the model is trained and performs well on a validation dataset, it can be deployed for inference. Deployment involves making the model available for use in real-world scenarios.

Input Data

In the inference phase, the model receives new, unseen data as input. This data could be similar to what the model encountered during training but was not part of the training dataset.

Prediction/Decision

The model processes the input data and produces an output, which could be a prediction, classification, or decision, depending on the type of machine learning task (e.g., regression, classification, or clustering).

Evaluation and Feedback

The model's predictions are often evaluated against the actual outcomes to assess its performance on the new data. Feedback from the inference results can be used to further refine and improve the model over time.

NVIDIA H100 Tensor Core GPU Performance On Current GPT Models

$NVIDIA Corp(NVDA)$ can get up to 4X higher in AI training on GPT-3, the GPU is built with 80 billion transistors using a cutting-edge TSMC 4Nprocess custom tailored for NVIDIA’s accelerated compute needs, H100is the world’s most advanced chip ever built. It features major advances to accelerate AI,HPC, memory bandwidth, interconnect, and communication at data center scale

Source: nvidia-tensor-core-gpu-datasheet

As AMD has mentioned that their chip (MI300) would be much better in inference, I hope to see AMD make a performance on the largest models available and see whether MI300 can perform up to the mark.

Source: nvidia-tensor-core-gpu-datasheet

AMD Instinct™ MI300A Accelerators

The new AMD chip is using AMD CDNA™ 3 Architecture, and it has more than 150 billion transistors and 2.4 times as much memory as Nvidia’s H100,

The memory bandwidth is 1.6 times as much, this would definitely boost performance. According to AMD CEO, the new chip is equal to Nvidia’s H100 in its ability to train AI software.

But if we look at the AI Performance (Peak TFLOPs), MI300 seem to perform much better, but I think we need to see more test cases on actual large model size.

Not forgetting that Nvidia is also planning to come up with H200.

Popular Large Language Models

Since we are on the topics on chip performance, these are the current popular LLMs which I have knowledge of.

It will be interesting to see if we can have a performance run against the Top 5 LLMs from Nvidia and AMD.

Summary

Machine learning inference is crucial in practical applications where predictions or decisions need to be made on new, real-time data.

Common applications include image recognition, natural language processing, speech recognition, fraud detection, recommendation systems, and more. The ability to efficiently and accurately make predictions on new data is a key aspect of the practical utility of machine learning models.

With chips that is focusing on the performance of inference, this is an important part, as this is where it is closest to end-users (consumers).

I will be monitoring AMD new chip and see if there are more literature or technical specification released in due course.

Appreciate if you could share your thoughts in the comment section whether you think AMD can advance in the chip battle race with its new MI300 chip?

@TigerStars @Daily_Discussion @Tiger_Earnings @TigerWire appreciate if you could feature this article so that fellow tiger would benefit from my investing and trading thoughts.

Disclaimer: The analysis and result presented does not recommend or suggest any investing in the said stock. This is purely for Analysis.