2030 Creating an All-Optical - Intelligent Internet Huawei Asia Pacific ISP Summit

Google Cloud and Amazon Web Services

After Google Cloud and Amazon Web Services (AWS) announced the general availability of their most recent custom AI accelerators this week, cloud-based AI training became a little more varied.

News analysis (Tobias Mann)

Starting things off is Amazon, whose Trainium chips are currently broadly accessible on AWS. Amazon's Trainium-powered Trn1n instances, which were first showcased at AWS re:Invent last year, are intended to train sizable machine-learning models, such as those used in image and natural language processing.

According to internal testing conducted by Amazon, the instances outperform Nvidia A100-powered P4d instances by 40–250 percent in BF16 and 32-bit TensorFlow workloads. Additionally, the accelerator supports the FP8 datatype, FP32, FP16, and UINT8. In recent years, FP8 has gained popularity in the AI community as a way to compromise accuracy for raw performance.

The instances are available in two sizes: The Amazon trn1.2xlarge system combines eight virtual CPUs with a single Trainium processor, 64GB of RAM shared evenly between the CPU and accelerator, 12.5Gbit/sec networking, and 500GB of on-board SSD storage.

The trn1.32xlarge is 16 times larger, with 128 vCPUs, 16 Trainium processors, 1TB of combined RAM, and 800Gbit/sec of network bandwidth per instance, but it's made to handle greater workloads thanks to its 1TB of combined RAM and 800Gbit/sec of network bandwidth.

Using Amazon's FSx Lustre storage service and "petabit-class" non-blocking top-of-rack switches, several trn1.32xlarge instances can be clustered for large-scale model training.

The accelerator makes use of the identical Neuron SDK as Amazon's previously revealed Inferentia inferencing processor, which includes a compiler, framework extensions, a runtime library, and developer tools. Collectively, Amazon claims workloads created in well-known machine learning frameworks like PyTorch and TensorFlow can be converted to run on Trainium with little to no reworking.

Google Amazon’s latest AI chips have arrived review

This week, Amazon's US East and US West regions will have access to the Trn1n instances.

TPU v4 from Google is now generally accessible.

At its Cloud Next event this week, Google also disclosed a slew of hardware improvements, including the wide availability of its fourth-generation Tensor Processing Units (TPU).

The TPU v4-powered virtual machines on Google Cloud are offered in configurations ranging from a single TPU module with four chips to a pod with up to 4,096 chips connected through high-speed fabric.

If you're unfamiliar, Google's TPU accelerators were created expressly to accelerate in hardware big machine-learning models, such as those used in computer vision, recommender systems, and natural language processing.

The accelerator, which can be programmed, is essentially a collection of huge bfloat matrix math engines called MXUs, supported by some high-bandwidth memory, and a few CPU cores.

The CPU cores are told to feed the MXUs the AI math operations for a workload so that they can process them quickly. Apiece TPU VM is made up of four processors with two processing cores each and 128GB of RAM overall.

We suggest visiting our sister site The Next Platform for a detailed analysis of Google's most recent TPU architecture.

The unique accelerators were first developed to accelerate Google's internal AI workloads, but they were later made available to GCP users. TPUs support a number of well-known ML frameworks, including JAX, PyTorch, and TensorFlow, as might be expected. Additionally, the TPU v4 offers 40 percent higher performance per dollar while being more than twice as fast as its predecessor, claims Google.

TPU v4 Pod slices are now offered in the Oklahoma region of GCP at a price ranging from $0.97 to $3.22 per chip, per hour. With a one-year commitment, that amounts to $5,924 per month for Google's smallest instance.

More Context Read :  San Francisco in a Driverless Car information technology

Google gives users a glimpse of Intel's next CPUs and smartNICs.

This week, a secret preview of Google Cloud included Intel's Sapphire Rapids CPUs and Mount Evans IPUs.

Select clients can now test out Intel's long-delayed Sapphire Rapids CPUs, although today's statement provides scant details about the microprocessors' capabilities. Instead, the company highlighted the Mount Evans IPUs it created in collaboration with Intel.

According to Nick McKeown, who oversees Intel's network and edge group, "C3 VMs will run workloads on 4th Gen Intel Xeon Scalable processors as they liberate programmed packet processing to the IPUs securely at line rates of 200Gbit/sec."

Mount Evans, now known as the E2000, was first introduced during Intel's Architecture Day last year. It is the company's first IPU ASIC. Infrastructure Processing Units, or IPUs, are essentially another type of hardware accelerator for networking and storage activities.

The smartNIC-class silicon will speed up Google's cloud infrastructure workloads. One of the first things will be storage. The cloud provider claims that its IPU-boosted C3 instances produce 10 times the IOPS and four times the throughput of its outgoing C2 instances when using its recently launched Hyperdisk service.

IPUs, DPUs, and SmartNICs are not exactly a recent development in the realm of clouds.

SmartNICs are also being used by Amazon, Microsoft Azure, and Alibaba Cloud to offload infrastructure activities from the host, such as networking, storage, and security, freeing up CPU cycles for tenant workloads in the process.

Sapphire Rapids from Intel is still confined to the cloud.

Although Sapphire Rapids teases that the C3 instances are the "first VM in the public cloud," the word "public" is definitely not the best choice in this case. Google's C3 instances are still only available to a limited number of customers, presumably under a stringent NDA.

Sapphire Rapids CPU family debut date still hasn't been announced by Intel as of this week, more than a year behind schedule. However, Intel seems more eager than ever to get its next-generation datacenter CPUs into some customers' hands — at least virtually — given that AMD's fourth-generation Epyc processors are expected to launch this autumn.

Google is merely the most recent Intel partner to offer clients certain Sapphire Rapids-based resources. To give consumers the chance to experiment with the new capabilities made possible by the chips, Supermicro and Intel are each allowing remote access to bare-metal computers while Google is supplying cloud VMs.

Some OEMs, cloud partners, and government organisations are now receiving fourth-generation Xeon Scalable processors from Intel that are Sapphire-Rapids-powered. How many chips the x86 titan has actually distributed to consumers is unknown.