AWS Announces Next Generation Graviton3 Processors
In 2018 Amazon Web Services unveiled the first generation of its AWS Graviton chip-powered instances.
At the AWS re: Invent 2021 conference in Las Vegas, AWS announced the third generation of Graviton processors – the AWS Graviton3.
The intention was to build a processor that was optimized specifically for modern workloads. To improve the performance, reduce the cost, and increase the efficiency for the majority of workloads in the cloud.
AWS Graviton3 will power all new Amazon Elastic Compute 2 (EC2) C7g instances that are now available in preview.
What is AWS Graviton3?
Graviton3 Performance Gains
Comprehensive performance improvements across computing dimensions
Compute Performance
0%General workload improvement
Floating-Point
2XScientific computing boost
Cryptographic
2XSecurity operations
ML Performance
3XAI/ML workload acceleration
Memory Bandwidth
0%Memory-intensive apps
Energy Efficiency
0%Power per operation
Key Insight: The 3X ML performance boost with bfloat16 support positions Graviton3 as a game-changer for AI workloads, while maintaining the same 100W power envelope through sophisticated architectural optimizations.
AWS Graviton3 is the latest in the Graviton family of processors. It is 60% more efficient than its predecessor.
Graviton3 will deliver up to 25% more compute performance. It offers up to 2x as much floating-point & cryptographic performance.
On the machine learning side, Graviton3 includes support for bfloat16 data. Which is a truncated 32-bit floating-point value.
Graviton3 is ideal for machine learning where speed is more important than precision. It will deliver up to 3x better machine learning performance.
Overall the performance claim is for a more modest 25%.
Graviton3 achieves 50% higher memory bandwidth. This is done while keeping the entire CPU power consumption in that same ~100W range.
Amazon is using a chiplet design with 7 different dies. They are packaged using advanced packaging. The size of the bumps connecting each die is <55um. Whereas every CPU from Intel and AMD is still at >100um.
Amazon is keeping costs down on the IP side by leveraging ARM’s stock cores and Synopsys / Cadence IP. The price-performance advantage of Graviton3 is huge as it extends beyond just vertical integration.
Graviton servers feature a Nitro card capable of managing three Graviton3 processors.
Graviton3 Specifications
Graviton3 vs Graviton2
Architectural Evolution Through 25 Billion Extra Transistors
AWS invested 25 billion extra transistors not in adding cores, but in making each core substantially more capable - addressing the memory bandwidth bottleneck that limited Graviton2 workloads.
- 2.6 GHz clock speed
- 300 GB/sec max memory bandwidth
- DDR5 RAM
- 64 cores
- Seven silicon die chiplet-based design
- 256-bit SVE (Scalable Vector Extension)
- 55 billion transistors (Graviton2: 30 billion)
Benefits of Graviton3
- 25% higher performance
- 50% more memory bandwidth
- Support for bfloat16 data
- 2x floating-point
- 2x cryptographic performance
- 3x machine learning performance
- 60% less energy consumption
- Enhanced security
Pointer Authentication on AWS Graviton3: Enhancing Security
Graviton3 Advanced Architecture
Silicon-level security meets cutting-edge packaging technology
Hardware Pointer Authentication
Cryptographic signing of return addresses with secret keys prevents stack-based attacks
Blocks ROP attacks by validating signatures before execution
Always-On Memory Encryption
Continuous protection of data in memory against side-channel attacks
Critical for multi-tenant cloud security
7-Die Chiplet Design
Advanced multi-die architecture for improved modularity and manufacturing yield
Enables mix-and-match silicon optimization
Sub-55um Interconnects
Superior to Intel and AMD processors still using greater than 100um connections
50% better memory bandwidth enabled
Nitro Card Management
Single Nitro card manages three Graviton3 processors for data center efficiency
Higher density and resource utilization
bfloat16 Data Format
Optimized for machine learning inference where speed matters more than precision
3x better ML performance for inference
256-bit SVE Support
Scalable Vector Extensions for enhanced SIMD parallel processing capabilities
Accelerates scientific and HPC workloads
Cost-Effective IP Model
Leveraging ARM stock cores with Synopsys and Cadence IP for optimal economics
Focus engineering on system optimization
Comprehensive Cloud Processor Innovation
Graviton3 combines hardware-level security, advanced packaging technology, and strategic IP licensing to deliver superior price-performance that extends beyond vertical integration, setting new standards for cloud computing infrastructure.
Graviton3 includes a new pointer authentication feature to improve security.
Before return addresses are pushed onto the stack, they’re first signed with a secret key.
The generated key is additional context information, including the current value of the stack pointer.
When the signed addresses are popped off the stack, they’re validated before being used.
An exception is raised if the address isn’t valid. It works to block attacks by overwriting the stack contents with the address of harmful code.
The Graviton 3 also supports always-on memory encryption.
Graviton3 Improvements Over Graviton2
Graviton3 Performance Gains
Real-world benchmark results showing performance improvements across different workload types
Redis Database
Solid improvement for memory-intensive caching operations
General Workloads
Energy Efficiency
Technical Capabilities
Performance gains vary by workload architecture - Nginx benefits from doubled IPC while Redis leverages 50% memory bandwidth increase
Graviton2 was first previewed in late 2019. Most workloads running on Graviton2 could run more efficiently if they had:
- More memory bandwidth
- Lower latency access to memory
In Graviton3, rather than using our extra transistors to pack more cores, AWS uses transistors to improve memory performance.
Real world cloud workloads are using more memory bandwidth. They’re becoming more sensitive to memory latency.
Graviton2 instances already had a lot of memory bandwidth per VCPU. More memory bandwidth has been added to Graviton3 instances. Each Graviton3 core has 50% more memory bandwidth than Graviton2.
Graviton2 was 64-core and at 2.5 GHz clock speed. Graviton3 has greater width, meaning that twice as much data can be processed in a single clock cycle.
The number of instructions that each core can work on concurrently has increased from 5 to 8 per cycle.
Graviton2 had Arm's Neoverse N1 cores. Graviton3 has Neoverse N2 or Neoverse V1 cores.
Graviton3-powered instances are up to 60% more energy efficient than the Graviton2-powered EC2 instances.
A slide shown by DeSantis shows gains from 25% for a Redis application, to 60% for Nginx.
Amazon EC2 C7g Instances
Amazon announced the preview of the EC2 C7g instances, the new instance type running the latest Arm-based processors.
AWS Chief Evangelist Jeff Barr said that the instances should be useful for compute-intensive workloads.
Such as:
- HPC
- Batch processing
- Electronic design automation (EDA)
- Media encoding
- Scientific modeling
- Ad serving
- Distributed analytics
- CPU-based machine learning inferencing
C7g is the only Graviton3 VM instance that supports Elastic Fabric Adaptor (EFA). It will offer up to 30 Gbps of network bandwidth.
C7g instances are the first in the cloud industry to be equipped with DDR5 memory.
Another feature of the C7g instance is that it is built on the AWS Nitro System.
The new instances will be available in multiple sizes (including bare metal).
Conclusion
AWS Graviton3 will have a significant impact on performance in current generation instances.
It aims to deliver the best performance and the lowest cost for modern workloads.
AWS Graviton3-powered instances, including C7g, are supported by many popular Linux operating systems. Such as Amazon Linux 2, Red Hat Enterprise Linux, Suse Enterprise Linux Server, and Ubuntu.
Many popular applications and services for security, monitoring, management, containers, and CI/CD also support AWS Graviton-based instances.
Users can do the sums and migrate to Graviton3 where applications are compatible.
The C7g instances are available in preview. It will likely be a few months before general availability.
To try the new instances, customers can sign up to join the preview.
