Skip to main content
Choose the right GPU instance type for your workload with detailed specifications and performance benchmarks.

​
GPU Comparison Matrix

GPU performance comparison chart showing TFLOPS, memory, and pricing

​
NVIDIA B200 Series

​
B200

Next-Generation AI Performance

Specifications

  • GPU Memory: 192GB HBM3e
  • Memory Bandwidth: 8.0 TB/s
  • CUDA Cores: 20,480
  • Tensor Cores: 640 (5th gen)
  • FP32 Performance: 100+ TFLOPS
  • Tensor Performance: 4000+ TFLOPS (BF16)

System Configuration

  • CPU: 64 vCPUs (Intel Xeon or AMD EPYC)
  • RAM: 512GB DDR5
  • Storage: 2TB NVMe SSD
  • Network: 200 Gbps
  • Pricing: $8.00/hour
Best Use Cases:
  • Large language model training (100B+ parameters)
  • Multi-modal AI model development
  • High-throughput inference serving
  • Scientific computing and simulation
  • Next-generation AI research
Performance Benchmarks:
  • LLaMA 70B Training: 2x faster than H100
  • GPT-4 Scale Models: 60% faster training
  • Stable Diffusion XL: 3x faster inference
  • BERT Large: 4x training speedup

​
NVIDIA H200 Series

​
H200

High-Performance AI Inference

Specifications

  • GPU Memory: 141GB HBM3e
  • Memory Bandwidth: 4.8 TB/s
  • CUDA Cores: 18,432
  • Tensor Cores: 576 (4th gen)
  • FP32 Performance: 80+ TFLOPS
  • Tensor Performance: 3000+ TFLOPS (BF16)

System Configuration

  • CPU: 48 vCPUs (Intel Xeon or AMD EPYC)
  • RAM: 384GB DDR5
  • Storage: 1.5TB NVMe SSD
  • Network: 150 Gbps
  • Pricing: $6.50/hour
Best Use Cases:
  • Large language model inference
  • High-throughput AI serving
  • Multi-modal AI applications
  • Scientific computing
  • Enterprise AI workloads
Performance Benchmarks:
  • LLaMA 70B Inference: 1.5x faster than H100
  • GPT-3.5 Scale Models: 2x faster inference
  • Stable Diffusion XL: 2.5x faster inference
  • BERT Large: 2.8x inference speedup

​
NVIDIA H100 Series

​
H100 80GB

Premium Performance for Enterprise Workloads

Specifications

  • GPU Memory: 80GB HBM3
  • Memory Bandwidth: 3.35 TB/s
  • CUDA Cores: 16,896
  • Tensor Cores: 528 (4th gen)
  • FP32 Performance: 67 TFLOPS
  • Tensor Performance: 2000 TFLOPS (BF16)

System Configuration

  • CPU: 32 vCPUs (Intel Xeon or AMD EPYC)
  • RAM: 256GB DDR5
  • Storage: 1TB NVMe SSD
  • Network: 100 Gbps
  • Pricing: $4.50/hour
Best Use Cases:
  • Large language model training (70B+ parameters)
  • Multi-modal AI model development
  • High-throughput inference serving
  • Scientific computing and simulation
  • Cryptocurrency mining and blockchain applications
Performance Benchmarks:
  • LLaMA 70B Training: 45% faster than A100
  • Stable Diffusion XL: 2.3x faster inference
  • BERT Large: 3.2x training speedup
  • GPT-3 175B: 40% reduction in training time

​
H100 NVL (Multi-GPU)

Scale to Multiple GPUs Available in 2x, 4x, and 8x configurations with NVLink for high-bandwidth GPU-to-GPU communication.
  • Total GPU Memory: 160GB
  • NVLink Bandwidth: 900 GB/s
  • Pricing: $8.50/hour
  • Best for: Medium-scale distributed training

​
NVIDIA A100 Series

​
A100 80GB

Versatile High-Performance Computing

Specifications

  • GPU Memory: 80GB HBM2e
  • Memory Bandwidth: 2.04 TB/s
  • CUDA Cores: 6,912
  • Tensor Cores: 432 (3rd gen)
  • FP32 Performance: 19.5 TFLOPS
  • Tensor Performance: 1248 TFLOPS (BF16)

System Configuration

  • CPU: 24 vCPUs
  • RAM: 192GB DDR4
  • Storage: 500GB NVMe SSD
  • Network: 25 Gbps
  • Pricing: $3.50/hour
Best Use Cases:
  • Deep learning model training (up to 30B parameters)
  • Multi-model inference serving
  • Data analytics and processing
  • Computer vision applications
  • Natural language processing

​
A100 40GB

Cost-Effective Performance

Specifications

  • GPU Memory: 40GB HBM2e
  • Memory Bandwidth: 1.56 TB/s
  • CUDA Cores: 6,912
  • Tensor Cores: 432 (3rd gen)
  • Performance: Same compute as 80GB model

System Configuration

  • CPU: 16 vCPUs
  • RAM: 128GB DDR4
  • Storage: 250GB NVMe SSD
  • Network: 25 Gbps
  • Pricing: $2.50/hour
Best Use Cases:
  • Medium-scale model training
  • Inference for production applications
  • Research and development
  • Batch processing workloads

​
NVIDIA V100 Series

​
V100 32GB

Proven Performance for Research

Specifications

  • GPU Memory: 32GB HBM2
  • Memory Bandwidth: 900 GB/s
  • CUDA Cores: 5,120
  • Tensor Cores: 640 (1st gen)
  • FP32 Performance: 15.7 TFLOPS
  • Tensor Performance: 125 TFLOPS

System Configuration

  • CPU: 12 vCPUs
  • RAM: 96GB DDR4
  • Storage: 200GB SSD
  • Network: 10 Gbps
  • Pricing: $2.00/hour

​
V100 16GB

Entry-Level Data Center GPU

Specifications

  • GPU Memory: 16GB HBM2
  • Memory Bandwidth: 900 GB/s
  • CUDA Cores: 5,120
  • Tensor Cores: 640 (1st gen)
  • Performance: Same compute as 32GB model

System Configuration

  • CPU: 8 vCPUs
  • RAM: 64GB DDR4
  • Storage: 100GB SSD
  • Network: 10 Gbps
  • Pricing: $1.50/hour
Best Use Cases:
  • Small to medium model training
  • Development and prototyping
  • Educational and research projects
  • Legacy application support

​
NVIDIA RTX Series

​
RTX 4090 48GB

High-Performance Development GPU

Specifications

  • GPU Memory: 48GB GDDR6X
  • Memory Bandwidth: 1008 GB/s
  • CUDA Cores: 16,384
  • RT Cores: 128 (3rd gen)
  • Tensor Cores: 512 (4th gen)
  • FP32 Performance: 83 TFLOPS
  • Tensor Performance: 165 TFLOPS (BF16)

System Configuration

  • CPU: 16 vCPUs (AMD Ryzen or Intel Core)
  • RAM: 64GB DDR5
  • Storage: 500GB NVMe SSD
  • Network: 1 Gbps
  • Pricing: $1.20/hour
Best Use Cases:
  • Game development and testing
  • 3D rendering and animation
  • AI art generation (Stable Diffusion)
  • Content creation and streaming
  • Development and learning

​
RTX 4080

Balanced Performance and Cost

Specifications

  • GPU Memory: 16GB GDDR6X
  • Memory Bandwidth: 717 GB/s
  • CUDA Cores: 9,728
  • RT Cores: 76 (3rd gen)
  • Tensor Cores: 304 (4th gen)

System Configuration

  • CPU: 12 vCPUs
  • RAM: 32GB DDR5
  • Storage: 250GB NVMe SSD
  • Network: 1 Gbps
  • Pricing: $0.90/hour

​
Performance Benchmarks

​
Training Performance (Relative to V100 16GB)

Model TypeV100 16GBA100 40GBA100 80GBH100 80GB
BERT Base1.0x2.1x2.1x3.2x
ResNet-501.0x1.8x1.8x2.5x
GPT-2 Medium1.0x2.3x2.3x3.8x
Stable Diffusion1.0x2.0x2.0x4.1x

​
Inference Throughput (Tokens/Second)

ModelA100 40GBA100 80GBH100 80GB
LLaMA 7B454578
LLaMA 13B282852
LLaMA 30BN/A1528
LLaMA 65BN/A818

​
Cost-Performance Analysis

Cost per TFLOPS comparison across different GPU types

​
Choosing the Right Instance

​
Decision Matrix

Small Models (< 7B parameters):
  • RTX 4090 or A100 40GB for development
  • A100 80GB for production training
Medium Models (7B - 30B parameters):
  • A100 80GB (single GPU)
  • 2x A100 80GB for faster training
Large Models (30B+ parameters):
  • H100 80GB (single GPU up to 70B)
  • Multi-GPU H100 NVL for 100B+ models
Low Latency Requirements:
  • H100 80GB for fastest response times
  • A100 80GB for balanced performance/cost
High Throughput Batch:
  • A100 40GB for cost-effective processing
  • Multiple A100s for parallel serving
Development/Testing:
  • RTX 4090 for cost-effective testing
  • V100 16GB for basic inference
3D Rendering:
  • RTX 4090 with RT cores
  • RTX 4080 for lighter workloads
AI Art Generation:
  • RTX 4090 for Stable Diffusion
  • A100 for custom model training
Video Processing:
  • RTX series with hardware encoders
  • A100 for AI-based video enhancement
Budget-Conscious:
  • V100 16GB for learning
  • RTX 4080 for small experiments
Production Research:
  • A100 40GB for most projects
  • A100 80GB for memory-intensive work
Cutting-Edge Research:
  • H100 80GB for latest capabilities
  • Multi-GPU setups for large experiments

​
Regional Availability

Available Instances:
  • ✅ All H100 configurations
  • ✅ All A100 configurations
  • ✅ All V100 configurations
  • ✅ All RTX configurations
Latency: < 50ms to major US cities

​
Spot Instance Pricing

Save up to 70% with spot instances for fault-tolerant workloads:
Instance TypeOn-DemandSpot PriceSavings
H100 80GB$4.50/hr$1.35/hr70%
A100 80GB$3.50/hr$1.40/hr60%
A100 40GB$2.50/hr$1.00/hr60%
V100 32GB$2.00/hr$0.70/hr65%
RTX 4090$1.20/hr$0.48/hr60%
Spot instances can be interrupted with 2-minute notice when demand increases. Use for training jobs with checkpointing.

​
Next Steps

Launch Instance

Deploy your first GPU instance with our instance launcher.

Instance Management

Learn how to monitor, scale, and manage your instances.

Pricing Calculator

Estimate costs for your specific workload requirements.