567 points by gpu_researcher 1 year ago flag hide 7 comments
coder1 1 year ago next
Fascinating post! I've been wondering how different GPU architectures compare for training neural networks. Thanks for sharing!
nvidia_guy 1 year ago next
At NVIDIA, we've put in a lot of work optimizing our GPUs for machine learning and deep learning. We believe our architecture, with its focus on concurrency, memory bandwidth, and CUDA cores, provides an excellent platform for neural network training.
amd_engineer 1 year ago prev next
AMD's Radeon Instinct GPUs also provide impressive performance for neural network training. Our MIOpen library and ROCm platform bring powerful open-source tools to the table for developers interested in deep learning.
ml_researcher 1 year ago prev next
What's your take on tensor core based architectures vs those that use CUDA cores or stream processors? Do you know of any good comparative benchmarks?
tensor_expert 1 year ago next
Tensor cores certainly provide a boost for tensor operations but they may not be as versatile for other types of workloads. The NVIDIA vs AMD comparison would heavily depend on the specific neural network architecture and workloads being used.
independent 1 year ago prev next
I've seen studies that show NVIDIA GPUs leading the pack for TensorFlow and PyTorch with their CUDA support. However, newer Radeon Instinct GPUs might catch up soon due to their improved specifications and lower cost.
hpc_user 1 year ago prev next
Do you think GPU-accelerated machine learning could further benefit from more diversified architectures or will one manufacturer become the de facto standard for training?