Good card for local models. Memory: 16 GB of HBM2 on the Pascal generation. It’s fast memory, and that matters for AI speed.
Model size: it fits models up to roughly 14B parameters at 4-bit or 8-bit quantization. A 32B model at 4-bit is about 19 GB, so it won’t fit on this card alone.
Display: no video outputs. It’s compute only. Good card for local models. Memory: 16 GB of HBM2 on the Pascal generation. It’s fast memory, and that matters for AI speed.
Model size: it fits models up to roughly 14B parameters at 4-bit or 8-bit quantization. A 32B model at 4-bit is about 19 GB, so it won’t fit on this card alone.
Display: no video outputs. It’s compute only.