I've done some investigation recently to try to figure out what's the cheapest GPUs around that would work for machine learning type tasks like running whisper or similar. I have a fairly beefy GPU in my computer, the A4000, which is an unusual configuration. It's a workstation GPU, not a consumer GPU. And it's a fairly high end GPU. I kind of got it because I mostly do productivity stuff on my computer, like photo, video, editing, some GPU intensive compute processes and things like that. But looking a bit into if there are lesser GPUs around just for recommendations to other people that would work. I think the obvious thing, and it's the one situation I tested with old equipment I have around would be the RTX 2060. It's kind of a consumer GPU, it goes used for about $200 from what I can tell on eBay and new for about $300 in a 12GB model. It's the cheapest consumer GPU that has high VRAM.

And for most machine learning tasks that I'm interested in, VRAM is the limiting factor to an extent that's not true of gaming. On eBay I was able to find old workstation graphics cards that have a lot of RAM. One good example is the Nvidia M40, it has 12GB of RAM and I'm seeing it used for around $100. Like the absolute cheapest one that has enough RAM that I'm seeing is the K40, the Nvidia K40. And that also has 12GB of RAM. I would say the M40 would get pretty reasonable performance. The M40 has a pass mark score on GPU compute of 3775 operations per second. Kind of comparing that to the GPUs that I've kind of ran Whisper on, I guess it would do the large model in approximately one to one timing. One minute of audio input would take about a minute to process.

The GPU that I have, the A4000, gets about four to one. Four minutes of audio input would take a minute to process. The cheapest GPU that I've found that has enough VRAM has the $45 K40, has a pass mark score of around 2000 ops per second and that would like I think get like two minutes of processing time for like each minute of audio or maybe slightly worse than that. But I think like, I think there are a lot of kind of cheap GPU options if you're using the type of workflow that I use. And you just feed the speech tech software a pre-recorded recording and let it transcribe.