New to WIP?
We're a group of makers shipping together. We help each other stay accountable and reach our goals.
Apply for access
Topics
Other questions
Back
Question
Which VPS can I use to have access to a powerful GPU like A100 and the terminal?
So, suppose you have a custom LoRA that you want to train from scratch.
And the script uses xformers, which looks like it requires CUDA, so I can't run it on my M2 MacBook.
I tried running on Google Colab (paid) with a V100, but it's always running out of memory (16GB)
Where else can I run a VPS - like having access to the terminal and installing things and running shell scripts - with a powerful GPU like A100 or anything >32GB?
I found a LoRA training at Replicate (https://replicate.com/cloneofsimo/lora-training), but I have to do something custom.
And the script uses xformers, which looks like it requires CUDA, so I can't run it on my M2 MacBook.
I tried running on Google Colab (paid) with a V100, but it's always running out of memory (16GB)
Where else can I run a VPS - like having access to the terminal and installing things and running shell scripts - with a powerful GPU like A100 or anything >32GB?
I found a LoRA training at Replicate (https://replicate.com/cloneofsimo/lora-training), but I have to do something custom.
👋 Join WIP to participate
AWS EC2 has instances with NVIDIA GPUs that you can rent. You'll probably have to install any dependencies yourself via SSH but that would work. You'll want to look at "Accelerated Computing" instances.
Also check Google Cloud, Azure to see what their pricing looks like. It's probably comparable.
Thanks! Will take a look
www.vultr.com/pricing/#cloud-…
Thanks! Will take a look
Replicate? replicate.com/pricing
Ouaww, almost twice the price of Vultr for the same.
It's not the same though. What you're paying for with Replicate is ease of use - they provide an API where you can just select any model you want and execute, which is useful for applications that need to make a API call to get a response from a LLM for example. Vultr doesn't have any of that AFAIK, and it will take you extra time to build it hence the lower price.
Replicate is great for hosting models, but they don't offer anything for development with SSH access, etc.
Exactly. I’d love to run this on Replicate, but I don’t know how to submit a custom shell script, run pip install, and train a model there etc. maybe it’s possible with Cog but I don’t know how to use it yet
I use Paperspace for all my AI related VPS needs. They have many GPUs, although you may need to contact support to enable the more powerful ones.
Nice! You use the Notebook feature or is something else?
you can train your model on paperspace (or use the already train one) and make your own API.
here is one tutorial i have used before: dev.to/saronyitbarek/deployin…
I use their VPS-like system where you can SSH into the instance
I don't know if it was like this before, but Paperspace now has these modes called "Gradient" and "CORE". I'm guessing the VPS-like system is the CORE one? I had to change it by clicking on the top left corner.
Yes, CORE is the one!