Back
Similar todos
experience 6 hours GPU total downtime at Vultr, knocking down all #mirage AI services, will migrate to Scaleway cause Vultr are 🤡
See similar todos

No replies yet

spend 1h debugging some k8s complexity-induced issue after a node failure at cloud provider for #crisp mirage gpus
See similar todos

No replies yet

fix broken nvidia a100 gpu server at vultr which has put #crisp mirage down for whole night due to being out-of-stock and no replacement physical node could be allocated
See similar todos

No replies yet

Fixing downed servers for the 3rd time in a week and tweet about it x.com/ScottWRobinson/status/1… #blocksender
See similar todos
try to upgrade #crisp Mirage AI Kuberbetes cluster version, miserably fail at it cause of broken GPU image NVIDIA drivers from the cloud provider which stalled the upgrade process, destroy cluster and rebuild all infrastructure from scratch all evening 🥲
See similar todos
Contact cloudways for a WTF on a small site... apparently hit another glitch. Hitting a lot of bugs this week - the amount of time support admin is taking me lately is out of control#admin
See similar todos

No replies yet

wake up from mayhem with my NVIDIA A16s, 70% of them were down all night and only fixed this morning by Vultr. A40 for large LLM still up fortunately #crisp
See similar todos

No replies yet

get cloudflare incident acknowledged that plagued me for 4 days #spectate
See similar todos

No replies yet

appear to have fixed a production latency by tweaking resources + procfile... but it has been a day of tinkering 💀 #wfhland
See similar todos

No replies yet

finally fix bug that caused trouble for 1-2 days :) #indie
See similar todos

No replies yet

#pintura fix issue with background flickering on certain GPUs
See similar todos

No replies yet

Experience hetzner network issues again, still down even after they acknowledged it on phone. 6 hours and counting.
See similar todos

No replies yet

Client invoice issues ~for 13 hours~ 🫠
See similar todos

No replies yet

debug with the new GPU API for hours #avatarai
See similar todos

No replies yet

Spent big part of the day with a production issue that couldn't be hot-fixed quickly while an event with 350+ people was happening. Javascript could be more permissive sometimes 🤦‍♂️
See similar todos

No replies yet

Finally get the backend deployed after fighting with AWS bugs for the entire day #unblockdomains
See similar todos

No replies yet

buy GPU credits cause they ran out during Christmas dinner #avatarai
See similar todos

No replies yet

not work for 3 days #life
See similar todos
#thecompaniesapi work on performances PR ; watch the GPU struggle with lots of queries
See similar todos

No replies yet

Move back #highscoredomains again back to aws & cloud66 since render seem to be down more offer these days
See similar todos

No replies yet