BTW, I suggest implementing it in such a way that you're not tied down to any specific API provider. There are many open source libraries out there with OpenAI compatible API endpoints (which Groq also supports). So if you use that, it will be relatively easy to switch service providers in the future.
@smitmartijn pointed this out on X just now, but also highly recommend looking at Cloudflare AI Gateway so you can have automatic fallbacks between different providers, real time logs, response caching, and a bunch of other useful things that will either improve your DX or save you money, or both
BTW, I suggest implementing it in such a way that you're not tied down to any specific API provider. There are many open source libraries out there with OpenAI compatible API endpoints (which Groq also supports). So if you use that, it will be relatively easy to switch service providers in the future.
@smitmartijn pointed this out on X just now, but also highly recommend looking at Cloudflare AI Gateway so you can have automatic fallbacks between different providers, real time logs, response caching, and a bunch of other useful things that will either improve your DX or save you money, or both