Daniel Nguyen
@daniel_nguyenx
I see. BoltAI currently uses the OpenAI-compatible server from Ollama. Maybe that's why it's slower than querying the model directly.
I will do more benchmarking and maybe switch to direct connection in the future.
Did you use Ollama or how are you using it?
yes I used Ollama llama3 8b. Feel the speed is slow compared to using it directly from terminal
I see. BoltAI currently uses the OpenAI-compatible server from Ollama. Maybe that's why it's slower than querying the model directly.
I will do more benchmarking and maybe switch to direct connection in the future.
Yogesh, you can follow this guide to use a custom AI service or a different model. You can also bulk edit
Hey @yogesh, Daniel from BoltAI here.
I can’t answer your question since there are dozens of AI clients now that I lost track of the competition.
But here are a few things I strongly believe in BoltAI:
It supports more AI services & models than just OpenAI. My customers usually alternate between GPT-4 and Claude 3 Opus for coding for example. Or to use Groq with their insanely fast inference. Or using a local model with Ollama. It’s easy to switch in BoltAI.
It supports “context-aware” AI Command. You highlight the content, press a keyboard shortcut then ask AI about it. You can build your own AI tools with this: text summarization, content rewrite, translation…
Quickly take a screenshot and ask AI about it. Currently it supports both GPT 4 and Claude 3. BoltAI will support local model in the next version (llava)
A more powerful plugin system. This is still in progress but imagine AI can help you with your local tasks such as renaming, photo/video manipulation etc. It would be super powerful.
It’s one-time payment, not a subscription.
Happy to answer any questions 😊
BTW here is the list of all features with demo: docs.boltai.com/docs/features
Hey Daniel, just purchased the product. Is there a way to change all the assistants model to gpt4o? I see it's gpt3.5 turbo rn.
Yogesh, you can follow this guide to use a custom AI service or a different model. You can also bulk edit
btw the llama3 req timeout most of the time and is too slow. It worked once or twice only. When I run it in the terminal it works pretty fast and always does.
It worked again when I checked stream response. I had unchecked it and it stopped working. But It is still slow compared to terminal.
Did you use Ollama or how are you using it?
yes I used Ollama llama3 8b. Feel the speed is slow compared to using it directly from terminal
I see. BoltAI currently uses the OpenAI-compatible server from Ollama. Maybe that's why it's slower than querying the model directly.
I will do more benchmarking and maybe switch to direct connection in the future.
I highly recommend read this book: www.amazon.com/Traction-Start…
Here is a good summary: readingraphics.com/book-summa…
Hmmmm good point actually. But I was thinking of providing collaboration features that won’t be available in an individual license.
My ideal org size is between 5-30 people (1-6 teams)
Ah good idea. I did receive a couple of inquiries about this but it was for PDF Pals, not BoltAI. Maybe I can merge the 2 products and offer this for businesses.
Thanks for the advice Ben. I was thinking of targeting small businesses, not enterprises. Hopefully that would minimize red tape and with shorter sales cycle :D
One direction I think could work is to keep BoltAI as a standalone app like it is now, then I started building more integrations. Though this seems to take a lot more dev efforts 🤔
Good idea, @screenfluent. Also a quick question, I wanted to move build-in-public from Twitter to WIP, as I think it would be more valuable to other builders than the general audience on Twitter.
Which type of content should I post? (I need it kinda longer form, doesn't seem to be a good fit for Todo/Question/Roast/Milestone/Intro 😅)
Tagging @marc as it's intended for Marc :D
Thanks, Daniel! I can't wait to read your posts here, free from the worry that Twitter crowds might make you second-guess sharing your experiences with the world.
I think cloning Milestones to Posts could be the best approach. The longer I think about it, the more I like the idea that WIP is a kind of micro (or not) blogging platform centered around our projects.
@daniel_nguyenx I had a similar question because I wanted to share long-form content here, and then I found @jasonleow's LifeLog. Many of us use this space to share long-form content about what we're building, how life is going, and our thoughts about the process.
Yo. Welcome to WIP 👋