Groq Chat App
- Frontend
- React
- Backend
- Node.js · Express · Groq API
Overview
Full-stack chat app — Node.js/Express backend calling the Groq API, React (Vite) frontend.
Problem
Getting hands-on with fast LLM inference by building something real against it, rather than reading about it — and finding out what actually breaks when you put a third-party inference API behind a public endpoint.
Architecture
A Node.js/Express backend calls the Groq API and keeps the key server-side; a React (Vite) frontend handles the chat interface. Each request is stateless: one system prompt, one user prompt, no conversation history. The backend is deployed on Render, the frontend on Vercel.
Technical Decisions
- Keeping the API key strictly server-side in the Express backend rather than calling Groq directly from the browser, where any key would be trivially extractable.
- Making the model name an environment variable (GROQ_MODEL) instead of hardcoding it — a direct response to being burned by a deprecation mid-project.
- Adding our own rate limiting as a precaution on a public endpoint: 10 requests per minute on /api/chat and 100 per 15 minutes overall. These are self-imposed guards, not limits hit in practice.
- Waiting for the complete response rather than streaming tokens, with a progress bar to fill the silence. A pragmatic choice, though streaming would meaningfully improve perceived latency.
Challenges
The concrete problem was one I did not anticipate: Groq deprecated llama-3.3-70b-versatile mid-project. The model the backend had been calling started returning a "does not exist" error in production — a working app broken by a change on someone else's platform, with no code change on my side.
The fix was migrating to openai/gpt-oss-120b, Groq's current general-purpose recommendation. The more useful fix was structural: moving the model name into a GROQ_MODEL environment variable so the next deprecation is a config change rather than a redeploy.
What I Learned
Model availability on fast-moving inference platforms is not stable. Hardcoding a model name without a config layer is a real maintenance risk, not a theoretical one — the deprecation took the app down in production and there was nothing in my code that caused it.
Worth being straight about the limits of what this project tested. Groq's own rate limits were never reached, context-window behaviour was never exercised (every prompt was short and single-turn), and because the app is stateless there is no multi-turn cost curve to report. Those are untested areas rather than findings.
Links
Related Reading & Listening
Need something similar?
Want to add AI to an existing application?
Tell me what you're trying to build and what the model needs to do.