Google's models and tools: Gemini CLI, AI Studio, Gemma.
Small language models are significantly more cost-effective, running datasets at a fraction of the cost of frontier models. While frontier models handle complex tasks easily, small models can solve a substantial percentage of common tasks when properly guided.
▶ 1:28
Agentic coding workflows with small models
Developers choose local models for data privacy, zero latency without round trips to servers, zero marginal cost during error loops, and offline access. These benefits make local execution ideal for sensitive code and disconnected environments.
▶ 6:18
Agentic coding workflows with small models
Quantization shrinks the precision of model weights, such as compressing a 16-bit floating-point model down to a 4-bit integer format. This dramatically reduces video RAM requirements so the model can fit on standard consumer hardware.
▶ 7:31
Agentic coding workflows with small models
Speculative decoding uses a smaller drafter model to predict a few upcoming tokens ahead of time, which are then validated by the larger model. Because they share the same context cache, this technique achieves significant speedups without degrading output quality.
▶ 8:58
Agentic coding workflows with small models
Small models can fail tasks if important files or logs are pushed out of memory by agent compaction. Increasing the context window ensures the model retains critical file information and significantly improves its success rate.
▶ 10:43
Agentic coding workflows with small models
Developers can use a readiness framework that scores tasks based on reliability, latency thresholds, and RAM usage alongside the model and hardware configuration. Green scores indicate the setup is ready, while other colors suggest adjustments are needed.
▶ 12:10
Agentic coding workflows with small models
Small models excel at small tasks like basic Compose layouts and medium tasks that require reading and writing just a few files. However, they struggle with long horizon tasks like massive migrations that span dozens of files.
▶ 20:22
Agentic coding workflows with small models
Developers can improve small model success by ensuring full context windows are enabled, using CLI tools for niche documentation, and adding verification hints for tests. Avoiding bloated system prompts also prevents performance degradation.
▶ 24:31
Agentic coding workflows with small models
Orchestration allows developers to set up fallback models to prevent workflow blocks during provider outages or offline scenarios. It also enables cascading tasks, routing simpler jobs to smaller models and complex planning to frontier models for cost efficiency.
▶ 27:54
Agentic coding workflows with small models
No answer mentions that.