Based on an experiment by Juuso Saranki. Read the full step-by-step guide with screenshots on Juuso's blog.
Key findings
- You can generate deployable Bicep code with a fully local LLM. No cloud service or token-based pricing is needed. The test ran on a three-year-old MacBook Pro M2 with 32 GB of memory.
- Without guidance, the output looks like Bicep but doesn't work. On the first try, every generated file had errors and wasn't deployable without manual fixes.
- An instructions file makes the biggest difference. A Bicep ruleset given to the model improved the output quality dramatically.
- Smaller tasks work better than one big prompt. Local models have limited context windows, so splitting the work improves both reliability and speed.
- Hardware sets the limit, but instructions still matter. More memory lets you run larger models that can finish a task in one prompt, but a good instructions file is valuable on any hardware.
Why this matters
Coding assistants are moving to token-based pricing, so every generated line of infrastructure code costs money. A local setup changes that:
- From paying per token to a fixed cost. You pay for the hardware once, not for every prompt.
- From sending code to the cloud to keeping it on your machine. Your infrastructure code and context never leave your laptop.
- From generic suggestions to output that follows your rules. With an instructions file, the model follows the conventions you define instead of guessing.
The catch is that local models don't know Bicep well out of the box. Most LLMs aren't heavily trained on it, so you have to provide the missing knowledge yourself.
What went wrong, and what fixed it
First attempt: no instructions, one big request. The generated files were marked red in VS Code:
- The resource group module was missing
targetScope = 'subscription'.
- The storage account module had three stray backticks at the end.
main.bicep got the scopes wrong.
Second attempt: instructions file plus smaller tasks. Juuso added a bicep.md ruleset to the .continue/rules folder. He generated it with a more capable cloud model (Claude Opus via Copilot). He then asked for one module at a time. The modules came out clean, and the final main.bicep was deployable.
The lesson: a large model is useful for writing the rules once, and a local model can then follow them for free.
The setup in brief
- LM Studio runs the model locally and exposes an OpenAI-compatible API in server mode. Ollama is an alternative that's better for long-term consistency. LM Studio is easier for trying out different models.
- The Continue extension connects VS Code to the local model and gives you a Copilot-style chat and agent workflow.
- A rules file in
.continue/rules/ tells the model how to write Bicep.
- Small prompts: generate one module at a time, then compose
main.bicep.
For installation steps and screenshots, see Juuso's original post.
FAQ
Is 32 GB of memory enough?
Yes, for mid-sized coding models, if you split the work into smaller tasks. More memory allows larger models and single-prompt results.
Why do LLMs struggle with Bicep?
Bicep has far less public training data than languages like Terraform or Python. An instructions file fills that gap.
Do I still need an instructions file with better hardware?
Yes. Larger models make fewer mistakes, but explicit rules still improve consistency and quality.
