Based on an experiment by Juuso Saranki. Read the full step-by-step guide with screenshots on Juuso's blog.
Key findings
- You can generate deployable Bicep code with a fully local LLM. No cloud service or token-based pricing is needed. The test ran on a three-year-old MacBook Pro M2 with 32 GB of memory.
- Without guidance, the output looks like Bicep but doesn't work. On the first try, every generated file had errors and wasn't deployable without manual fixes.
- An instructions file improves the output dramatically. A Bicep ruleset given to the model made a huge difference in output quality.
- Smaller tasks matter just as much. Local models have limited context windows, so splitting the work into smaller prompts improves both reliability and speed.
- Hardware sets the limit, but instructions still matter. A more powerful machine needs less tweaking and can often complete tasks in a single prompt. Even so, a good instructions file is valuable on any hardware.
Why run an LLM locally?
Coding assistants are moving to token-based pricing. A fully local setup is an alternative, and it also keeps the workflow private and offline.
The catch is that local models don't know Bicep well out of the box. Most LLMs aren't especially well trained on it, so you have to provide the missing knowledge yourself.
What went wrong, and what fixed it
First attempt: no instructions, one big request
The generated files were marked red in VS Code:
- The resource group module was missing
targetScope = 'subscription'.
- The storage account module had three stray backticks at the end.
main.bicep got the scopes wrong.
Second attempt: instructions file plus smaller tasks
Juuso added a bicep.md ruleset to the .continue/rules folder. He generated it with a more capable cloud model (Claude Opus via Copilot). He then asked for one module at a time. The modules looked much better, and the final main.bicep was deployable.
The setup in brief
- LM Studio runs the model locally and exposes an OpenAI-compatible API in server mode. Ollama is an alternative for a more consistent long-term setup. LM Studio is easier for trying out different models.
- The Continue extension connects VS Code to the local model and gives you a Copilot-style chat and agent workflow.
- A rules file in
.continue/rules/ tells the model how to write Bicep.
- Small prompts. Generate one module at a time, then compose
main.bicep with the modules as context.
For installation steps and screenshots, see Juuso's original post.
FAQ
Is 32 GB of memory enough?
It was enough for this experiment when the work was split into smaller tasks. Juuso recommends leaving at least 10 GB of RAM free for other tasks. More powerful machines need less tweaking and can often complete tasks in a single prompt.
Why do LLMs struggle with Bicep?
Bicep is widely used in Azure environments, but it doesn't have the same broad ecosystem exposure as Python or JavaScript, so most LLMs aren't well trained on it. An instructions file fills that gap.
Do I still need an instructions file with better hardware?
Yes. Larger models can often complete tasks correctly in a single prompt, but a proper instructions file is still extremely valuable, even with more capable hardware.
