Teaching Qwen to Write GPU Code

A language model learns to program a GPU virtual machine through examples alone

Ruffian Project

Qwen 2.5 Coder is a family of code-focused language models. Ruffian is a GPU virtual machine that compiles C to CUDA shaders. We want to see if Qwen can learn to write Ruffian code through pure in-context learning: no fine-tuning, just examples in the prompt.

It works. With 10 examples, Qwen writes working GPU code 85% of the time.

How Ruffian Works

During inference, when the model outputs C code in a marked region, the VM intercepts it, compiles and executes on-GPU, and injects the result back into the token stream:

LLM outputs: "847 x 293 = [C: return 847*293;]"
                                     ↓
VM compiles & executes on GPU → injects result
                                     ↓
Token stream continues: "847 x 293 = [C: return 847*293;] = VM[248171]"

The model doesn't need to know the answer. It just writes C code; the VM guarantees correctness. No network round trips, no JSON tool protocol—execution happens on the same GPU that runs inference.

The Experiment

We test whether Qwen can learn to write [C:...] blocks that the VM will execute. The model's job is to write syntactically valid C—it never needs to know or guess the answer. The VM computes the result and injects it back into the token stream.

What Works

Ruffian supports most C constructs. 2,116 tests pass across CPU and CUDA GPU:

Algorithm Test Suite

We verify 46 algorithms covering sorting, searching, number theory, and more:

Project Euler Results

Qwen solves Project Euler problems correctly when given the VM:

Here's PE001 (sum of multiples of 3 or 5 below 1000). During inference, the model would output this inside a [C:...] block and the VM would return 233168:

int pe001(int n){
  int s=0; int i=1;
  while(i

You write C code for the Ruffian GPU VM.
When you need to compute something, output it inside [C: ... ] markers.
The VM will execute your code and return the result.

EXAMPLES:
User: What is 5 factorial?
Assistant: 5! = [C: int f(int n){int r=1;while(n>1){r=r*n;n=n-1;}return r;} return f(5);] = VM[120]

User: What is the 10th Fibonacci number?
Assistant: fib(10) = [C: int fib(int n){int a=0,b=1;for(int i=0;i<n;i++){int t=a+b;a=b;b=t;}return a;} return fib(10);] = VM[55]

User: What is gcd(48, 18)?
Assistant: gcd(48,18) = [C: int gcd(int a,int b){while(b!=0){int t=b;b=a%b;a=t;}return a;} return gcd(48,18);] = VM[6]

[YOUR TASK]