How I Built a Private, Offline AI Image Generator in Node.js (Full Guide)

In the era of cloud-hosted AI, we've grown accustomed to relying on subscription models and third-party servers to run generative models like Stable Diffusion. But what if you could run a fully functional, GPU-accelerated Stable Diffusion model on your local machine—completely offline, private, and with zero subscription costs?

In this guide, I'll walk you through how I built a private, offline AI image generator dashboard using Node.js, Express, Socket.io, and the Qvac SDK.

The app features:

·        100% Offline execution: Zero external API calls once the model weights are cached.

·        Real-time denoising progress: Streaming intermediate steps directly from the local C++ inference worker to the browser.

·        In-memory image delivery: Delivering generated images as memory-serialized Base64 Data URLs with zero disk writes for maximum privacy and performance.

·        Advanced customization: Adjustable sliders for steps, CFG scale, samplers, seeds, and custom negative prompts.

·        Automatic hardware fallback: Instantly falling back to CPU mode if a Vulkan/GPU driver crash is detected.

Here is a look at what the generator can produce locally:

Figure: Sample Output generated by Local Offline AI Image Generator

____________________________________________________

The Tech Stack

1.     Express & Node.js: The core backend web server.

2.     Socket.io: Establishes a bidirectional WebSocket channel to stream step-by-step denoising progress in real-time.

3.     Qvac SDK (`@qvac/sdk`): A lightweight SDK that manages local model downloads (via Hypercore) and spawns a native C++ runtime (`bare`) to perform fast tensor math on consumer GPUs (via Vulkan) or CPUs.

4.     Tailwind CSS: A clean, premium dark-mode user interface.

____________________________________________________

Step 1: Project Setup and Directory Structure

Code URL: https://github.com/ramsharma12345/image_generation-node

Create a clean directory and initialize a Node.js project:

mkdir local-image-gen
cd local-image-gen
npm init -y

Install the required packages. Note that @qvac/sdk installs native prebuilts for your platform:

npm install express socket.io @qvac/sdk

Configure your package.json to use ES modules by adding "type": "module":

{
  "name": "local-image-gen",
  "version": "1.0.0",
  "type": "module",
  "main": "server.js"
}

____________________________________________________

Step 2: The Backend Server (`server.js`)

We need a backend that loads the Stable Diffusion 2.1 model, validates the local cache, manages hardware configuration, runs inference, and streams steps to the client.

Here is the complete annotated backend code:

import express from 'express';
import path from 'path';
import http from 'http';
import { Server } from 'socket.io';
import fs from 'fs';
import { fileURLToPath } from 'url';
import { loadModel, unloadModel, getLoadedModelInfo, diffusion, SD_V2_1_1B_Q8_0 } from "@qvac/sdk";

// Redirect local model cache directory to another drive if main disk space is limited
process.env.USERPROFILE = 'D:\\qvac-home';

const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);

const app = express();
const server = http.createServer(app);
const io = new Server(server);

const PORT = process.env.PORT || 3000;
const CONFIG_PATH = path.join(__dirname, '.device-preference.json');

app.use(express.json());
app.use(express.static(path.join(__dirname, 'public')));

// Persistent configuration helpers
function getPreferredDevice() {
  try {
    if (fs.existsSync(CONFIG_PATH)) {
      const data = JSON.parse(fs.readFileSync(CONFIG_PATH, 'utf8'));
      return data.device || null;
    }
  } catch (err) {}
  return null;
}

function setPreferredDevice(device) {
  try {
    fs.writeFileSync(CONFIG_PATH, JSON.stringify({ device }), 'utf8');
  } catch (err) {}
}

let loadedModelId = null;
let modelLoadPercent = 0;
let modelLoadStatus = 'Awaiting trigger...';
let isModelLoading = false;
const modelSize = (SD_V2_1_1B_Q8_0.expectedSize / (1024 * 1024 * 1024)).toFixed(2) + ' GB';

function broadcastModelProgress(percent, status) {
  io.emit('model-download-progress', { percent, status, size: modelSize });
}

io.on('connection', (socket) => {
  console.log('Client connected:', socket.id);
  socket.emit('device-preference', { device: getPreferredDevice() || 'gpu' });

  // Handle Model Download & Initialization Check
  socket.on('trigger-model-download', async () => {
    if (loadedModelId) {
      try {
        await getLoadedModelInfo({ modelId: loadedModelId });
        socket.emit('model-download-progress', {
          percent: 100,
          status: 'Model fully loaded locally.',
          size: modelSize
        });
        return;
      } catch (err) {
        loadedModelId = null;
      }
    }

    if (isModelLoading) {
      socket.emit('model-download-progress', {
        percent: Math.round(modelLoadPercent),
        status: modelLoadStatus,
        size: modelSize
      });
      return;
    }

    isModelLoading = true;
    modelLoadPercent = 0;
    modelLoadStatus = 'Initiating cache check...';
    broadcastModelProgress(modelLoadPercent, modelLoadStatus);

    try {
      const preferredDevice = getPreferredDevice() || 'gpu';
      const loadConfig = { prediction: "v", device: preferredDevice };
      if (preferredDevice === 'cpu') loadConfig.threads = 4;

      // loadModel resolves instantly if model weights are already cached locally
      loadedModelId = await loadModel({
        modelSrc: SD_V2_1_1B_Q8_0,
        modelType: "sdcpp-generation",
        modelConfig: loadConfig,
        onProgress: (p) => {
          modelLoadPercent = p.percentage;
          modelLoadStatus = p.percentage >= 100
            ? 'Model loaded successfully.'
            : `Downloading weights... (${p.percentage.toFixed(1)}%)`;
          broadcastModelProgress(Math.round(modelLoadPercent), modelLoadStatus);
        }
      });

      isModelLoading = false;
      console.log('Model loaded. ID:', loadedModelId);
    } catch (err) {
      isModelLoading = false;
      modelLoadPercent = 0;
      modelLoadStatus = 'Failed to load model: ' + err.message;
      broadcastModelProgress(0, modelLoadStatus);
    }
  });

  // Switch Hardware Device (GPU vs CPU) dynamically
  socket.on('change-device', async (data) => {
    const { device } = data;
    setPreferredDevice(device);

    if (loadedModelId) {
      try {
        await unloadModel({ modelId: loadedModelId, clearStorage: false });
      } catch (err) {}
      loadedModelId = null;
    }

    isModelLoading = true;
    broadcastModelProgress(0, `Switching to ${device.toUpperCase()}...`);
    // Reload model with new device preference...
    // (See full source repository for the complete handler logic)
  });

  // Core Image Generation Handler
  socket.on('generate', async (data) => {
    const { prompt, steps, cfg_scale, negative_prompt, width, height, sampler, seed } = data;

    const runDiffusion = async (modelIdToUse) => {
      io.emit('progress', { percent: 0, status: 'Initializing...', sub: 'DIFFUSION INITIALIZING' });

      const options = { modelId: modelIdToUse, prompt };
      if (negative_prompt) options.negative_prompt = negative_prompt;
      if (steps) options.steps = parseInt(steps);
      if (cfg_scale) options.cfg_scale = parseFloat(cfg_scale);
      if (width) options.width = parseInt(width);
      if (height) options.height = parseInt(height);
      if (sampler) options.sampling_method = sampler;
      if (seed && parseInt(seed) !== -1) options.seed = parseInt(seed);

      const { progressStream, outputs, stats } = diffusion(options);

      // Stream denoising steps over WebSocket in real-time
      for await (const { step, totalSteps } of progressStream) {
        const percent = Math.round((step / totalSteps) * 100);
        io.emit('progress', {
          percent,
          status: `Denoising step ${step}/${totalSteps}...`,
          sub: 'RUNNING DIFFUSION'
        });
      }

      const buffers = await outputs;
      const base64Data = Buffer.from(buffers[0]).toString('base64');
      const dataUrl = `data:image/png;base64,${base64Data}`;

      // Emit final base64 string directly to browser memory
      io.emit('success', { url: dataUrl, prompt, seed: (await stats).seed || -1 });
    };

    try {
      await runDiffusion(loadedModelId);
    } catch (err) {
      console.error('Inference failed:', err.message);
      io.emit('error_event', { message: err.message });
    }
  });
});

app.get('*', (req, res) => {
  res.sendFile(path.join(__dirname, 'public', 'index.html'));
});

server.listen(PORT, () => {
  console.log(`Server running at http://localhost:${PORT}`);
});

____________________________________________________

Step 3: Resolving Windows Startup Latencies (Important Note)

When running local models via Node.js on Windows, the SDK spawns a native worker process (bare.exe) to execute the C++ tensor math. On the first launch, Windows Defender intercepts and scans this newly spawned binary.

By default, the SDK client has a 30-second initialization timeout. Under slow disk configurations or heavy scanning, the boot can timeout. To solve this, modify the initialization timeout constant in node_modules/@qvac/sdk/dist/client/rpc/node-rpc-client.js:

// Change this line from 30_000 to 120_000
const RPC_INIT_TIMEOUT_MS = 120_000;

This raises the timeout to 2 minutes, giving Windows Defender plenty of time to scan the binary on the first cold boot. Subsequent boots are nearly instantaneous.

____________________________________________________

Step 4: The Frontend Dashboard (`index.html`)

We'll place our HTML page inside public/index.html. We will create a dark-themed UI that manages advanced settings, resolution settings, aspect ratios, and seed inputs:

<!DOCTYPE html>
<html lang="en" class="dark">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Private AI Image Generator</title>
  <link rel="stylesheet" href="/style.css">
  <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.4.0/css/all.min.css">
</head>
<body class="bg-black text-white min-h-screen flex flex-col items-center justify-center p-4">
  <main class="w-full max-w-2xl border border-neutral-800 rounded-xl p-6 flex flex-col gap-6">
    <div class="text-center">
      <h1 class="font-bold text-2xl tracking-wider text-transparent bg-clip-text bg-gradient-to-r from-white to-neutral-500 uppercase">Local Generator</h1>
      <p class="text-[10px] text-neutral-500 font-mono mt-1">OFFLINE WORKSPACE</p>
    </div>

    <!-- Prompt Input -->
    <div class="flex flex-col gap-2">
      <label for="prompt-input" class="text-xs font-mono text-neutral-400">PROMPT</label>
      <textarea id="prompt-input" rows="3" class="w-full bg-neutral-900 border border-neutral-800 rounded-lg p-3 text-sm focus:outline-none focus:border-cyan-500 placeholder:text-neutral-700" placeholder="A futuristic city in the clouds..."></textarea>
    </div>

    <!-- Size selector, Sliders, CFG scale, Negative Prompt, Seed, and Hardware device -->
    <!-- (Detailed Advanced Settings details tag goes here) -->

    <button id="generate-btn" disabled class="w-full bg-cyan-600 hover:bg-cyan-500 text-white font-semibold py-3 rounded-lg flex items-center justify-center gap-2">
      <i id="generate-btn-icon" class="fa-solid fa-download"></i>
      <span id="generate-btn-text">Checking Cache...</span>
    </button>

    <!-- Canvas display, Progress spinner, and download buttons -->
  </main>
 
  <script src="/socket.io/socket.io.js"></script>
  <script src="/script.js"></script>
</body>
</html>

____________________________________________________

Step 5: How the Local Inference Behaves (My Experience)

When I click Generate, here is exactly how the system processes the request:

5.     GPU Vulkan Pipeline: On modern systems, the worker utilizes the GPU. If it is an integrated graphics chip (like an Intel Iris Xe), it compiles Vulkan pipeline shaders on the first execution.

6.     Streaming Progress: The progress is emitted step-by-step. If you configure a generation with 28 steps, Socket.io pushes progress objects (`Denoising step 1/28`, `2/28`, etc.) which render a live progress bar.

7.     CPU Fallback Mode: If you run out of GPU memory (VRAM) or experience a driver crash, the server automatically unloads the model and boots it on CPU (Safe Mode) using your core threads. CPU generation is highly stable and precise, eliminating driver-level artifacts, but requires about 8 minutes compared to 1–2 minutes on GPU.

____________________________________________________

Conclusion & Code Repository

Running AI models locally is no longer just for machine learning researchers. By compiling native backends and streaming progress using WebSockets, we can build responsive, offline-first web apps that put the power of generative art entirely on our own machines.

The complete code, CSS compilation utilities, and frontend script files are available in the project folder. Enjoy generating completely private art!