Why Deno's new sandboxes could let your app scale infinitely

7 min read

1857 words

Thoughts Deno Firecracker

An exciting new development has recently emerged in the Deno universe. The new sandbox project, which is currently in the alpha phase, allows any JavaScript code to be executed in an ephemeral environment. This means that user- or AI-generated code can be executed without the risk of malicious code being brought to the host system.

It's a combination of technologies that makes my nerd heart beat faster: Deno and Firecracker VMs combined.

The technology behind it: Firecracker MicroVMs

These sandboxes are based on Firecracker MicroVMs, the technology behind AWS Lambda Functions and AWS Fargate. Firecracker is a virtual machine monitor (VMM) that uses the Linux Kernel Virtual Machine (KVM) to create and run micro virtual machines (MicroVMs). Designed specifically for serverless workloads, it combines the security of hardware-based virtualisation with the resource efficiency and fast start-up times of containers.

Why is Firecracker so incredibly fast?

The secret lies in its minimalist design. Firecracker excludes all unnecessary devices and guest functions to reduce memory footprint and attack surface. Unlike traditional VMs, which emulate dozens of virtual devices (e.g. graphics cards, sound cards and USB controllers) that are not needed in serverless environments, Firecracker focuses on the essentials.

This is why its start-up times are so impressive: AWS Lambda functions start up in just 125 milliseconds! [1] This is around 40 times faster than a traditional VM, which takes several seconds to boot up.

KVM: The Foundation

KVM (Kernel-based Virtual Machine) is a hypervisor built into the Linux kernel. When enabled, it transforms the Linux kernel into a type 1 hypervisor, allowing virtual machines (VMs) to run with near-native performance. Both QEMU and Firecracker use KVM for CPU virtualisation, but Firecracker is much more lightweight.

Battle-tested in the AWS infrastructure

Firecracker is battle-tested and already forms the basis of several high-volume AWS services. This means that the technology behind Deno Sandboxes has been running stably on AWS for years, processing millions of requests every day. So you're not using experimental technology, but proven enterprise infrastructure.

These microVMs now come with Deno pre-installed. This enables you to execute any JavaScript, TypeScript or Deno-specific code, as well as ephemeral VSCode instances that remain accessible throughout the microVM's lifetime. For example, this would enable you to set up a remote IDE similar to GitHub Codespaces.

Sandbox specifications:

Each sandbox has the following specifications:

  • CPU: 1 vCPU
  • Memory: 512 MB
  • Disk: 10 GB

The sandboxes are hosted on the Deno Deploy infrastructure provided by the Deno Company. When you start the sandboxes, you can select the data centre in which your sandbox should be hosted. Currently, eu-west-1 and us-east-2 are available.

See the examples below for a code example.

How your app could scale endlessly as a result:

Nevertheless, sandboxes allow us to scale our apps more easily. For instance, if we want to avoid running computationally intensive processes in the main application, we can simply send such calculations to a Deno sandbox and process the response, which is returned via stdout or stderr. The commands we execute in the sandbox are based on promises, so we can process the results asynchronously without any hassle.

In theory, this would enable us to achieve unlimited scaling, since we can also scale our entire application. We can even host HTTP servers and publish ports on the internet. This enables us to provide each user with their own sandbox to handle their requests.

Examples

Let’s take a look at a couple examples.

User-specific environments

// Every user gets her own environment
async function createUserEnvironment(userId, region = "eu-west-1") {
  const sandbox = await Sandbox.create({ region });

  // User-spezifische Config laden
  await sandbox.writeTextFile("config.json", JSON.stringify({
    userId,
    permissions: await getUserPermissions(userId),
    settings: await getUserSettings(userId)
  }));

  const url = await sandbox.exposeHttp({ port: 3000 });
  return { sandbox, url };
}

Starting a sandbox

The region attribute is optional here; the default region is used by default (currently us-east-2):

await using sandbox = await Sandbox.create({region: "eu-west-1"});

Simple calculations

import { Sandbox } from "@deno/sandbox";

await using sandbox = await Sandbox.create();

const result = await sandbox.eval(`
	const fibonacci = [];
	let a = 0;
	let b = 1;

	for (let i = 0; i < 10; i++) {
	  [a, b] = [b, a + b];
	  fibonacci.push(a);
	}
	fibonacci
`) as number[];

for (const num of result) {
	console.log(num);
}

DNS Check

import { Sandbox } from "@deno/sandbox"

await using sandbox = await Sandbox.create();

const result = await sandbox.eval(`
  await Deno.resolveDns("deno.com", "A");
`)

console.log(result); // [ "69.67.170.170" ]

Fetching the data of a website or API

import { Sandbox } from "@deno/sandbox"

await using sandbox = await Sandbox.create();

const result = await sandbox.fetch("https://deno.com") as Response

console.log(result.status); // 200

HTTP Server in a Sandbox

Now it gets really interesting. We can run complete HTTP servers in sandboxes and make them accessible via the internet:

import { Sandbox } from "@deno/sandbox";

await using sandbox = await Sandbox.create();

await sandbox.writeTextFile(
  "server.ts",
  `import { Hono } from 'hono'

  const app = new Hono()

  app.get('/', (c) => c.json('Hello Deno!'))
  app.get('/user/:id', (c) => c.json({
    user: c.req.param('id'),
    timestamp: new Date().toISOString()
  }))

  Deno.serve({port:3000}, app.fetch)
`,
);

console.log("Installing deps...");
const npm = await sandbox.sh("deno install npm:hono", {
  stdout: "piped",
  stderr: "piped",
});
const npmOut = await npm.output();
if (!npmOut.status.success) {
  throw new Error(npmOut.stderrText);
}

console.log("Starting server...");
const server = await sandbox.createJsRuntime({
  entrypoint: "server.ts",
});

// Publish on the Internet
const publicUrl = await sandbox.exposeHttp({ port: 3000 });
console.log("Public URL:", publicUrl); // e.g. https://<random>.sandbox.deno.net

// Fetching from the local machine
const resp = await fetch(`${publicUrl}/`);
console.log(await resp.json());

console.log(await server.status)

VSCode instances with Deno Extension

This is still a fairly new feature and definitely not stable yet, but we are able to start a VSCode instance in the sandbox:

import { Sandbox } from "@deno/sandbox"

await using sandbox = await Sandbox.create();

await sandbox.upload("app", "/app")

const vscode = await sandbox.vscode("/app", {
  extensions: ["denoland.vscode-deno"]
})
console.log(vscode.url)

await vscode.status

After that, we can go to the page displayed in the console. For example, https://b9e0ea523bcc47e89be75d7fc31042b1.sandbox.deno.net. After a short time, the familiar VSCode interface should appear.

These are still quite unstable at the moment and occasionally lose connection (I mean as of this writing the package version is 0.0.12). This is probably due to the limited underlying resources of 1 CPU and 512 MB RAM. VSCode is not known for being the most resource-efficient editor.

Advanced use cases

Offload computationally intensive operations

Instead of blocking your main application, you can elegantly delegate CPU-intensive tasks to sandboxes:

async function processImageInSandbox(imageData) {
  await using sandbox = await Sandbox.create();

  const result = await sandbox.eval(`
    // run your computing intensive task here
    const processImage = (data) => {
      // 🦕🦕 doing complex stuff here
      return processedData;
    };

    processImage(imageData)
  `);

  return result;
}

Containers for AI-generated code

Another use case would be using containers for AI-generated code. Imagine cloning your project into the VM and granting the AI full access to the file system. Through branching, you could then develop asynchronous features created entirely by the AI.

async function aiCodeExecution(prompt) {
  await using sandbox = await Sandbox.create();

  // Execute AI-generated code in a secure environment
  const aiCode = await generateCodeFromPrompt(prompt);

  try {
    const result = await sandbox.eval(aiCode);
    return { success: true, result };
  } catch (error) {
    return { success: false, error: error.message };
  }
}

The question of whether this is the best way to create high-quality code is a whole other debate. However, it is a use case that does not put our host system at risk of destruction.

Multi-tenant architectures

This opens up completely new possibilities for SaaS apps:

class TenantManager {
  private tenantSandboxes = new Map<TenantId, Sandbox>();

  async getTenantSandbox(tenantId: TenantId): Promise<Sandbox> {
    if (!this.tenantSandboxes.has(tenantId)) {
      const sandbox = await Sandbox.create({
        region: (await this.getOptimalRegion(tenantId) || "eu-west-1")
      });

      // Tenant-specific configuration
      await this.setupTenantEnvironment(sandbox, tenantId);
      this.tenantSandboxes.set(tenantId, sandbox);
    }

    return this.tenantSandboxes.get(tenantId)!; // Non-null assertion since we just set it
  }
}

We can isolate environments even further and ensure an additional security mechanism.

Secure code playgrounds

Any apps that accept and execute user code would also become significantly more secure, as the code could run in isolation from any main application logic. This means we no longer have to check every user input for possible malicious code before executing it.

async function runUserCode(studentCode: string, testCases: TestCase[]): Promise<TestResult[]> {
  await using sandbox = await Sandbox.create();

  const result = await sandbox.eval(`
    ${studentCode}

    // Run test cases
    const results = [];
    ${testCases.map((test, i) => `
      try {
        const result = ${test.functionCall};
        results.push({
          test: ${i},
          result,
          expected: ${JSON.stringify(test.expected)},
          passed: JSON.stringify(result) === '${JSON.stringify(test.expected)}',
          description: "${test.description || ''}"
        });
      } catch (e) {
        results.push({
          test: ${i},
          error: e.message,
          passed: false,
          expected: ${JSON.stringify(test.expected)},
          description: "${test.description || ''}"
        });
      }
    `).join('')}

    results;
  `) as TestResult[];

  return result;
}

We can now run the user test cases in the sandboxes without hesitation and then view the results in our app.

Requirements and setup

Once we are part of the alpha and have access to the sandboxes, we can create a deploy token at https://console.deno.com/<YOUR_ORG>/~/settings. We need this to authenticate an application with Deno Deploy and start the sandboxes.

Getting started

import { Sandbox } from "@deno/sandbox";

// Set the token as an environment variable
Deno.env.set("DENO_DEPLOY_TOKEN", "YOUR_TOKEN");

await using sandbox = await Sandbox.create({
  region: "eu-west-1", // Optional
});

console.log("Sandbox is ready!");

Performance and limitations

Since the project is still in the alpha phase, we should assume that changes to the API or workflows may still occur. Furthermore, no information is yet available regarding the pricing model or different VM sizes. The latter could affect VSCode instances, for example.

While it is great to experiment with, it should not yet be considered production-ready.

Conclusion: A game changer?

The vision is fascinating: since each sandbox can host a complete application, and we can create as many sandboxes as we want, the scaling possibilities are theoretically unlimited. (For now, and as long as your credit card holds out.)

Although the project is still in the alpha phase, it demonstrates how we could approach application architecture. We could use Deno Deploy to manage the orchestration of our containers / sandboxes.

We can increase the isolation of our code through Firecracker microVMs, which already offer excellent services on the AWS platform.

I used it for some small prototypes and it worked great so far! The startup times are insane. It still had some unexpected errors, but this is expected as the project is still in the alpha phase. I am optimistic about the future and looking forward to seeing what the Deno Company has in store for sandboxes.

Thanks for reading!

Niklas

  1. Firecracker microVMs : the power behind AWS Lambda ↩︎