Lesson 27-Deno Core Architecture

Deno Architecture Overview

Deno’s Design Goals

Deno was created as a response to reflections on Node.js shortcomings. In his 2018 talk, “10 Things I Regret About Node.js,” Ryan Dahl highlighted issues with Node.js’s module system, security, and dependency management. Deno’s design goals include:

  1. Security First: No file or network access by default; explicit permissions required.
  2. Modern Standards Support: Built-in TypeScript support, ES Modules compatibility, no need for additional build tools.
  3. Simplicity: Eliminates package.json and node_modules, using URL-based module imports.
  4. High Performance: Rewritten in Rust, leveraging the V8 engine and Tokio event loop.

These goals directly shape Deno’s core architecture.

Core Architecture Components

Deno’s architecture comprises several key layers:

  1. V8 JavaScript Engine: Parses and executes JavaScript/TypeScript code.
  2. Rust Core Layer: Written in Rust, handles OS interactions and resource management.
  3. Tokio Event Loop: Rust-based async runtime for I/O operations.
  4. Deno CLI: Command-line interface for developer experience (e.g., deno run, deno fmt).
  5. Built-in Tools and Standard Library: Includes TypeScript compiler, HTTP server, etc.

The following diagram illustrates Deno’s architecture:

+---------------------+
|   Deno CLI          |
| (run, fmt, test...) |
+---------------------+
|   Standard Library  |
| (http, fs, crypto..)|
+---------------------+
|   Rust Core         |
| (Bindings, Ops)     |
+---------------------+
|   Tokio Event Loop  |
| (Async I/O)         |
+---------------------+
|   V8 Engine         |
| (JS/TS Execution)   |
+---------------------+
|   Operating System  |
+---------------------+

Deep Dive into V8 Engine Integration with Deno

V8 Engine Overview

V8 is Google’s open-source JavaScript engine, used in Chrome and Node.js. It compiles JavaScript to machine code for high-performance execution. Deno relies on V8 to run JavaScript and TypeScript but builds a more modern runtime on top, unlike Node.js.

How Deno Uses V8

Deno interacts with V8 through Rust, using the deno_core library to bridge the JavaScript and Rust worlds. Let’s analyze this with a simple example.

Example Code: Hello World

console.log("Hello, Deno!");

When you run deno run hello.js, what happens?

  1. CLI Parses Command: Deno CLI invokes a Rust entry point to load hello.js.
  2. TypeScript Compilation (Optional): If the file is .ts, the built-in compiler converts it to JavaScript.
  3. V8 Execution: The JavaScript code is passed to V8, invoking the built-in console.log.

Underlying Analysis

In Deno’s source, deno_core provides a JsRuntime, encapsulating V8. Here’s a simplified simulation of its logic:

use deno_core::v8;
use deno_core::JsRuntime;

fn main() {
    // Create a new V8 Isolate (isolated execution environment)
    let mut runtime = JsRuntime::new(Default::default());

    // Define JavaScript code
    let code = "console.log('Hello, Deno!');";

    // Execute code in V8
    runtime.execute_script("<anon>", code).unwrap();
}
  • JsRuntime::new initializes a V8 Isolate.
  • execute_script passes JavaScript source to V8 for execution.
  • console.log is a predefined global function implemented via Rust-V8 bindings.

Binding Rust and JavaScript

Deno exposes Rust functions to JavaScript via “Operations” (Ops). For instance, console.log maps to a Rust logging function. Here’s a simplified binding example:

use deno_core::{op, OpDecl};

// Define a Rust operation
#[op]
fn op_print(msg: String) -> Result<(), deno_core::error::AnyError> {
    println!("{}", msg);
    Ok(())
}

// Register operation with V8
fn main() {
    let mut runtime = JsRuntime::new(Default::default());
    deno_core::extension!(print_extension, ops = [op_print]);
    runtime.register_extension(print_extension);

    let code = "Deno.core.ops.op_print('Hello from Rust!');";
    runtime.execute_script("<anon>", code).unwrap();
}
  • #[op] marks a Rust function as callable from JavaScript.
  • Deno.core.ops is an internal namespace for invoking registered operations.
  • op_print maps to Rust’s println!.

Tokio Event Loop and Asynchronous Processing

Tokio Overview

Tokio is a leading async runtime in the Rust ecosystem, used by Deno for network, file, and other async I/O operations. Unlike Node.js’s libuv, Tokio offers a modern async model based on Rust’s async/await.

Deno’s Event Loop

Deno’s event loop consists of two parts:

  1. V8 Microtask Queue: Handles JavaScript Promises and microtasks.
  2. Tokio Macrotask Queue: Manages async I/O (e.g., file reads, network requests).

Example Code: Async File Read

async function readFile() {
    const data = await Deno.readTextFile("example.txt");
    console.log(data);
}
readFile();

Workflow

  1. JavaScript Call: Deno.readTextFile is a built-in async API.
  2. Rust Operation: Maps to Rust’s op_read_file_async.
  3. Tokio Scheduling: Rust submits the file read task to Tokio, awaiting completion.
  4. Callback to V8: The result is passed back to JavaScript via a Promise.

Simplified Rust Implementation

use deno_core::{op_async, OpDecl};
use tokio::fs;

#[op_async]
async fn op_read_file_async(path: String) -> Result<String, deno_core::error::AnyError> {
    let content = fs::read_to_string(path).await?;
    Ok(content)
}
  • #[op_async] denotes an async operation.
  • tokio::fs provides async file operations.

Module System – URL Imports and Caching Mechanism

Module System Design Philosophy

Unlike Node.js’s node_modules and package.json, Deno uses URL-based module imports, offering:

  • Decentralization: No local npm package manager; modules load directly from the network.
  • Transparency: Clear dependency relationships, avoiding hidden dependencies.
  • Caching: Downloaded modules are cached locally to prevent redundant downloads.

Example: Module Import

import { assert } from "https://deno.land/std@0.224.0/assert/mod.ts";

assert(true);
console.log("Assertion passed!");

Running deno run script.js downloads and executes the module from the specified URL.

Module Loading Process

Deno’s module loading involves:

  1. Parsing Import Path: Identifies URLs or local file paths.
  2. Module Fetching: Downloads from the network or loads from cache.
  3. Module Compilation: Compiles TypeScript to JavaScript if needed.
  4. Module Execution: Passes to V8 for execution.

Caching Mechanism

Deno stores downloaded modules in a local cache (default: $HOME/.cache/deno). Use --reload to force re-download:

deno run --reload script.js

Cache directory structure:

$HOME/.cache/deno/
├── deps/
│   └── https/
│       └── deno.land/
│           └── std/
│               └── assert/
│                   └── mod.ts
└── gen/
    └── <compiled_files>
  • deps: Stores original module files.
  • gen: Stores compiled JavaScript files.

Source Analysis: Module Loader

Deno’s module loading logic, implemented in Rust (deno_core and deno_runtime), is simplified below:

use deno_core::ModuleSpecifier;
use deno_runtime::deno_fetch::reqwest;

async fn load_module(specifier: &str) -> Result<String, anyhow::Error> {
    // Parse module specifier
    let module_specifier = ModuleSpecifier::parse(specifier)?;

    // Check cache
    let cache_path = get_cache_path(&module_specifier);
    if cache_path.exists() {
        return Ok(fs::read_to_string(cache_path)?);
    }

    // Download from network
    let response = reqwest::get(specifier).await?.text().await?;

    // Save to cache
    fs::write(cache_path, &response)?;
    Ok(response)
}
  • ModuleSpecifier parses URLs.
  • reqwest handles HTTP requests.
  • Downloaded modules are cached.

Code Example: Custom Module Loader

Simulate Deno’s module loading:

// loader.js
async function loadModule(url) {
    const response = await fetch(url);
    const code = await response.text();
    const blob = new Blob([code], { type: "application/javascript" });
    const moduleUrl = URL.createObjectURL(blob);
    return import(moduleUrl);
}

const module = await loadModule("https://deno.land/std@0.224.0/assert/mod.ts");
module.assert(true);
console.log("Module loaded and executed!");

Run: deno run --allow-net loader.js

This demonstrates Deno’s core approach to loading remote modules via dynamic imports.

Permission System – Foundation of Sandbox Security

Permission System Design Philosophy

Deno’s security is a core feature. By default, scripts have no permissions (file, network, environment variables, etc.), requiring explicit command-line flags:

deno run --allow-read script.js

This avoids Node.js’s risk of unrestricted system access, resembling a browser’s sandbox model.

How Permission Checks Work

Deno implements permission checks in Rust. When a script accesses a restricted resource, a permission check is triggered. For example, Deno.readTextFile involves:

Rust-Side Implementation

use deno_core::{op, OpState};
use deno_runtime::permissions::Permissions;

#[op]
fn op_read_file(
    state: &mut OpState,
    path: String,
) -> Result<String, deno_core::error::AnyError> {
    // Check permissions
    let permissions = state.borrow::<Permissions>();
    permissions.read.check(&path.into())?;

    // Perform file read
    let content = std::fs::read_to_string(path)?;
    Ok(content)
}
  • Permissions stores the script’s permission state.
  • check verifies read permission, throwing an error if denied.

JavaScript-Side Call

try {
    const content = await Deno.readTextFile("example.txt");
    console.log(content);
} catch (err) {
    console.error("Permission denied:", err.message);
}

Without --allow-read, a PermissionDenied error is thrown.

Permission Configuration Example

Fine-grained permissions can be specified:

deno run --allow-read=/path/to/dir --allow-net=deno.land script.js
  • --allow-read=/path/to/dir: Allows reading only the specified directory.
  • --allow-net=deno.land: Allows network access to specific domains.

Source Analysis: Permission Parsing

Permission parsing occurs during CLI initialization:

use deno_runtime::permissions::PermissionsOptions;

fn parse_permissions(args: Vec<String>) -> PermissionsOptions {
    let mut opts = PermissionsOptions::default();
    for arg in args {
        if arg.starts_with("--allow-read=") {
            opts.read = Some(arg.replace("--allow-read=", "").into());
        }
    }
    opts
}
  • PermissionsOptions configures permissions.
  • CLI converts command-line arguments to permission settings for the runtime.

Standard Library – HTTP Server Implementation

Standard Library Overview

Deno’s standard library (https://deno.land/std@0.224.0) includes HTTP servers, file operations, and more. Let’s analyze the HTTP server implementation.

Example Code: Simple Server

import { serve } from "https://deno.land/std@0.224.0/http/server.ts";

await serve((req) => {
    return new Response("Hello, Deno!", { status: 200 });
}, { port: 8000 });

Run: deno run --allow-net server.js

Implementation Principle

The serve function relies on Deno’s TCP bindings and Tokio event loop. Simplified Rust implementation:

use tokio::net::TcpListener;

async fn serve_http(port: u16) -> Result<(), anyhow::Error> {
    let listener = TcpListener::bind(format!("0.0.0.0:{}", port)).await?;
    loop {
        let (stream, _) = listener.accept().await?;
        tokio::spawn(async move {
            // Handle HTTP request
            println!("New connection: {:?}", stream);
        });
    }
}
  • TcpListener listens on the specified port.
  • Each connection is handled by Tokio’s task scheduler.

Comparison with Node.js

Unlike Node.js’s http.createServer, Deno’s HTTP server uses Web standards (e.g., Response object) directly, eliminating intermediate layers.

TypeScript Compiler – Secrets of Built-in Support

TypeScript Support Design Philosophy

Deno’s built-in TypeScript support eliminates the need for tools like ts-node or tsc, reducing configuration overhead and making TypeScript “out-of-the-box.”

Example: TypeScript File

// example.ts
function greet(name: string): string {
    return `Hello, ${name}!`;
}

console.log(greet("Deno"));

Running deno run example.ts compiles and executes the code automatically.

Compilation Process

Deno’s TypeScript compilation involves:

  1. Source Parsing: Reads .ts file, generating an Abstract Syntax Tree (AST).
  2. Type Checking: Verifies type safety, producing diagnostics.
  3. Code Generation: Converts TypeScript to JavaScript.
  4. Caching: Stores compiled results locally to avoid redundant compilation.

Rust-Side Implementation

Deno integrates the TypeScript compiler (swc or typescript) in Rust, with core logic in the deno/cli/tsc module. Simplified compilation flow:

use deno_core::ModuleSpecifier;
use deno_ast::MediaType;
use deno_ast::parse_module;

fn compile_ts(specifier: &str, source: &str) -> Result<String, anyhow::Error> {
    // Parse module
    let parsed = parse_module(specifier, source, MediaType::TypeScript)?;

    // Type checking (optional)
    // Omitted here; handled by TypeScript or SWC

    // Convert to JavaScript
    let js_code = parsed.transpile()?;
    Ok(js_code)
}
  • deno_ast is Deno’s AST parsing library, based on swc (a high-performance Rust compiler).
  • transpile converts TypeScript to JavaScript, stripping type annotations.

Caching and Performance Optimization

Compiled JavaScript files are stored in $HOME/.cache/deno/gen, e.g.:

$HOME/.cache/deno/gen/file_hash/example.js

If the source file is unchanged, Deno loads the cached JavaScript, improving startup performance.

Code Example: Manual TypeScript Compilation

Simulate Deno’s compilation:

// compile.js
import { transpile } from "https://deno.land/x/emit@0.41.0/mod.ts";

async function compileTsCode(source) {
    const result = await transpile(source, { sourceName: "example.ts" });
    return result.get("example.ts");
}

const tsCode = `
function greet(name: string): string {
    return \`Hello, \${name}!\`;
}
console.log(greet("Deno"));
`;

const jsCode = await compileTsCode(tsCode);
eval(jsCode); // Output: Hello, Deno!

Run: deno run --allow-net compile.js

  • deno/x/emit mimics Deno’s compilation capabilities.
  • transpile converts TypeScript to executable JavaScript.

Deno CLI – Implementation Details of the Command-Line Tool

CLI Functionality Overview

Deno CLI is the primary interface for interacting with the runtime, supporting commands like:

  • deno run: Executes scripts.
  • deno fmt: Formats code.
  • deno test: Runs tests.
  • deno compile: Compiles modules.

These are implemented in Rust within the deno/cli directory.

CLI Working Principle

The CLI is a Rust executable that parses command-line arguments and dispatches to subcommand handlers. For example:

deno run --allow-read script.js

Rust Entry Point Analysis

Simplified CLI implementation:

use clap::Parser;
use deno_runtime::WorkerOptions;

#[derive(Parser)]
struct Cli {
    #[clap(subcommand)]
    command: Command,
}

#[derive(Parser)]
enum Command {
    Run {
        #[clap(long)]
        allow_read: bool,
        script: String,
    },
    // Other subcommands...
}

fn main() -> Result<(), anyhow::Error> {
    let cli = Cli::parse();
    match cli.command {
        Command::Run { allow_read, script } => {
            let mut options = WorkerOptions::default();
            if allow_read {
                options.permissions.read = Some(true.into());
            }
            run_script(&script, options)?;
        }
    }
    Ok(())
}

fn run_script(script: &str, options: WorkerOptions) -> Result<(), anyhow::Error> {
    // Initialize runtime, execute script
    println!("Running {}", script);
    Ok(())
}
  • clap parses command-line arguments.
  • WorkerOptions configures runtime permissions and behavior.

Subcommand Dispatch

Each subcommand (e.g., fmt, test) has dedicated logic. For example, deno fmt invokes the built-in formatter:

fn format_file(file: &str) -> Result<(), anyhow::Error> {
    let source = fs::read_to_string(file)?;
    let formatted = deno_fmt::format(&source)?;
    fs::write(file, formatted)?;
    Ok(())
}
  • deno_fmt uses dprint for formatting.

Code Example: Custom CLI

Build a simplified Deno CLI:

// custom_cli.js
function parseArgs(args) {
    const script = args[0];
    const flags = args.slice(1);
    return { script, allowRead: flags.includes("--allow-read") };
}

async function runScript(script, options) {
    if (options.allowRead) {
        const content = await Deno.readTextFile(script);
        eval(content);
    } else {
        console.error("Error: --allow-read required");
    }
}

const args = Deno.args.slice(1); // Ignore "deno run"
const { script, allowRead } = parseArgs(args);
runScript(script, { allowRead });

Run: deno run --allow-read custom_cli.js script.js

This demonstrates CLI argument parsing and script execution.

Source Code Analysis – Exploring the deno Repository

Repository Structure

Deno’s source is hosted on GitHub (deno_land/deno), with key directories:

  • ./cli: CLI implementation.
  • ./core: Core runtime logic.
  • ./runtime: Permissions, module loading, etc.
  • ./std: Standard library.

Key File Analysis

cli/main.rs

CLI entry point, initializes and dispatches:

fn main() {
    let args: Vec<String> = std::env::args().collect();
    if let Err(e) = deno::run_from_args(args) {
        eprintln!("Error: {}", e);
        std::process::exit(1);
    }
}

core/runtime.rs

Defines JsRuntime, bridging V8 and Rust:

pub struct JsRuntime {
    v8_isolate: v8::OwnedIsolate,
    // Other fields...
}

impl JsRuntime {
    pub fn new(options: RuntimeOptions) -> Self {
        let isolate = v8::Isolate::new(options.v8_options);
        JsRuntime { v8_isolate: isolate }
    }
}

Testing Framework – Implementation of deno test

Testing Framework Design Philosophy

Deno’s built-in testing framework, invoked via deno test, supports async tests and assertions natively, unlike Node.js’s reliance on tools like Mocha or Jest.

Example: Test File

// test.ts
Deno.test("simple test", () => {
    const sum = 2 + 2;
    if (sum !== 4) throw new Error("Math is broken!");
});

Deno.test("async test", async () => {
    const data = await Promise.resolve("Hello");
    if (data !== "Hello") throw new Error("Async failed!");
});

Run: deno test test.ts

Output:

running 2 tests
test simple test ... ok
test async test ... ok

Test Execution Process

deno test involves:

  1. File Discovery: Finds test files (matches *_test.ts, test.ts, etc.).
  2. Test Collection: Parses files, extracts Deno.test cases.
  3. Runtime Isolation: Runs each test in an isolated environment.
  4. Result Reporting: Summarizes and outputs results.

Rust-Side Implementation

Core logic resides in deno/cli/test_runner.rs. Simplified implementation:

use deno_core::JsRuntime;
use std::path::Path;

struct TestCase {
    name: String,
    code: String,
}

async fn run_tests(file: &Path) -> Result<(), anyhow::Error> {
    let source = fs::read_to_string(file)?;
    let tests = collect_tests(&source)?;

    for test in tests {
        let mut runtime = JsRuntime::new(Default::default());
        let result = runtime.execute_script(&test.name, &test.code);
        println!("test {} ... {}", test.name, if result.is_ok() { "ok" } else { "FAILED" });
    }
    Ok(())
}

fn collect_tests(source: &str) -> Result<Vec<TestCase>, anyhow::Error> {
    // Simulate parsing Deno.test calls
    Ok(vec![TestCase {
        name: "simple test".to_string(),
        code: source.to_string(),
    }])
}
  • collect_tests extracts test cases from JavaScript.
  • Each test runs in a separate JsRuntime for isolation.

JavaScript-Side Support

Deno.test is a global API, bound via Rust Ops:

use deno_core::{op, OpState};

#[op]
fn op_register_test(state: &mut OpState, name: String, code: String) {
    let tests = state.borrow_mut::<Vec<TestCase>>();
    tests.push(TestCase { name, code });
}

Deno.test calls trigger this operation, registering tests with the runtime.

Code Example: Custom Test Runner

Build a simplified test runner:

// test_runner.js
const tests = [];

function registerTest(name, fn) {
    tests.push({ name, fn });
}

async function runTests() {
    for (const test of tests) {
        try {
            await test.fn();
            console.log(`test ${test.name} ... ok`);
        } catch (err) {
            console.log(`test ${test.name} ... FAILED (${err.message})`);
        }
    }
}

// Define tests
registerTest("simple test", () => {
    if (2 + 2 !== 4) throw new Error("Math failed");
});

registerTest("async test", async () => {
    const data = await Promise.resolve("Hello");
    if (data !== "Hello") throw new Error("Async failed");
});

// Run tests
runTests();

Run: deno run test_runner.js

This illustrates the core concept of Deno’s testing framework: collecting and executing tests.

Web API Support – fetch and WebSocket

Web API Design Philosophy

Deno aligns with browsers, supporting standard Web APIs like fetch, WebSocket, and URL, enhancing code portability and reducing the learning curve. For example, Deno’s fetch mirrors the browser’s:

// fetch_example.js
const response = await fetch("https://api.example.com/data");
const data = await response.json();
console.log(data);

Run: deno run --allow-net fetch_example.js

fetch Implementation Principle

Deno’s fetch API is built on Rust’s HTTP client (reqwest) and Tokio for async requests.

Rust-Side Implementation

use deno_core::{op_async, OpState};
use deno_fetch::reqwest;

#[op_async]
async fn op_fetch(url: String) -> Result<String, deno_core::error::AnyError> {
    let response = reqwest::get(&url).await?.text().await?;
    Ok(response)
}
  • op_fetch is an async operation, executing network requests via Tokio.
  • The response string is wrapped in a Promise for JavaScript.

JavaScript-Side Binding

fetch relies on Deno.core.ops to call the underlying operation:

globalThis.fetch = async (url) => {
    const text = await Deno.core.ops.op_fetch(url);
    return new Response(text);
};

The actual implementation is more complex, supporting headers, methods, etc., but follows this pattern.

WebSocket Support

Deno supports WebSocket, e.g.:

// websocket.js
const ws = new WebSocket("ws://localhost:8080");
ws.onmessage = (event) => console.log("Message:", event.data);
ws.onopen = () => ws.send("Hello from Deno!");

Run: deno run --allow-net websocket.js

Rust-Side Implementation

use deno_websocket::tokio_tungstenite;
use tokio::net::TcpStream;

async fn connect_websocket(url: String) -> Result<(), anyhow::Error> {
    let (ws_stream, _) = tokio_tungstenite::connect_async(&url).await?;
    println!("WebSocket connected");
    Ok(())
}
  • tokio_tungstenite handles WebSocket communication in Rust.

Comprehensive Source Analysis – Key Files Deep Dive

runtime/permissions.rs

Core permission system implementation:

pub struct Permissions {
    pub read: PermissionState,
    pub net: PermissionState,
    // Other permissions...
}

impl Permissions {
    pub fn check_read(&self, path: &Path) -> Result<(), deno_core::error::AnyError> {
        if self.read == PermissionState::Denied {
            bail!("PermissionDenied: read access to {:?}", path);
        }
        Ok(())
    }
}
  • PermissionState defines Granted, Prompt, and Denied states.

cli/tsc.rs

TypeScript compilation logic:

pub fn compile_source(source: &str, specifier: &str) -> Result<String, anyhow::Error> {
    let parsed = deno_ast::parse_module(specifier, source, MediaType::TypeScript)?;
    let js = parsed.transpile()?;
    Ok(js)
}
Course: Mastering Core Concepts of Bun.js, Deno.js, and Winter.js
Share your love

Trending now

Exploring Web 3D Core Technologies Development
WebRTC Advanced Guide：Media, Transport & Architecture
WebSocket Deep Guide：Building High-Performance Realtime Systems
Micro Frontends & Serverless: Modular Web Development