Asynchronous I/O Model
Blocking I/O vs. Non-Blocking I/O
- In operating systems, I/O operations are categorized into two types: blocking and non-blocking.
- Blocking I/O: The call does not return until all operations are completed at the system kernel level. For example, when reading a file, the call waits for the kernel to complete disk seeking, data reading, and copying data to memory.
- Non-Blocking I/O: The call returns immediately without waiting for operations like disk seeking, data reading, or copying to memory.
- Non-blocking I/O requires repeated calls (polling) to retrieve data.
- The epoll polling mechanism is the most efficient polling scheme. It sleeps when no I/O events are detected and wakes up when an event occurs.
- Node.js implements non-blocking asynchronous I/O using a thread pool, where some threads handle polling to retrieve data, while others perform computations. Data is passed between threads via communication.
- Node.js uses the libuv library to achieve asynchronous I/O on both *nix and Windows platforms.
Event Loop
Order of the Event Loop in Node.js
The event loop in Node.js follows this sequence:
External input data → Poll phase → Check phase → Close callbacks phase → Timers phase → I/O callbacks phase → Idle/Prepare phase → Poll phase (repeating in this order).
- Timers Phase: Executes callbacks for timers (
setTimeout,setInterval). - I/O Callbacks Phase: Handles unexecuted I/O callbacks from the previous loop cycle.
- Idle/Prepare Phase: Used internally by Node.js.
- Poll Phase: Retrieves new I/O events; Node.js may block here under certain conditions.
- Check Phase: Executes
setImmediate()callbacks. - Close Callbacks Phase: Handles close event callbacks for sockets.
Observers
In Node.js, events primarily originate from network requests, file I/O, etc., each associated with corresponding observers. The event loop follows a producer-consumer model: asynchronous I/O and network requests act as event producers, passing events to observers, which the event loop then retrieves and processes.
Request Object
During the transition from a JavaScript call to the completion of an I/O operation by the kernel, an intermediate entity called a request object is created. Callbacks are not invoked directly by developers but by the request object.
- JavaScript calls a Node.js core module.
- The core module invokes a C++ built-in module.
- The built-in module makes a system call via libuv, generating a request object that encapsulates parameters and methods from the JavaScript layer, including the callback function (stored in the
oncompleteproperty). - On Windows, the request object is pushed to the thread pool for execution.
Executing Callbacks
- Once an I/O operation in the thread pool completes, the result is stored in the
req->resultproperty, and the IOCP (Windows’ asynchronous I/O solution) is notified that the operation is complete. - During each tick of the event loop, the I/O observer calls IOCP-related methods to check for pending requests in the thread pool. If found, the request object is added to the I/O observer’s queue and processed as an event, completing the asynchronous I/O operation.
Asynchronous Programming Solutions
Publish-Subscribe Pattern
class EventEmitter {
private events = {}; // Stores events
private key = 0; // Unique key for events
on(name, event) {
event.key = ++this.key;
this.events[name]
? this.events[name].push(event)
: (this.events[name] = []) && this.events[name].push(event);
return this;
}
once(name, cb) {
let callback = (...args) => {
cb.call(this, ...args);
this.off(name);
};
this.on(name, callback);
return this;
}
off(name, key) {
if (this.events[name]) {
this.events[name] = this.events[name].filter((x) => x.key !== key);
} else {
this.events[name] = [];
}
return this;
}
emit(name, key) {
if (this.events[name].length === 0) throw Error(`Sorry, no ${name} listener defined`);
if (key) {
this.events[name].forEach((x) => x.key === key && x());
} else {
this.events[name].forEach((x) => x());
}
return this;
}
}Avalanche Problem
In scenarios with high traffic and concurrency, cache invalidation can cause a flood of simultaneous requests to hit the database, overwhelming it and slowing down the website’s overall response time.
// Using a partial function
// Example of on-demand loading
let after = function (times, cb) {
let count = 0,
results = {};
return function (key, value) {
results[key] = value;
count++;
if (count === times) cb(results);
};
};
const emitter = new EventEmitter();
let done = after(times, render);
emitter.on('done', done);
emitter.on('done', other);
fs.readFile(template_path, 'utf8', function (err, template) {
emitter.emit('done', 'template', template);
});
db.query(sql, function (err, data) {
emitter.emit('done', 'data', data);
});
l10n.get(function (err, resources) {
emitter.emit('done', 'resources', resources);
});Promise/Deferred Pattern
Promise.thenattaches callback functions.- Callbacks are executed by
resolveorrejectin the deferred object.
function MyPromise(constructor) {
let self = this;
this.status = 'pending';
this.value = undefined;
this.reason = undefined;
this.resolveQueue = [];
this.rejectQueue = [];
function resolve(value) {
if (self.status === 'pending') {
self.status = 'fulfilled';
self.value = value;
self.resolveQueue.forEach((fn) => fn());
}
}
function reject(reason) {
if (self.status === 'pending') {
self.status = 'rejected';
self.reason = reason;
self.rejectQueue.forEach((fn) => fn());
}
}
try {
constructor(resolve, reject);
} catch (e) {
reject(e);
}
}
MyPromise.prototype.then = function (res, rej) {
this.status === 'fulfilled' && res(this.value);
this.status === 'rejected' && rej(this.reason);
if (this.status === 'pending') {
this.resolveQueue.push(() => res(this.value));
this.rejectQueue.push(() => rej(this.reason));
}
};
let p = new MyPromise((res, rej) => {
setTimeout(() => res(1), 1000);
}).then((e) => console.log(e));Async and Await
async function fn() {
const a = await new Promise((res) => {
res(1);
});
console.log(a);
}
// Output: 1
