Most of all programming languages for creating server-side software come with multi-threading feature, however, NodeJS is single-threaded and can spawn new processes only. NodeJS has built-in process cloning which is called ‘Cluster‘; this is to handle enormous amount of requests on the same port. The number of nodes in a physical server should equal the number of hardware threads (not the number of cores, eg. the number of NodeJS nodes should be 8 for Intel i7 with 4 cores, 8 threads).

This ‘cluster’ solution doesn’t completely solve the multiple-heavy-request problem. With 8 NodeJS nodes in the cluster, only 8 heavy requests can be served, the rest of the requests is blocked. Only when NodeJS is upgraded with multi-threading feature, all requests will be served at slow speed instead of having only N requests served (N is the number of CPU hardware threads).

Another solution is to launch a NodeJS process for a heavy request received by ExpressJS.

Sample code:

const cluster = require('cluster');
const http    = require('http');
const numCPUs = require('os').cpus().length;

if (cluster.isMaster) {
  console.log(`Master ${process.pid} is running`);

  // Fork workers.
  for (let i = 0; i < numCPUs; i++) {     
    cluster.fork();   
  }   

  cluster.on('exit', (worker, code, signal) => {
    console.log(`worker ${worker.process.pid} died`);
  });
} 
else {
  // Workers can share any TCP connection
  // In this case it is an HTTP server
  http.createServer((req, res) => {
    res.writeHead(200);
    res.end('hello world\n');
  }).
  listen(8000);

  console.log(`Worker ${process.pid} started`);
}

Reference:

https://nodejs.org/api/cluster.html

Advertisements