• Bytes Route logo

RESTFul APIs: Servers & Node.js

This is the third article in a five part series on RESTful APIs.

What is Node.js?

Node.js is an open-source runtime for JavaScript. It was created by Ryan Dall in 2009. Dahl criticized the limited possibilities of Apache HTTP Server and the sequential programming style. Node.js it's cross-platform, it can run on Unix, Windows, and macOS operating systems and 32/64 bit or ARM architectures.

The core proposition of Node.js is the language and non-blocking environment. The non-blocking environment makes Node.js a good choice for I/O intensive applications.

JavaScript developers can now, with the help of Node.js, create server-side applications.

With a team of JavaScript developers, a full-stack application can be built. The JSON format is widely used, especially in single-page applications.

Some NoSQL databases, like MongoDB, also use JSON and JavaScript. All in all is a great technology to be used.

The non-blocking nature is also a strong feature. Asynchronous programming practices like callbacks, promises, async/await, are first-class citizens in Node.js.

The possibility of sharing code between server and client is also a plus.

Using JavaScript and it's ecosystem a broad spectrum of applications can be built:

NPM

Node.js has a cool community and a lot of modules that everybody can use. For managing them, the NPM project was created.

Npm is an acronym for node package manager and acts as a package manager for node projects. Npm is also a CLI tool that aids with the installation.

All modules used in a project are defined in the package.json file and are installed in the node_modules folder. Due to the dependencies being defined in the package.json file committing the node_modules folder is bad practice, the folder should be added to .gitignore.

Choosing a good package for your problem implies checking how often the package is updated, how long bugs and issues remain unaddressed and how many people are using it, we do this to make sure it supports our requirements now and in the future.

You can check this article for a more detailed approach and more tips: NPM guide.

Internals

Runtime

Node.js is a JavaScript runtime. What does this mean? It means that it can interpret JavaScript. Cool, but that's what the browser is doing too. Node.js shares the V8 JavaScript engine with chromium-based browsers. The V8 engine is single-threaded but it is also asynchronous by using a concept called Event Loop.

Event Loop

The Event Loop is the architecture that makes this possible. And event-driven architecture that promotes loosely-coupled systems coupled with the queue system makes concurrency on a single thread possible.

 while(queue.waitForNextMessage()) {
   queue.processNextMessage();
 }

How? By delegating the actual work to the operating system internals or network and processing the next message.

For example, let's say that we send a request to a server, there is no point in blocking the thread until the data is transferred by the operating system and the network, processed by the other server and sent back.

Another example might be reading a file from the operating system, delegating the task to the operating system thread, the main thread can continue doing other things. When the operating system has data, an event is sent to the queue, the loop sends it to the main thread and it handles the rest.

The work is done by other threads of the operating system, routers, and/or another computer we just don't block our thread. In the meanwhile, we can process other messages. This is why asynchronicity works on a single thread. The problems appear when a task is CPU intensive. In CPU intensive tasks there is no work that the main thread can delegate to the OS or other computers, it has to do that itself.

I/O API

Having just the JavaScript engine is not enough. In browsers, we don't have direct access to operating system resources because of security reasons. So an API for accessing low-level resources of the operating system was needed. Node.js provides such API. Accessing resources like files, threads or the network is possible from JavaScript.

Programming style

Async Programming

The core of the Node.js programming style lays in its event-driven architecture and using the event loop and handlers.

Callbacks

What are handlers? handlers are unnamed functions that are passed as arguments to other functions. Usually, the other functions are event handling declarations.

We refer to a handler as a callback when there is an actual async operation involved. All callbacks are handlers, only handlers involved in async operations are callbacks.

The anonymous function passed as the second argument to the readFile function is the callback. The callback will be called back when the file is read. The reading of the file is an async operation.

 const fs = require('fs');
 fs.readFile('example.txt', function(err, content) {
   if (err) return console.error(err);
   console.log(content.toString());
 });

We can see in the example above the common error handling pattern for callbacks. The pattern says that the callbacks first parameter should be null if there is no error, or an error object if something wrong happened. The rest of the parameters can be used freely.

Promises

What are promises? Built on top of callbacks, promises are instances of the Promise class.

Promises are JavaScript objects that represent internally the state of an asynchronous operation. When the operation is completed, its value is contained in the Promise instance. The Promise internal state can be one of: pending, fulfilled, and rejected.

Pending means that the asynchronous operation is in progress. Fulfilled means that the operation completed successfully and that we can access the value. Rejected means that the operation failed.

A promise that is pending can be completed with one of the two possible states, fulfilled or rejected.

Promises can be chained and can be passed as arguments.

The Promise constructor receives as an argument a callback, function or arrow function, with two parameters to be called when the operations succedees and when it fails.

 const fs = require('fs');
 const readFilePromisified = new Promise(function (resolve, reject) => {
   fs.readFile('example.txt', function(err, content) {
     if (err) return reject(err);
     resolve(content.toString());
   });
 });
 readFilePromisified
 .then(function (data) { console.log(data); })
 .catch(function (err) { console.error(err); });
Async/Await

Built on top of promises is the concept of async/await. In order to be used you need a special kind of function or arrow function marked with the word async. The marking states that that function will return a promise no mather what.

Every function that returns a promise is awaitable.

 async function getCar(name) {
   if (name !== 'Fiat') throw new Error('We do not have this car');
   return 'Fiat 500'
 }
 async function callReadFilePromisified() {
    try {
      const car = getCar('fiat');
      console.log(car)
    } catch (err) {
       console.error(err);
    }
 }

Fitting together

We mentioned that everything is based on the event loop architecture. Callbacks are made possible by events, being handlers for them. Promises are built on top of callbacks. Promises are just proxies for a value that will be available in the future. Another layer of abstraction is brought by async/await feature which is built on top promises.

Every callback or handler is placed in a queue and waits for the event loop to call it when the necessary state is completed (file read, network request completed, etc.).

A handler queue is a data structure that Node.js uses internally to organize the async operations. A callback is added to the call stack when it is about to be executed. The event loop continually checks the call stack if it's empty so it can pic a callback from the queue and add it to the call stack.

There are several types of callback queues that handle different types of operations. IO queue, Timer queue, Microtask queue, Check/Immediate queue, and Close queue. It's important to note that the event loop checks and executes the microtask queue before other queues. The queue order is, microtask, timer, IO, check, and, lastly, close. Please refer to this article for more details, Deep dive into queues @ logrocket.

Events

Promoting low coupling, a lot of objects in Node.js, and every API in Node.js emits events. Anybody can create custom events.

 const EventEmitter = require('events').EventEmitter;

 const myEventEmitter = new EventEmitter();
 const eventHandler = () => { console.log('Handled event'); }
 myEventEmitter.on('myCustomEvent', eventHandler);
 myEventEmitter.emit('myCustomeEvent');

As we can see from the example above there is no need for async functionality but the event loop queue is still used.

A common error that causes memory leaks is not removing an event handler once it ceases to be useful. This is particularly bad if the event handler is set in a loop. We need to keep a reference of the event handler, this way the function that removes the handler, knows which one to remove.

 const eventHandler = () => { console.log('Handled event'); }
 myEventEmitter.on('myCustomEvent', eventHandler);
 myEventEmitter.removeEventListener('myCustomEvent', eventHandler);
Buffers & Streams
What are buffers?

Buffers are instance of class Buffer that are used to manipulate raw data, octets in memory.

You can do Unicode in JavaScript but, sometimes, you need to process binary data. Buffer class is here to help in processing octet streams. The output of the readfile function is a buffer.

We have more details in the unicode article.

What are streams?

Using streams is a faster way of processing data. If the data can be processed in parts, then you have a very good chance of speeding the operations via streams.

Think of it this way, reading-writing means that no work is done by the writing counterpart until the reading of the file is done. Writing and Reading at the same time means at least doubling the speed of the operations, but, in real life, you can gain an order of magnitude.

 const fs = require('fs');
 const readStream = fs.createReadStream('source.txt');
 const writeStream = fs.createWriteStream('destination.txt');
 readStream.pipe(writeStream);

That's all for now, next Persistency & MongoDB.