Nightmare.js library for data parsing, testing and more

The topic Web Scraping with the help of Node.js, we have already raised repeatedly. We looked at various libraries that offer many ready-made solutions for fast and efficient data parsing. Now we will dwell in more detail on the library Nightmare — high-level browser automation solution from Segment. Along with packages for data parsing, it also offers solutions for testing sites, automating the input of parameters to Internet resources. The closest analogues of this library — Casper.js, watir, phantom.js. There are other solutions that can be found in any search engine. But, along with them, Nightmare is distinguished by ease of use, full support for html5, no conflicts with sites, and the ability to expand through actions.

Now let's take a closer look at how the Nightmare library works and how it works, and its main advantages. Let's take a look at how it works with an example. Let us draw your attention to a number of points that must be considered when working with this library. We will show you how to ensure the stability and efficiency of work related to data parsing, resource testing and browser automation without risk and with maximum efficiency with the help of mobile proxies. But, first things first.

A brief introduction to the features of the Nightmare library

The Nightmare library at work is based on the Electron framework. For each page, it generates an object such as BrowserWindow, which in turn launches the Chromium browser shell.

In operation, this library is quite simple and logically built:

1. Nightmare creates a new app directly from the — the one that will be subjected to subsequent processing.
2. Scripts are loaded before the page being examined is loaded. This solution allows you to support two-way interaction between the programmer and the web page using a series of emitters.
3. The library provides the programmer with a set of – action chains. This makes it possible to perform various manipulations with the resource in order to obtain the required data.

Key Benefits of the Nightmare Library

If we talk about the advantages that the Nightmare library is endowed with, then first of all it is worth highlighting the following points:

• The code on the site side and on the client side is written in the same language. This solution eliminates the need for template engines.
• The ability to work with the Nightmare library both in browser display mode and as a background process headless browser.
• All commands are chains with the mandatory return of the promise of each of them. Thanks to this, you will be able to write code both directly in the style of crafts, and inside the generator, asing functions.
• Thanks to the ability to create actions, you can extend the modules, adjusting their capabilities to the work ahead. So, you can create an action at the Nightmare class level, or at the same time both at the Nightmare class level and at the Electron level. In the latter case, you get the opportunity to additionally use devapi Chromium in your work. Along with this, a fairly large selection of ready-made expansion modules is already built into the Nightmare library. You can connect any of them to your project. Alternatively, if you need to emulate mouse hovering when working with iframes, you can use the realMouse option. Without such a solution, the browser's security system would instantly block your actions.
• The minimum load on the memory and processor of your device. Let's make a reservation right away that we do not compare this library with elementary Post- and Get-requests. The fact is that in terms of speed and memory, classic browser parsers will lose significantly.
• You can enable or disable the display of images, webGL support, and many other features.
• It is possible to create preload scripts. In order to add your own libraries or individual options to the download page. So, if you wish, you can make adjustments to the addEventListener function, turning it into a decorator of a real function, as well as additionally connect a number of analytical options. The latter will help you check what the site actually does when users are on it. It can also be used to combat the intrusiveness of Fingerprint.
• Proxy support. It also provides for installing a user agent, setting browser extensions and a number of other options.

A little about the features of the Nightmare library using an example

This is how a quite classic application example would look like:

This example shows connecting the library and creating an object in browser visible mode. The whole process is written here, including the following options:

• going to the page;
• search by CSS selector for the required element;
• text input;
• button click;
• waiting for a new CSS selector to appear;
• function implementation on the browser side;
• function return after the chain of tasks in then is completed: either the result of the work, or the exception.

In this example, we see that the task is solved quite simply, quickly and conveniently. But, the situation will become more serious in the event that the page bypass script becomes quite impressive in volume. In this case, the above description of the commands will not be particularly convenient to use. For such cases, the following option would be more correct and reasonable:

Why did we consider this particular coding option? The fact is that it allows you to receive site data through evaluate as many times as you need. He will also be able to analyze them, applying certain behavioral scenarios to them, and at the same time describe this in a script.

Peculiarities of working with the Nightmare library

It makes no sense to give examples of all cases of using the Nightmare library in practice. It will be enough to carefully study its official documentation to understand everything personally. But what I would like to draw your attention to is the ability of the module to read any data, take screenshots, save pages in pdf and html, transfer data directly to the site. You can also use additional modules that allow you to upload files to the server using the form input type=”file” command. Also, the Nightmare library can respond to confirm, alert, prompt and broadcast data from the console as events.

But, you should also understand that any action will have only two decisions: committed or throwing an exception. In cases where you are not 100% sure that the code will pass, it will be optimal to wrap requests in a try catch, which will require appropriate handling. Let's take the wait(selector) function as an example. This instruction gives a command to pause the execution of the script until the html element appears with the required CSS selector. The default timeout present in the module can be optionally changed. If it occurs, an exception will be thrown, and you can analyze which element and why is not on the page, what measures should be taken to make it appear.

Ensuring stable work with the Nightmare library

Yes, the Nightmare library — it is quite a serious tool with broad functionality. It will be quite easy to learn, flexible in settings. With its help, you can solve a huge range of tasks in the field of site testing and analysis. But before moving on to direct work, you need to create a stable platform for unlimited actions on the network. And this can be ensured using mobile proxies from the MobileProxy.Space service.

In this case, you get a reliable and efficient tool that guarantees:

• stable and efficient networking without the risk of getting banned for multi-accounting, using software to automate actions;
• Effective bypass of regional blocking, access to any resources from around the world;
• the ability to test a site or a software product in a specific geolocation;
• complete confidentiality of online activities, protection against any unauthorized access, including hacker attacks;
• higher connection speed, which is provided by high-speed communication channels of mobile network operators, as well as data caching.

Using mobile proxies from the MobileProxy.Space service in your work, you can easily and simply change the IP-address by a timer in the range from 2 minutes to 1 hour, or by force using a link from your personal account. Also, directly in the workflow, you can make adjustments to the geolocation and the operator of cellular networks. Such proxies work simultaneously on the HTTP(S) and Socks5 protocols. To find out more information about this product, please follow the link https://mobileproxy.space/en/user.html?buyproxy. Please note: the different price of mobile proxies is related to the duration of the period for which you buy them. The longer it is, the cheaper one day of use will cost you.

Take advantage of the offers of the MobileProxy.Space service and see for yourself the high efficiency and convenience of these proxies, including when used in conjunction with the Nightmare library.

Share this article: