Thursday, March 30, 2017

Sneaky JavaScript Technics III

A ninja is a master of disguise. How do you apply the JavaScript ninjutsu to hide an object member so that nobody can see (and use) it? Here are several ways.

The context

These days, I started a new task on my gpf-js project and it implies manipulating an AST structure within NodeJS. Because the structure analysis requires to have access to the parent nodes, I augmented the AST items with a link to their parent. The result is later serialized into JSON for tracing purpose but the generated circular reference broke the conversion.

Actually, I could have used the replacer parameter of the JSON.stringify method but this circular reference also caused trouble in my own code. Hence I had to find another way.

Reflection

The JavaScript language offers some reflection mechanisms. Indeed, the for..in syntax is capable of listing all enumerable properties of any object.

I would recommend using Object.keys instead. However, the lack of support of old browsers requires that you polyfilled it with for..in combined with hasOwnProperty .

This fundamental machinery is widely used:

  • ESlint offers a rule to detect unsafe uses
  • JavaScript evolved to make the enumerable state configurable on a property

Object.defineProperty

The simplest way to add a property to an object is to simply do it: var obj = {}; obj.newProperty = "value";

This newly created property will be enumerable by default.

ECMAScript 5.1 introduced Object.defineProperty to create a property with options. This feature is implemented by most recent browsers with some limitations when using it on DOM objects.

It can be used in different ways and most of them are beyond the scope of this article. I will mostly focus on the possibility to create a non-enumerable property:

var obj = {}; Object.defineProperty(obj, "newProperty", { value: "value", writable: true, enumerable: false, configurable: false });

By setting enumerable to false, this property will not be enumerated when using the for..in syntax. By setting configurable to false, this property can't be deleted and can't be reconfigured to make it visible again.

The advantages are:

  • It is easy to implement
  • It is available since IE9

But, regarding the initial purpose, it comes with drawbacks:

To make the task harder for a hacker to figure out the property name, you may generate a complex random name and store it in a 'private' variable.

Symbol

Another simple way to add a property to an object is to use the square bracket syntax: var obj = {}; var newPropertyName = "newProperty"; obj[newPropertyName] = "value";

In that example, the newly created property will be named "newProperty" as per the value of the variable. The only difference with the dot syntax is the use of a variable to name the property.

But what if the variable is not a string?

For most standard objects (and primitive types), the variable value is converted to string. Consequently, the following syntax is valid (even if it does not make sense): var obj = {}; function any () {} obj[any] = "value";

The created property name will be "function any() {}"

This obviously means that you can use names that are not valid identifiers. Hence, it is mandatory to use the bracket syntax to access them.

However, there is one mechanism that behaves differently. It was introduced with ECMAScript 2015. Every time you call the Symbol function, it returns a unique value. This value type is primitive but it does not convert to string.

This feature is implemented only by most recent browsers and not by IE.

The advantages are:

  • It is easy to implement
  • There is no way to access the property unless you have the symbol value

Tuesday, March 28, 2017

Release 0.1.8

This new release introduces the interface concept and also prepares future optimizations of the release version.

New version

Here comes the new version:

Interfaces

A new entity type can now be defined: interface.

Being able to define interfaces and to check if an object conforms with their specification will normalize and improve the way encapsulations are handled within the library but also when using it.

Release optimization

When the library is built, some source manipulations occur:

Before this release, the debug and release versions were almost the same. The main differences were:

  • The release version is serialized without the comments (see rewriteOptions)
  • Also, the release version is minified using UglifyJS

Because I am planning to implement some optimization patterns on the release version, I started to redesign the build process. After converting all the relevant sources to ES6, I realized that the former AST manipulation was not working properly. Indeed, it was trying to rename all variables to shorten them but... it didn't work. Not a big deal because UglifyJS was already doing the job.

Finally, I rewrote the AST manipulation to start optimizing the concatenated AST structure.

Estimating functions usage

I implemented a detection of unused functions which revealed that _gpfIsUnsignedByte was not used! Hence, it is removed from the output.

NOP

The build process includes a pre-processor capable of conditioning some JavaScript lines using C-like defines. However, as a lazy developer, I don't want to wrap all DEBUG specific functions inside a #ifdef / #endif pair.

That's why some variables are tagged with a comment containing only gpf:nop. This is a way to signal that the variable represents a no operation.

For instance: javascript /*#ifdef(DEBUG)*/ // DEBUG specifics _gpfAssertAttributeClassOnly = function (value) { _gpfAsserts({ "Expected a class parameter": "function" === typeof value, "Expected an Attribute-like class parameter": value.prototype instanceof _gpfAttribute }); }; _gpfAssertAttributeOnly = function (value) { _gpfAssert(value instanceof _gpfAttribute, "Expected an Attribute-like parameter"); }; /* istanbul ignore if */ // Because tested in DEBUG if (!_gpfAssertAttributeClassOnly) { /*#else*/ /*gpf:nop*/ _gpfAssertAttributeClassOnly = _gpfEmptyFunc; /*gpf:nop*/ _gpfAssertAttributeOnly = _gpfEmptyFunc; /*#endif*/ /*#ifdef(DEBUG)*/ }

The optimizer is now capable of locating all variables flagged with gpf:nop and safely remove them from the output.

Automated release (final)

In the last release, I forgot one last step: closing the milestone. This is now completed.

Lessons learned

Performance measurement

When dealing with performances, one of the biggest challenge is to establish the point of reference. Indeed, if you want to quantify how much you gain, you need to make sure that the measurement environment is stable enough so that you can compare your results.

In my case, this is tough: the JavaScript hosts are evolving very fast and, since I started this project, I already changed my workstation twice. Furthermore, I don't have any performance specific tests.

So, I decided to take a pragmatic approach.

I reuse the test cases (that cover almost all the library) and compare the debug version with the release one. By quantifying the difference of execution between the two versions, this would give me a good indication on how much the release is optimized.

Today, both version demonstrates similar performances, even after implementing some of the optimizations.

jsdoc plugin

In the article My own jsdoc plugin, I explained how I tweaked jsdoc to facilitate generation of the documentation.

However, I noticed that the defineTags could be a better alternative to define custom tags. After trying to experiment it (knowing that jsdoc plugins are badly documented), it appeared that it is extremely limited:

  • No lookup function to access other existing doclets
  • Very limited information on the current doclet (for instance: I tried to implement @gpf:chainable but the memberOf property was not set). Indeed, we can't know when the onTagged is triggered (no control on the order).

Next release

The next release will introduce some features that are required for a side project I created.

Monday, March 6, 2017

Release 0.1.7

This new release secures the class mechanism and improves project tools.

New version

Here comes the new version:

Improved $super keyword

When writing the article My own super implementation I found several issues that were immediately fixed.

Better tooling

The following tools were modified:

  • It is now possible to remove unused files from the sources page
  • Each file modification triggers one pass of tests (it takes less than 1 second to execute). This way, I know as soon as possible if something goes wrong
  • fs middleware is secured to limit access to the project files only. It now supports the DELETE verb
  • The watch and serve tasks are monitoring the sources.json file modifications to update the list of linted files. This way, it is no more required to restart the grunt task.

More flavors for browser testing

Selenium was upgraded to version 3 and the detection has been fixed to make it more reliable.

On top of Selenium, a command line execution wrapper combined with a caching service (to store & grab the results) allows testing of non-automated browsers (such as Safari on Windows). Once the tests are done, the command line is killed.

Automated release

Releasing a version has never been so easy, a script using github-api module to call the GitHub API implements the following steps:

  • Check version number
  • Update package.json (if needed)
  • Check GitHub milestones to identify the milestone details and check that all issues are closed
  • Update README.md
  • Grunt make
  • Copy tmp/plato/report.history. to build/ (grunt copy:releasePlatoHistory)
  • commit & push
  • Create a new release on GitHub
  • Copy build/tests.js into test/host/legacy/{version}.js
  • commit & push

However, once last step was forgotten: closing the milestone. An incident is created.

Lessons learned

Documenting features makes them better

I will task the risk of repeating myself here but the article about super made me realize several mistakes in the implementation of the $super equivalent. Furthermore, taking the time to explore the ECMAScript super keyword gave me a better understanding of the feature.

In general, it is valuable to step back from the code and document the intent behind a feature.

Better Selenium detection

One of the reasons why I wanted to remove Selenium was the buggy detection. Indeed, there are some failures which are not encapsulated properly in a Promise. As a result, the whole process fails when it happens.

After digging on the web, I found this excellent thread on NodeJS Exception handling. It allowed me to handle those unmanaged exceptions the proper way and it secured the detection.

Next release

The next release will introduce the interface concept.