tag:blogger.com,1999:blog-33880386192076397932024-02-19T00:40:45.047+01:00Javascript and developmentArnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.comBlogger52125tag:blogger.com,1999:blog-3388038619207639793.post-59353765376080303392019-10-30T03:38:00.003+01:002019-10-30T03:38:43.615+01:00Release 1.0.0: First productive version<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/release.png" align="left">
This release is required for the first application based on the library.
</div>
<div class="code markdown"><h2>Release 1.0.0: First productive version</h2><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/18?closed=1" target="_blank">Stories and bugs</a> implemented</li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/tree/v1.0.0" target="_blank">Sources</a></li><li><a href="https://arnaudbuchholz.github.io/gpf/doc/index.html" target="_blank">Documentation</a></li><li><a href="https://www.npmjs.com/package/gpf-js" target="_blank">NPM package</a></li></ul><h2>Release content</h2><h3>Bug fixes</h3><p>This release took a long time to mature but, in the end, does not contain much.</p><p>I already mentionned in the last releases that I am working on a side project. It's been a while. This development revealed many problems in the library that were addressed in different releases (such as the <a href="http://gpf-js.blogspot.com/2019/03/release-029-es6-support.html" target="_blank">support
of ES6</a>).</p><p>Because the pressure increased, the focus completely shifted to this project on the last months and several little bugs were fixed during that time.</p><p>Now that the project is ready for production, an 'official' release of thoses bug fixes is required.</p><p>Finally, it also validates the use of the library in a productive environment.</p><h2>Next release</h2><p>The <a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/19" target="_blank">next release content</a> <em>may be</em> about performances.</p><p>Several other projects requiring my attention, the library might be put on the back burner for a while.</p></div>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-27934694198498760962019-03-14T05:02:00.002+01:002019-03-14T05:02:56.938+01:00Release 0.2.9: ES6 Support<br>
<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/release.png" align="left">
This release mainly introduces ES6 support as well as improvements to the serialization helpers.
A new flavor is created for Node.js users.
</div>
<div class="code markdown"><h2>Release 0.2.9: ES6 Support</h2><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/17?closed=1" target="_blank">Stories and bugs</a> implemented</li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/tree/v0.2.9" target="_blank">Sources</a></li><li><a href="https://arnaudbuchholz.github.io/gpf/doc/index.html" target="_blank">Documentation</a></li><li><a href="https://www.npmjs.com/package/gpf-js" target="_blank">NPM package</a></li></ul><h2>Release content</h2><h3>ES6 support</h3><p>While working on a side project which is based on Node.js, I realized that <strong>the library did not support ES6 classes</strong>. Not only the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.html#.define__anchor" target="_blank">gpf.define</a> API was not able to extend any of them (even if it does not really make sense) but it was also <strong>impossible to add attributes</strong> to a class that was not previously created with <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.html#.define__anchor" target="_blank">gpf.define</a> (which is more problematic).</p><p>After doing a quick test, a solution was drafted to detect and handle these classes the right way. It is all explained in the article <a href="http://gpf-js.blogspot.com/2019/01/how-i-learned-from-crazy-idea.html" target="_blank">How I learned from a crazy idea</a>.</p><p>The <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-DEFINE.html" target="_blank">$singleton and $abstract</a> syntaxes were adapted accordingly.</p><div class="note"><p> It is clearly <strong>not recommended</strong> to extend ES6 class using <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.html#.define__anchor" target="_blank">gpf.define</a>.</p></div><p>In order to integrate attributes properly, a quick look in the <strong>coming ES6 features</strong> pointed out the fact that <strong><a href="https://medium.com/google-developers/exploring-es7-decorators-76ecb65fb841" target="_blank">decorators</a></strong> are used to qualify class members. Hence, an <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.attributes.html#.decorator__anchor" target="_blank">attribute decorator</a> was created.</p><p>Last but not least, since decorators are not yet supported without <a href="https://scotch.io/tutorials/javascript-transpilers-what-they-are-why-we-need-them" target="_blank">transpiling</a>, the library allows <strong><a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.typedef.html#.requireOptions" target="_blank">preprocessing</a> of resources</strong> so that decorators can be substituted with <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/bb6936760963f7dc4fc942da1eadeead5976b3b1/test/require/es6/class_with_attributes.js#L45" target="_blank">manual call of the decorator</a>.</p><p>This was also the opportunity to <strong>refactor</strong> in depth the validation of the require configuration options.</p><h3>Improved serialization</h3><p>The side project is <strong>extensively using <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.attributes.Serializable.html" target="_blank">serialization attributes</a></strong>. Quickly, the need for <strong>code simplification</strong> became obvious.</p><p>First, it does not make sense to repeat the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.typedef.html#.serializableProperty" target="_blank">property name</a> when it can be easily <strong>deduced</strong> from the member the attribute is assigned to.</p><div class="note"><p> When set on a 'private' member, the result property name won't include the underscore.</p></div><p>Then, these attributes are used in a context where serialization is used to implement an <strong><a href="https://www.odata.org/" target="_blank">ODATA</a> service</strong>. Consequently, they are used to <strong>describe</strong> how the data should be sent back the client but also how it is received.</p><p>For instance, an entity unique identifier must be transmitted to the client but it will never be modified by the client.</p><p>With the introduction of the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.typedef.html#.serializableProperty" target="_blank">readOnly property</a>, it is possible to make this distinction and have <strong>asymmetric serialization</strong>.</p><p>But, as for names, it does not make sense to repeat something that is <strong>already built in the class</strong>. Indeed, with the use <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/defineProperty" target="_blank">Object.defineProperty</a> - or using ES6 class getter / setter - it is possible to define the (get, set) couple and, when not setter exists, configure read-only members.</p><p>That's why, <strong>when the hosts supports it</strong>, the serialization code will leverage <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/getOwnPropertyDescriptor" target="_blank">Object.getOwnPropertyDescriptor</a> recursively on the class hierarchy to <strong>determine if the member is read-only</strong>.</p><h3>Improved compatibility</h3><p>Browser's <strong>base64 helpers</strong> (<a href="https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/atob" target="_blank">atob</a> and <a href="https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/btoa" target="_blank">btoa</a>) were added to the <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-COMPATIBILITY.html" target="_blank">compatibility</a> layer.</p><p>The Function.prototype.compatibleName method has been <strong>removed</strong> since it induced an <strong>extension of the Function prototype</strong>. Usually, libraries should avoid doing that because it is <strong>against best practices</strong>.</p><p>Because of the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.http.html#.mock__anchor" target="_blank">mocking</a> implementation, the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.http.html#.request__anchor" target="_blank">gpf.http.request</a> was <strong>limited</strong> in terms of which <a href="https://www.restapitutorial.com/lessons/httpmethods.html" target="_blank">http methods</a> could be used. Some hosts do not support custom verbs (and this is <strong>documented</strong> in the <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-COMPATIBILITY.html" target="_blank">compatibility page</a>) but browsers & node supports almost any verb. The code was modified to allow the use of <strong>custom verbs</strong>.</p><p>Surprisingly, the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/substr" target="_blank">String method .substr</a> is documented as <em>"to be avoided when possible"</em>. Since it was massively used in the sources, an ESLint <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/.eslintrules/no-substr.js" target="_blank">custom rule</a> was developed and the code reworked.</p><h3>New flavor node</h3><p>A <strong>Node.js flavor</strong> was created and is used as the <strong>default library</strong> being loaded when using <code class="javascript gpf-blog"><span class="identifier">require</span><span class="symbol">(</span><span class="string">"gpf-js"</span><span class="symbol">)</span></code></p><p>It implements <strong>all features</strong> including the compatibility's atob and btoa.</p><h2>Lessons learned</h2><h3>asymmetric Serialization</h3><p>The user story <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/299" target="_blank">asymmetric serialization</a> required several updates since it was pretty difficult to find the right <strong>balance between simplicity of use and flexibility</strong>. In the end, this feature is really powerful when applied with a converter function. Indeed, this is the place where one can control if the value will be serialized or not.</p><h3>Refactoring of classes</h3><p>Integrating ES6 classes was only the visible part of the challenging iceberg. Actually, <strong>the library was suffering from a structural defect regarding how classes were handled</strong>.</p><p>Initially, each class was associated to a <strong>class definition</strong> created only when using <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.html#.define__anchor" target="_blank">gpf.define</a>. This object <strong>holds important information</strong> such as the list of attributes.</p><p>When subclassing, the parent class definition was looked up by searching the one that matches the condition <code class="javascript gpf-blog"><span class="identifier">instanceBuilder</span><span class="symbol">.</span><span class="identifier">prototype</span><span class="space"> </span><span class="keyword">instanceof</span><span class="space"> </span><span class="identifier">entityDefinition</span><span class="symbol">.</span><span class="identifier">getInstanceBuilder</span><span class="symbol">(</span><span class="symbol">)</span></code> (<em>see <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.8/src/define/build.js#L30" target="_blank">full code</a></em>)</p><p>As a result, you could have classes in the hierarchy that were invisible because not created with the library.</p><p>To solve this issue, a new code was put in place to <strong>import any class as well as its hierarchy up to the root class</strong> (i.e. <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object" target="_blank">Object</a>). It also means that <strong>base classes are now associated with a class definition</strong> during the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/src/define/class/standard.js" target="_blank">startup of the library</a>.</p><p>This also implies that the library may have to deal with anonymous functions when importing a class.</p><p>It is still not possible to use gpf.define without a class name but, internally, <strong>the library can import <em>any</em> class</strong>.</p><h3>Refactoring of tests</h3><p>Introduction of ES6 in the library had a <strong>significant impact on how the tests are executed</strong>.</p><p>Indeed, it is mandatory to check <strong>if the host is really supporting the ES6 class syntax</strong> before trying to create one.</p><p>So a new algorithm was built to:</p><ul><li><strong>detect features</strong> (with the possibility to override them, like for nodewscript), result is transmitted in a global</li></ul><p>object available during the tests</p><ul><li><strong>include test files</strong> dynamically</li></ul><h3>Improved flavor mechanism</h3><p>Writing the Node.js flavor was harder than expected. The main struggle came from the <strong>inclusion of base64 helpers without getting the whole compatibility layer</strong>. Furthermore, without the compatibility layer, the <em>compatibleName</em> function member was no more available. This broke the code at many places. That's why it was decided to replace it with an <strong>internal helper</strong> to extract the function name where needed (it points to the name property by default).</p><p>Also, a <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/make/flavor.html" target="_blank">flavor debugging page</a> was created to <strong>ensure that any update on the flavor algorithm would fit the expectations</strong>.</p><h3>New eslint rules</h3><p>As mentioned before, a <strong>custom eslint rule</strong> was created to forbid the use of .substr: <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/.eslintrules/no-substr.js" target="_blank">no-substr</a>.</p><p>The same way, another rule was created to ensure that when a module has no function, a default one is being created: <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/.eslintrules/no-empty-module.js" target="_blank">no-empty-modules</a>.</p><div class="note"><p> One weakness of <a href="https://www.npmjs.com/package/plato" target="_blank">plato</a> is the evaluation of a module with no function</p></div><p>As the linters are applied every time a module is modified, <strong>more custom rules will be created to solve common problems</strong> (such as dependencies update).</p><h3>Release notes</h3><p>Today, there are <strong>more than 14 releases</strong> for the library. It takes some time to access the <strong>release notes</strong> since one has to go to the <a href="https://github.com/ArnaudBuchholz/gpf-js/releases" target="_blank">release information</a> in GitHub in order to find them.</p><p>It was decided to change the <a href="https://github.com/ArnaudBuchholz/gpf-js#versions" target="_blank">readme</a> to embed a direct link to each note.</p><p>However, regarding the last version, its notes are usually <strong>written after the release was created</strong>. A <a href="http://gpf-js.blogspot.com/p/release-notes-are-coming.html" target="_blank">page</a> was built to redirect the reader <strong>when the notes are out</strong>.</p><h2>Next release</h2><p>The <a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/18" target="_blank">next release content</a> is about performances. It's been a while I wanted to manipulate the release code to inline functions as much as possible and substitute loops for performances.</p><p>Still, I need to work on the side project because it <strong>really</strong> requires all my attention.</p></div>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-33630720717712409182019-01-25T00:16:00.000+01:002019-01-25T00:16:13.679+01:00How I learned from a crazy idea<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/student.png" align="left">
According to wikipedia, a particle accelerator is a machine that uses eletromagnetic fields to propel charged
particles to very high speed and energies. A collider accelerator causes them to collide head-on, creating
observable results scientists can learn from.
<br>
Sometimes, I feel like an idea collider producing experiments I can learn from.
</div>
<div class="code markdown"><p><a href="https://arnaudbuchholz.github.io/blog/post/How%20I%20learned%20from%20a%20crazy%20idea/5680466430_fb62b00e17_b.jpg" target="_blank">CERN / LHC tunnel<br><img src="https://arnaudbuchholz.github.io/blog/post/How%20I%20learned%20from%20a%20crazy%20idea/5680466430_fb62b00e17_b.jpg" alt="CERN / LHC tunnel" title="CERN / LHC tunnel" border="0" style="max-width: 100%"></a> (photo of CERN / LHC tunnel from <a href="https://www.flickr.com/photos/arselectronica/5680466430" target="_blank">Ars Electronica</a>)</p><h2>The idea</h2><p>I have <strong>millions of ideas</strong>: some are stupid <em>(and it's ok)</em> and some <em>may</em> be interesting. Sadly, for most of them, <strong>time and resources are missing</strong> to shape and mature them properly. Somehow, we must <strong>choose our battles</strong>.</p><p>On rare occasions, <strong>that crazy idea which doesn't make sense</strong> comes up... and it would be really <strong>cool to try</strong> it.</p><p>This is more or less <strong>how the <a href="https://github.com/ArnaudBuchholz/gpf-js" target="_blank">GPF-JS library</a> project started</strong> several years ago. Building a library supporting most of the hosts existing at that time and that would allow experimenting some cool concepts with JavaScript was appealing (classes, interfaces, streams, documentation generation, TDD, code coverage, backward compatibility testing...).</p><p>So far, it is a success since <strong>a lot was learned from that experience</strong>:</p><ul><li><a href="https://gpf-js.blogspot.com/2018/01/my-own-require-implementation.html" target="_blank">My own require implementation</a></li><li><a href="https://gpf-js.blogspot.com/2017/06/5-ways-to-make-http-request.html" target="_blank">5 ways to make an http request</a></li><li><a href="https://gpf-js.blogspot.com/2017/02/my-own-super-implementation.html" target="_blank">My own super implementation</a></li><li><a href="https://gpf-js.blogspot.com/2016/12/my-own-jsdoc-plugin.html" target="_blank">My own jsdoc plugin</a></li><li><a href="https://gpf-js.blogspot.com/2016/08/my-own-templates-implementation.html" target="_blank">My own templates implementation</a></li><li><a href="https://gpf-js.blogspot.com/2016/02/date-override.html" target="_blank">Date override</a></li><li><a href="https://gpf-js.blogspot.com/2015/10/my-own-bdd-implementation.html" target="_blank">My own BDD implementation</a></li><li>And all the <a href="https://github.com/ArnaudBuchholz/gpf-js/releases" target="_blank">release notes</a> of the library</li></ul><p>These days, I am working on a side project that requires <strong>a backend to hold the data</strong>. Obviously, the implementation started with NodeJS as it is good opportunity to push my <strong>ES6 knowledge</strong> a little bit further.</p><p>The project reached the point where <strong>some features of the GPF-JS library could be leveraged</strong>.</p><p>This means that the library needs to <strong><a href="https://github.com/ArnaudBuchholz/gpf-js/issues/303" target="_blank">support ES6 code</a></strong>.</p><p>However, since it is designed to be compatible with so many hosts <em>(and some of them are deprecated)</em>, it somehow sets the language support to quite a <strong>low standard (some may say 'old' JavaScript)</strong>.</p><div class="note"><p> <a href="https://scotch.io/tutorials/javascript-transpilers-what-they-are-why-we-need-them" target="_blank">Transpiling</a> could have been an option but some hosts are not even supporting the resulting ES5 code.</p></div><p>Among the <a href="https://arnaudbuchholz.github.io/gpf/doc/index.html" target="_blank">exposed api</a>, there is an entry point to <strong>define classes: <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.html#.define__anchor" target="_blank">gpf.define</a></strong>. Inheritance is specified by setting the <em>$extend</em> property in the <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-DEFINE.html" target="_blank">entity definition</a>.</p><p>The library even offers a <a href="https://gpf-js.blogspot.com/2017/02/my-own-super-implementation.html" target="_blank">$super</a> helper to <strong>facilitate calling the base class method</strong> whenever it makes sense.</p><p>So far so good.</p><p>But <strong>what would happen if</strong> one applies this API with an ES6 class?</p><p><code class="javascript gpf-blog"><span class="identifier">const</span><span class="space"> </span><span class="identifier">gpf</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">require</span><span class="symbol">(</span><span class="string">"gpf-js"</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">class</span><span class="space"> </span><span class="identifier">A</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">constructor</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">this</span><span class="symbol">.</span><span class="identifier">_a</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="string">"A"</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="identifier">const</span><span class="space"> </span><span class="identifier">B</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">define</span><span class="symbol">(</span><span class="symbol">{</span><span class="space">
</span><span class="identifier">$class</span><span class="symbol">:</span><span class="space"> </span><span class="string">"B"</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">$extend</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">A</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">constructor</span><span class="symbol">:</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">this</span><span class="symbol">.</span><span class="identifier">$super</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">this</span><span class="symbol">.</span><span class="identifier">_b</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="string">"B"</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="space">
</span><span class="identifier">const</span><span class="space"> </span><span class="identifier">b</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">B</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span></code></p><p>The result is:</p><p><code class="javascript gpf-blog"><span class="identifier">TypeError</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">Class</span><span class="space"> </span><span class="identifier">constructor</span><span class="space"> </span><span class="identifier">A</span><span class="space"> </span><span class="identifier">cannot</span><span class="space"> </span><span class="identifier">be</span><span class="space"> </span><span class="identifier">invoked</span><span class="space"> </span><span class="identifier">without</span><span class="space"> </span><span class="string">'new'</span></code></p><div class="note"><p> Before going any further, it is important to check that your browser supports the ES6 syntax. If you are using Internet Explorer, please switch to a different one.</p></div><p><a href="https://runkit.com/arnaudbuchholz/5c3f75a55e78ac001246366b" target="_blank">See it on runkit</a></p><p>Two <strong>colliding ways of creating classes</strong> producing sparks... Let see what can be <strong>learned</strong> from that.</p><h2>Building JavaScript classes</h2><h3>The 'old' way</h3><p>There are many ways to build a class and leverage <strong><a href="https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Objects/Inheritance" target="_blank">prototype inheritance</a></strong> in JavaScript. Here is <strong>the pattern used in GPF-JS</strong>.</p><p>First, a class is represented by its <strong>constructor function</strong>.</p><p>Every <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions" target="_blank">function</a> object exposes a <strong>prototype property</strong>, an object which members are <strong>inherited by all instances</strong> created with this function.</p><p>Therefore, the class <strong>members are added to the function prototype</strong>.</p></div><div class="code markdown"></div><script async="" src="//jsfiddle.net/ArnaudBuchholz/8bhgLqt6/7/embed/js,result/"></script><div class="code markdown"><p>To create a subclass, <strong>a new constructor function is needed</strong>.</p><p>The base constructor is called by <strong><a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Function/apply" target="_blank">applying</a> the base function</strong> on the newly created instance.</p><div class="note"><p> As mentioned previously, The GPF-JS library facilitates calling the base class constructor using <a href="[$super](https://gpf-js.blogspot.com/2017/02/my-own-super-implementation.html" target="_blank">this.$super()</a>.</p></div><p>On the other hand, the new function prototype is initialized with an <strong>object that inherits from the base class prototype</strong> using <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/create" target="_blank">Object.create</a>.</p><p>Consequently, all members of the base class will be inherited by the subclass. Also, instances will be recognized by the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/instanceof" target="_blank">instanceof</a> operator.</p></div><div class="code markdown"></div><script async="" src="//jsfiddle.net/ArnaudBuchholz/8bhgLqt6/embed/js,result/"></script><div class="code markdown"><h3>The ES6 way</h3><p>The <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/class" target="_blank">class keyword</a> was introduced and the syntax is <strong>self-explanatory</strong>.</p></div><div class="code markdown"></div><script async="" src="//jsfiddle.net/ArnaudBuchholz/b81aesr5/embed/js,result"></script><div class="code markdown"><p>It is interesting to observe that <strong>Es6A and Es6B are also functions</strong>.</p></div><div class="code markdown"></div><script async="" src="//jsfiddle.net/ArnaudBuchholz/dtrayepf/embed/js,result"></script><div class="code markdown"><h3>Subclassing an 'old' class in an ES6 class</h3><p>Good news, the following code <strong>works smoothly</strong>.</p></div><div class="code markdown"></div><script async="" src="//jsfiddle.net/ArnaudBuchholz/hqb490wL/embed/js,result"></script><div class="code markdown"><h3>Subclassing an ES6 class in an 'old' class</h3><p>The code presented in the introduction almost <strong>looks like the following example</strong>. It obviously leads to the <strong>same error</strong>.</p></div><div class="code markdown"></div><script async="" src="//jsfiddle.net/ArnaudBuchholz/1kbozc3w/embed/js,result"></script><div class="code markdown"><h2>Class constructor cannot be invoked without 'new'</h2><p>If you think about the old way of creating classes, this error fully makes sense. Indeed, in the old way, there is <strong>no syntax difference between a normal function and a class constructor</strong>. As a result, both could be either invoked or called with the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/new" target="_blank">new keyword</a>.</p><p>However, all JavaScript functions are not <strong>constructors</strong>. Indeed, most <strong>native methods</strong> are secured:</p></div><div class="code markdown"></div><script async="" src="//jsfiddle.net/ArnaudBuchholz/a96gfdm1/embed/js,result"></script><div class="code markdown"><p>When it comes to the new syntax, <strong>the intention of the developer</strong> is to build a function that will be used to create instances. Therefore, <strong>the language doesn't expect this function to be invoked for a simple function call</strong>.</p><h3>Reproduce the behavior in the 'old' way</h3><p>It is possible to reproduce this behavior with the 'old' syntax. The <em>new</em> operator will instantiate an object and pass it during the <strong>function invocation</strong>. This means that <strong>testing <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/this" target="_blank">this</a> with <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/instanceof" target="_blank">instanceof</a> will do the trick</strong>.</p></div><div class="code markdown"></div><script async="" src="//jsfiddle.net/ArnaudBuchholz/tauek0pL/embed/js,result/"></script><div class="code markdown"><p>Note that, in this implementation pattern, the base constructor call works because <strong>any instance of OldB is also an instance of OldA</strong>.</p><div class="note"><p> This behavior is already <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.8/src/define/class/constructor.js#L59" target="_blank">implemented</a> in GPF-JS.</p></div><div class="note"><p> A simpler alternative consists in using <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/new.target" target="_blank">new.target</a> but it was defined in ECMAScript 2015.</p></div><div class="note"><p> Again, there were many ways to create classes before the introduction of the class keyword. Some may disagree with the use of instanceof. To be fair and complete, I invite you to check the article <a href="https://medium.com/javascript-scene/javascript-factory-functions-vs-constructor-functions-vs-classes-2f22ceddf33e" target="_blank">JavaScript Factory Functions vs
Constructor Functions vs Classes</a> from Eric Elliott who exposes a different point of view.</p></div><h3>Notable differences between the two ways of creating class</h3><p>As explained previously in the 'Old' way of creating classes, the base constructor is called by applying the base function on <em>this</em>. However, there is <strong>no limitation on when the base constructor can be applied</strong>. Furthermore, it is <strong>not even required to call it</strong>.</p><p>As a matter of fact, you can start leveraging the newly created instance even <strong>before it was properly built</strong>.</p></div><div class="code markdown"></div><script async="" src="//jsfiddle.net/ArnaudBuchholz/t8ef1p5y/embed/js,result/"></script><div class="code markdown"><p>It has been enforced with ES6 as <strong>it is not possible to use <em>this</em> before calling the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/super" target="_blank">super</a> constructor</strong>.</p></div><div class="code markdown"></div><script async="" src="//jsfiddle.net/ArnaudBuchholz/wxa30178/embed/js,result"></script><div class="code markdown"><p>The result shows: <code class="javascript gpf-blog"><span class="identifier">Uncaught</span><span class="space"> </span><span class="identifier">ReferenceError</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">Must</span><span class="space"> </span><span class="identifier">call</span><span class="space"> </span><span class="identifier">super</span><span class="space"> </span><span class="identifier">constructor</span><span class="space"> </span><span class="keyword">in</span><span class="space"> </span><span class="identifier">derived</span><span class="space"> </span><span class="identifier">class</span><span class="space"> </span><span class="identifier">before</span><span class="space"> </span><span class="identifier">accessing</span><span class="space"> </span><span class="string">'this'</span><span class="space"> </span><span class="identifier">or</span><span class="space"> </span><span class="identifier">returning</span><span class="space"> </span><span class="identifier">from</span><span class="space"> </span><span class="identifier">derived</span><span class="space"> </span><span class="identifier">constructor</span></code></p><h3>Detecting an ES6 class constructor</h3><p>If the library has to deal with ES6 classes, it needs <strong>a safe way to detect such constructors</strong>.</p><p>Actually, this can easily be done by <strong>converting the function to string and by checking if it starts with the <em>class</em>
keyword</strong>.</p></div><div class="code markdown"></div><script async="" src="//jsfiddle.net/ArnaudBuchholz/prg4L5mk/embed/js,result"></script><div class="code markdown"><h2>Subclassing an ES6 class in an 'old' class</h2><p>The 'old' pattern does not work because the <strong>ES6 constructor can't be applied like a normal function</strong>. What can be done to create an 'old' class that would subclass an ES6 one?</p><h3>ConstructorOfB</h3><p>First, let set the right <strong>context and expectations</strong>:</p><ul><li><em>gpf.define</em> is used with a dictionary having the <em>$extend</em> property set to an ES6 class constructor</li><li>a <em>constructor</em> property points to a JavaScript function</li><li><em>this.$super</em> is called to invoke the base constructor</li><li>to respect the ES6 constraints, <em>this.$super</em> must be called before any use of this or the construction should fail</li></ul><strong>To validate the implementation</strong>, we will place it within the following statements:</div><div class="code markdown"></div><script async="" src="//jsfiddle.net/ArnaudBuchholz/zg70m9of/embed/js,result"></script><div class="code markdown"><h3>Attempt number 1: The copycat</h3><p>The very first attempt exploits a trick that I mastered while developing the library.</p><p>It is possible to <strong>create JavaScript functions dynamically</strong> using the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Function" target="_blank">Function</a> constructor.</p><p>This is <strong>detected as <a href="https://eslint.org/docs/rules/no-new-func" target="_blank">an issue</a> by most linters</strong> because it looks like an <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/eval" target="_blank">eval</a>. However, it is more secure since the created function has a <strong>restricted access to the current scope</strong>.</p><p>As explained in MDN: <em>Functions created with the Function constructor do not create closures to their creation contexts; they always are created in the global scope. When running them, they will only be able to access their own local variables and global ones, not the ones from the scope in which the Function constructor was created. This is different from using eval with code for a function expression.</em>
This reduces the risk of conflicts with existing names. But, in consequence, you need to <strong>pass the values you want to access in the created function</strong>.</p><p>To summarize, the first attempt consists in creating a <strong>class factory</strong> function where:</p><ul><li>the constructor is a copy of <em>constructorOfC</em> with <em>this.$super</em> being replaced with <em>super</em>
</li><li>the base class will be passed as a parameter</li></ul></div><div class="code markdown"></div><script async="" src="//jsfiddle.net/ArnaudBuchholz/zg70m9of/13/embed/js,result"></script><div class="code markdown"><div class="note"><p> In the previous code, an ES6 template literal has been used to build the constructor string. This syntax would not be allowed in the library because it would not compile on older hosts.</p></div><strong>It works !</strong><p>But...</p><p>As explained previously, the class constructor that is created from the <em>constructorOfC</em> function code <strong>does not keep the context of its source</strong>. If <em>constructorOfC</em> is a <strong>closure accessing names from its local scope</strong>, everything is lost.</p><p>This would probably work for some situations but, to keep its context, <strong>we must execute <em>constructorOfC</em> as-is</strong>.</p><h3>Attempt number 2: Applying constructorOfC on this</h3><p>Considering we must keep and call <em>constructorOfC</em> as-is, <strong>what happens if we can call it directly ?</strong>.</p></div><div class="code markdown"></div><script async="" src="//jsfiddle.net/ArnaudBuchholz/zg70m9of/16/embed/js,result"></script><div class="code markdown"><p>But <em>this</em> is used <strong>before</strong> <em>super</em> is called. As a consequence, it fails with: <code class="javascript gpf-blog"><span class="identifier">Uncaught</span><span class="space"> </span><span class="identifier">ReferenceError</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">Must</span><span class="space"> </span><span class="identifier">call</span><span class="space"> </span><span class="identifier">super</span><span class="space"> </span><span class="identifier">constructor</span><span class="space"> </span><span class="keyword">in</span><span class="space"> </span><span class="identifier">derived</span><span class="space"> </span><span class="identifier">class</span><span class="space"> </span><span class="identifier">before</span><span class="space"> </span><span class="identifier">accessing</span><span class="space"> </span><span class="string">'this'</span><span class="space"> </span><span class="identifier">or</span><span class="space"> </span><span class="identifier">returning</span><span class="space"> </span><span class="identifier">from</span><span class="space"> </span><span class="identifier">derived</span><span class="space"> </span><span class="identifier">constructor</span></code></p><h2>Attempt number 3: Building this</h2><p>Let consider the assumption that the <strong>base class supports instantiation with no parameters</strong>.</p><p>Because we must call <em>super</em> before using <em>this</em>, we start the constructor with a call to <em>super()</em> to initialize <em>this</em>. Then we apply <em>constructorOfC</em> on <em>this</em> and we make sure that<strong> <em>this.$super</em> would call the constructor</strong> again (through <em>super.constructor</em>).</p><p>What will happen?</p></div><div class="code markdown"></div><script async="" src="//jsfiddle.net/ArnaudBuchholz/zg70m9of/19/embed/js,result"></script><div class="code markdown"><p>It throws the error: <code class="javascript gpf-blog"><span class="identifier">Uncaught</span><span class="space"> </span><span class="identifier">TypeError</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">Class</span><span class="space"> </span><span class="identifier">constructor</span><span class="space"> </span><span class="identifier">B</span><span class="space"> </span><span class="identifier">cannot</span><span class="space"> </span><span class="identifier">be</span><span class="space"> </span><span class="identifier">invoked</span><span class="space"> </span><span class="identifier">without</span><span class="space"> </span><span class="string">'new'</span></code></p><h2>Attempt number 4: An unusual way to invoke a constructor</h2><p>So far, trying to <strong>apply the base constructor on the newly created instance</strong> appears to be a dead end. Indeed, the only way to call the base constructor is to use the <em>new</em> keyword.</p><p>But doing so would create a new instance of the base class: it won't <strong>be chained to the right prototype</strong>. This prevents subclassing.</p><div class="note"><p> One ugly hack would consist in switching the base constructor function's prototype value before calling <em>new</em> and restore it after.</p></div><p>Is it really the only way?</p><p>After doing some <a href="//lmgtfy.com/?q=JavaScript+How+to+call+an+ES6+constructor" target="_blank">research on the web</a>, this particular <a href="https://stackoverflow.com/questions/30689817/es6-call-class-constructor-without-new-keyword" target="_blank">stackoverflow thread</a> gave the answer.</p><p>A comment from 2015 says:</p><p>"<em>For subclassing Foo (...) with class Bar extends Foo ..., you should use return <strong>Reflect.construct(_Foo, args, new.target)</strong> instead (...). Subclassing in ES5 style (with Foo.call(this, ...)) is <strong>not possible</strong>.</em>"</p><p>I got my first <em>Aha</em> moment ! And I immediately checked the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Reflect" target="_blank">Reflect object
documentation</a>.</p><p>In particular, the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Reflect/construct" target="_blank">Reflect.construct
method</a> acts like the new operator, but as a function. <strong>It is equivalent to calling new target(...args)</strong>. It also gives the added option to <strong>specify a different prototype</strong>.</p><p>This helper not only <strong>solves the problem of calling the base class constructor</strong>, but it also allocates a <strong>new object with the right prototype chain</strong>.</p><p>However, the method <strong>returns a new instance: it can't be applied on an existing one</strong>.</p><p>Considering the construction will happen while executing the function <em>constuctorOfC</em>, we need to provide an initial value for <em>this</em> that supports the <em>$super</em> method. Then we need to 'substitute' it <strong>after</strong> the final instance was allocated.</p><p>Then came second <em>Aha</em> moment !</p><p>Why not creating a <strong>wrapper</strong> that would expose the <strong>same interface than a new instance of C</strong> but would redirect all properties access to the instance created with <em>Reflect.construct</em> using <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/defineProperties" target="_blank">Object.defineProperties</a> ?</p><p>An additional method of the wrapper, named <em>$super</em>, would call <strong>Reflect.construct</strong> to create the final instance.</p><p>As the <strong>final instance is not created unless this.$super has been called</strong>, any call forward would fail. It validates that <em>super</em> must be called first.</p><p>One last problem remains.</p><p>The whole purpose is to create a subclass which constructor will be invoked with <em>new</em>
and that <strong>must return an instance of the proper class</strong>. By default, when <em>new</em> is invoked with a constructor, the JavaScript engine is responsible of allocating the new instance and it invokes the constructor with it. The result of the new expression is this initially allocated instance.</p><p>In our case, <em>Reflect.construct</em> will create another instance that <strong>must be the result of the new expression</strong>.</p><p>Luckily, JavaScript supports replacing the allocated object by another one <strong>using the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/return" target="_blank">return statement</a> at the end of the constructor code</strong>.</p><p>This behavior is described in the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/new" target="_blank">new operator documentation</a>: <em>The object (not null, false, 3.1415 or other primitive types) returned by the constructor function becomes the result of the whole new expression.</em>
</p><div class="note"><p> This is how <a href="https://en.wikipedia.org/wiki/Singleton_pattern" target="_blank">singletons</a> are implemented in GPF-JS.</p></div><p>To summarize:</p><ul><li>Reflect.construct builds an instance of C initialized with constructor of B</li><li>The initial value of <em>this</em> is ignored, a wrapper is used to invoke <em>constructorOfC</em>
</li><li>The final instance is returned at the end of the class constructor</li></ul></div><div class="code markdown"></div><script async="" src="//jsfiddle.net/ArnaudBuchholz/zg70m9of/27/embed/js,result"></script><div class="code markdown"><strong>It works !</strong><p>But...</p><p>The problem is that <strong>all the accessed properties must be redirected from the wrapper to the instance</strong>. In this example, it is easy because the content of the object and its constructors is <strong>already known</strong>.</p><p>In the library, it won't be true.</p><h2>Attempt number 5: Shadowing the object</h2><p>Digging further in the same <a href="https://stackoverflow.com/questions/30689817/es6-call-class-constructor-without-new-keyword" target="_blank">stackoverflow
thread</a>, some proposed the use of <strong>proxies</strong> but for a different purpose.</p><p>Checking the documentation again, I realized that the <strong><a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Proxy" target="_blank">Proxy
object</a></strong> can be used to define <strong>custom behavior for fundamental operations</strong>... like property lookup.</p><p>This last piece of the information leaded to the <strong>final solution below</strong>.</p></div><div class="code markdown"></div><script async="" src="//jsfiddle.net/ArnaudBuchholz/zg70m9of/32/embed/js,result"></script><div class="code markdown"><strong>It works !</strong><p>I didn't find any drawback yet <em>(but it is still under study)</em>.</p><h2>Conclusion</h2><p>As useless as it may sound, <strong>the library will now support ES6 classes</strong>. But besides this feature, this small experience (or should I say challenge) <strong>introduced me to new JavaScript objects</strong> which, at first sight, had no interest but that are really helpful to solve the problems I faced.</p><p>Regarding the support of the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Reflect" target="_blank">Reflect</a> <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Proxy" target="_blank">Proxy</a> objects, they will be used only when an ES6 class is detected.</p></div>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-50114403652076537922018-12-07T01:11:00.002+01:002018-12-07T20:09:56.844+01:00Release 0.2.8: Serialization attributes<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/release.png" align="left">
This release took longer as it was developed in parallel with several side projects.
It includes new asynchronous helpers, a brand new mechanism to serialize classes
and new classes designed to validate attributes usage.
</div>
<div class="code markdown"><h2>Release 0.2.8: Serialization attributes</h2><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/16?closed=1" target="_blank">Stories and bugs</a> implemented</li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/tree/v0.2.8" target="_blank">Sources</a></li><li><a href="https://arnaudbuchholz.github.io/gpf/doc/index.html" target="_blank">Documentation</a></li><li><a href="https://www.npmjs.com/package/gpf-js" target="_blank">NPM package</a></li></ul><h2>Release content</h2><h3>A longer release</h3><p>As explained in the <a href="http://gpf-js.blogspot.com/2018/08/release-027-quality-and-xml.html" target="_blank">last release notes</a>, I am concentrating on a <a href="https://github.com/ArnaudBuchholz/bubu-cms" target="_blank">side project</a> and the library evolved to <strong>support its development</strong>.</p><p>In the meantime, other projects (<a href="https://github.com/ArnaudBuchholz/mockserver-server" target="_blank">mockserver-server</a> and <a href="https://www.npmjs.com/package/node-ui5" target="_blank">node-ui5</a>) were started since <strong>interesting challenges</strong> were submitted over the last month. Not to mention that <strong>more documentation</strong> was requested on the linting rules but also on the evolution of the library statistics.</p><p>As a consequence, this release took <strong>more time than usual</strong> (around 4 months).</p><h3>Asynchronous helpers</h3><h4>Interface wrappers</h4><p>When the XML serialization was introduced, a <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/272" target="_blank">generic wrapper</a> was <strong>required to simplify</strong> the use of the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.interfaces.IXmlContentHandler.html" target="_blank">IXmlContentHandler interface</a>.</p><p>The new function <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.interfaces.html#.promisify__anchor" target="_blank">gpf.interfaces.promisify</a> builds a <strong>factory method</strong> that takes an object implementing the given interface. This method returns a wrapper exposing the interface methods but returning <strong>chainable <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise" target="_blank">promises</a></strong>.</p><p>To put it in a nutshell, it converts this code:</p><p><code class="javascript gpf-blog"><span class="identifier">const</span><span class="space">
</span><span class="identifier">writer</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">xml</span><span class="symbol">.</span><span class="identifier">Writer</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">output</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">WritableString</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">pipe</span><span class="symbol">(</span><span class="identifier">writer</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">output</span><span class="symbol">)</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">console</span><span class="symbol">.</span><span class="identifier">log</span><span class="symbol">(</span><span class="identifier">output</span><span class="symbol">.</span><span class="identifier">toString</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">writer</span><span class="symbol">.</span><span class="identifier">startDocument</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="identifier">writer</span><span class="symbol">.</span><span class="identifier">startElement</span><span class="symbol">(</span><span class="string">"document"</span><span class="symbol">)</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="identifier">writer</span><span class="symbol">.</span><span class="identifier">startElement</span><span class="symbol">(</span><span class="string">"a"</span><span class="symbol">)</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="identifier">writer</span><span class="symbol">.</span><span class="identifier">startElement</span><span class="symbol">(</span><span class="string">"b"</span><span class="symbol">)</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="identifier">writer</span><span class="symbol">.</span><span class="identifier">endElement</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="identifier">writer</span><span class="symbol">.</span><span class="identifier">endElement</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="identifier">writer</span><span class="symbol">.</span><span class="identifier">startElement</span><span class="symbol">(</span><span class="string">"c"</span><span class="symbol">)</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="identifier">writer</span><span class="symbol">.</span><span class="identifier">endElement</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="identifier">writer</span><span class="symbol">.</span><span class="identifier">endElement</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="identifier">writer</span><span class="symbol">.</span><span class="identifier">endDocument</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">)</span><span class="symbol">;</span></code></p><p>into this code:</p><p><code class="javascript gpf-blog"><span class="identifier">const</span><span class="space">
</span><span class="identifier">writer</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">xml</span><span class="symbol">.</span><span class="identifier">Writer</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">output</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">WritableString</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">IXmlContentHandler</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">interfaces</span><span class="symbol">.</span><span class="identifier">IXmlContentHandler</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">xmlContentHandler</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">interfaces</span><span class="symbol">.</span><span class="identifier">promisify</span><span class="symbol">(</span><span class="identifier">IXmlContentHandler</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">promisifiedWriter</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">xmlContentHandler</span><span class="symbol">(</span><span class="identifier">writer</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">pipe</span><span class="symbol">(</span><span class="identifier">writer</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">output</span><span class="symbol">)</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">console</span><span class="symbol">.</span><span class="identifier">log</span><span class="symbol">(</span><span class="identifier">output</span><span class="symbol">.</span><span class="identifier">toString</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">promisifiedWriter</span><span class="symbol">.</span><span class="identifier">startDocument</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">startElement</span><span class="symbol">(</span><span class="string">"document"</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">startElement</span><span class="symbol">(</span><span class="string">"a"</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">startElement</span><span class="symbol">(</span><span class="string">"b"</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">endElement</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">endElement</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">startElement</span><span class="symbol">(</span><span class="string">"c"</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">endElement</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">endElement</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">endDocument</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span></code></p><p>When using this wrapper, it quickly appeared that <strong>something was missing</strong>. It sometimes happens that the chain is broken by a <strong>normal promise</strong>. The wrapper was modified to deal with it.</p><p><code class="javascript gpf-blog"><span class="comment">/*...*/</span><span class="space">
</span><span class="identifier">promisifiedWriter</span><span class="symbol">.</span><span class="identifier">startDocument</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">startElement</span><span class="symbol">(</span><span class="string">"document"</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">startElement</span><span class="symbol">(</span><span class="string">"a"</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">startElement</span><span class="symbol">(</span><span class="string">"b"</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="identifier">anyMethodReturningAPromise</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">endElement</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">endElement</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">startElement</span><span class="symbol">(</span><span class="string">"c"</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">endElement</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">endElement</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">endDocument</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span></code></p><p>The best example of use is <a href="https://github.com/ArnaudBuchholz/bubu-cms/blob/77240f29513b359cd22ba15f46adfa5ad67be1b9/api/metadata.js" target="_blank">$metadata implementation</a> of the side project.</p><h4>gpf.forEachAsync</h4><p>There are many <a href="https://stackoverflow.com/questions/40328932/javascript-es6-promise-for-loop" target="_blank">solutions</a> to handle <strong>loops with promises</strong>.</p><p>Since the library offers <strong>iteration helpers</strong> (<a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.html#.forEach__anchor" target="_blank">gpf.forEach</a>), it made sense to provide the equivalent for asynchronous callback: <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.html#.forEachAsync__anchor" target="_blank">gpf.forEachAsync</a>. It obviously returns a promise <strong>resolved when the loop is over</strong>.</p><h3>$singleton</h3><p>Among the <a href="https://en.wikipedia.org/wiki/Software_design_pattern" target="_blank">design patterns</a>, the <strong><a href="https://en.wikipedia.org/wiki/Singleton_pattern" target="_blank">singleton</a></strong> is probably the most easy to describe and implement.</p><p>Here again, there are many ways to implement a singleton in JavaScript.</p><p>In the library, an entity definition may include the <strong><a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-DEFINE.html" target="_blank">$singleton</a> property</strong>. When used, any attempt to create a new instance of the entity will return the same instance.</p><p>The singleton is allocated the <strong>first time it is instantiated</strong>.</p><p>For instance: <code class="javascript gpf-blog"><span class="keyword">var</span><span class="space"> </span><span class="identifier">counter</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="number">0</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">Singleton</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">define</span><span class="symbol">(</span><span class="symbol">{</span><span class="space">
</span><span class="identifier">$class</span><span class="symbol">:</span><span class="space"> </span><span class="string">"mySingleton"</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">$singleton</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">true</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">constructor</span><span class="symbol">:</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">this</span><span class="symbol">.</span><span class="identifier">value</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="symbol">++</span><span class="identifier">counter</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">instance1</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">Singleton</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">instance2</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">Singleton</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">assert</span><span class="symbol">(</span><span class="identifier">instance1</span><span class="symbol">.</span><span class="identifier">value</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="number">1</span><span class="symbol">)</span><span class="symbol">;</span><span class="space"> </span><span class="comment">// true</span><span class="space">
</span><span class="identifier">assert</span><span class="symbol">(</span><span class="identifier">instance2</span><span class="symbol">.</span><span class="identifier">value</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="number">1</span><span class="symbol">)</span><span class="symbol">;</span><span class="space"> </span><span class="comment">// true</span><span class="space">
</span><span class="identifier">assert</span><span class="symbol">(</span><span class="identifier">instance1</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="identifier">instance2</span><span class="symbol">)</span><span class="symbol">;</span><span class="space"> </span><span class="comment">// true</span></code></p><h3>Serialization and validation attributes</h3><p>A good way to describe these features is to start with the <strong>use case</strong>. As explained before, this release was made to support the development of a side project. Simply put, it consists in a <strong>JavaScript full stack</strong> application composed of:</p><ul><li>An <a href="https://openui5.org/" target="_blank">OpenUI5</a> interface</li><li>A NodeJS server exposing an <a href="https://www.odata.org/" target="_blank">ODATA</a> service</li></ul><div class="note"><p> There are many <a href="https://raygun.com/blog/popular-javascript-frameworks/" target="_blank">UI frameworks</a> out there. I decided to go with OpenUI5 for two reasons: the user interface is fairly simple and I want it to be responsive and look professional. Furthermore, it comes with <a href="https://youtu.be/HiZq-kuIbt0" target="_blank">OPA</a> that will allow - in this particular case - <strong>end 2 end</strong> test automation.</p></div><p>Since I am a <a href="https://www.linkedin.com/pulse/lazy-arnaud-buchholz/" target="_blank">lazy developer</a> building a backend on top of <a href="https://expressjs.com/" target="_blank">express</a>, <strong>flexibility</strong> is mandatory so that adding a new entity / property <strong>does not imply changes all across the project</strong>.</p><p>Indeed, a new property means that:</p><ul><li>The <strong><a href="https://www.odata.org/documentation/odata-version-2-0/overview/#MetadataForODataServices" target="_blank">schema</a></strong> must be updated so that the UI is aware of it</li><li><strong>Serialization</strong> <em>(reading from / writing to client)</em> must be adapted to handle the new property</li><li>Depending on the property type, the value might be <strong>converted</strong> <em>(in particular for date/time)</em>
</li><li>It may <em>(or may not)</em> support <strong>filtering / sorting</strong></li><li>...</li></ul><h4>gpf.attributes.Serializable</h4><p>In this project, the main entity is a <a href="https://github.com/ArnaudBuchholz/bubu-cms/blob/master/api/Record.js" target="_blank">Record</a>.</p><p>Since a class is defined to handle the instances, it makes sense to rely on its definition to determine what is exposed. However, we might need a bit of control on <strong>which members are exposed and how</strong>.</p><p>This is a <strong>perfect use case for attributes</strong>.</p><p>The <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.attributes.Serializable.html" target="_blank">gpf.attributes.Serializable</a> attribute <strong>describes the name and type</strong> as well as indicates if the property is required.</p><p>For instance, the <a href="https://github.com/ArnaudBuchholz/bubu-cms/blob/master/api/Record.js#L30" target="_blank">_name property</a> is exposed as the string field named "name".</p><div class="note"><p> The required part is not yet leveraged but it will be used to validate the entities.</p></div><p>This definition is <strong>documented</strong> in the structure <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.typedef.html#.serializableProperty" target="_blank">gpf.typedef.serializableProperty</a>.</p><p>Today, only three <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.serial.html#.types__anchor" target="_blank">types</a> are supported:</p><ul><li>string</li><li>integer</li><li>date/time</li></ul><h4>gpf.serial</h4><p>Once the members are flagged with the Serializable attribute, some helpers were created to <strong>utilize this information</strong>.</p><p><a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.serial.html#.get__anchor" target="_blank">gpf.serial.get</a> returns a dictionary indexing the Serializable attributes per the class member name.</p><p>Also, two methods <strong>convert/read the instance into/from a simpler object</strong> containing only the serializable properties:</p><ul><li><a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.serial.html#.fromRaw__anchor" target="_blank">gpf.serial.fromRaw</a></li><li><a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.serial.html#.toRaw__anchor" target="_blank">gpf.serial.toRaw</a></li></ul><p>These methods include a converter callback <strong>to enable value conversion</strong>.</p><p>For instance: <code class="javascript gpf-blog"><span class="keyword">var</span><span class="space"> </span><span class="identifier">raw</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">serial</span><span class="symbol">.</span><span class="identifier">toRaw</span><span class="symbol">(</span><span class="identifier">entity</span><span class="symbol">,</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">value</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">property</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">if</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">serial</span><span class="symbol">.</span><span class="identifier">types</span><span class="symbol">.</span><span class="identifier">datetime</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="identifier">property</span><span class="symbol">.</span><span class="identifier">type</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">if</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">value</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="string">'/Date('</span><span class="space"> </span><span class="symbol">+</span><span class="space"> </span><span class="identifier">value</span><span class="symbol">.</span><span class="identifier">getTime</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">+</span><span class="space"> </span><span class="string">')/'</span><span class="space">
</span><span class="symbol">}</span><span class="space"> </span><span class="keyword">else</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">null</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="keyword">if</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">property</span><span class="symbol">.</span><span class="identifier">name</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="string">'tags'</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">value</span><span class="symbol">.</span><span class="identifier">join</span><span class="symbol">(</span><span class="string">' '</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">value</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span></code></p><h4>attributes restrictions</h4><p>If you read carefully the documentation of the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.attributes.Serializable.html" target="_blank">gpf.attributes.Serializable</a> attribute, you may notice the section named <strong>Usage restriction</strong>.</p><p>It mentions:</p><ul><li>Only for member level (see <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.attributes.MemberAttribute.html" target="_blank">gpf.attributes.MemberAttribute</a>)</li><li>Only once (see <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.attributes.UniqueAttribute.html" target="_blank">gpf.attributes.UniqueAttribute</a>)</li></ul><p>If you check the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.8/src/attributes/serializable.js#L18" target="_blank">code</a>:</p><p><code class="javascript gpf-blog"><span class="keyword">var</span><span class="space"> </span><span class="identifier">_gpfAttributesSerializable</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">_gpfDefine</span><span class="symbol">(</span><span class="symbol">{</span><span class="space">
</span><span class="identifier">$class</span><span class="symbol">:</span><span class="space"> </span><span class="string">"gpf.attributes.Serializable"</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">$extend</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">_gpfAttribute</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">$attributes</span><span class="symbol">:</span><span class="space"> </span><span class="symbol">[</span><span class="space">
</span><span class="keyword">new</span><span class="space"> </span><span class="identifier">_gpfAttributesMemberAttribute</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="keyword">new</span><span class="space"> </span><span class="identifier">_gpfAttributesUniqueAttribute</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">]</span><span class="symbol">,</span><span class="space">
</span><span class="comment">/* ... */</span></code></p><p>This means that the Serializable attribute can be used only on class members and only once <em>(per class member)</em>.</p><p>This also means that <strong>new attribute classes were designed to secure the use of attributes</strong>. This will facilitate the <strong>adoption</strong> of the mechanism since any <strong>misuse of an attribute will generate an error</strong>. It is a better approach than having no effect and not letting the developer know.</p><p>The validation attributes are:</p><ul><li><a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.attributes.AttributeForInstanceOf.html" target="_blank">gpf.attributes.AttributeForInstanceOf</a> which checks that the attribute is applied to the right class.</li><li><a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.attributes.ClassAttribute.html" target="_blank">gpf.attributes.ClassAttribute</a> to restrict the use of an attribute to the class level (i.e. in the $attributes entity property)</li><li><a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.attributes.MemberAttribute.html" target="_blank">gpf.attributes.MemberAttribute</a> to restrict the use of an attribute to a member</li><li><a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.attributes.UniqueAttribute.html" target="_blank">gpf.attributes.UniqueAttribute</a> to enforce the uniqueness of the attribute (including through inheritance)</li></ul><div class="note"><p> Actually, ClassAttribute, MemberAttribute and UniqueAttribute are singletons.</p></div><p>Obviously, <strong>these attributes are also validated</strong>, check their documentation and <a href="https://github.com/ArnaudBuchholz/gpf-js/tree/v0.2.8/src/attributes" target="_blank">implementation</a>.</p><h3>Project metrics reporting</h3><p>Two years ago, the <a href="http://gpf-js.blogspot.com/2016/12/release-015.html" target="_blank">release 0.1.5 named "The new core"</a> marked the beginning of <strong>a new development start for the library</strong>. There are few traces of what happened before as the project was <strong>not structured</strong>. Since then, the project metrics were <strong>systematically</strong> added to the <a href="https://github.com/ArnaudBuchholz/gpf-js#metrics" target="_blank">Readme</a>.</p><p>With <a href="https://gpf-js.blogspot.com/2017/12/release-023-streams-and-pipes.html" target="_blank">release 0.2.3</a>, all these metrics were <strong>consolidated into one single file</strong>: <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/build/releases.json" target="_blank">releases.json</a>. This file is <strong>automatically updated</strong> by the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/make/release.js" target="_blank">release script</a>.</p><p>Using <a href="https://gionkunz.github.io/chartist-js/" target="_blank">chartist.js</a>, the dashboard tiles were modified to render a chart showing the <strong>progression of the metrics over the releases</strong>.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/Release%200.2.8/sources.png" target="_blank">sources<br><img src="https://arnaudbuchholz.github.io/blog/post/Release%200.2.8/sources.png" alt="sources" title="sources" border="0" style="max-width: 100%"></a></p><p><a href="https://arnaudbuchholz.github.io/blog/post/Release%200.2.8/plato.png" target="_blank">plato<br><img src="https://arnaudbuchholz.github.io/blog/post/Release%200.2.8/plato.png" alt="plato" title="plato" border="0" style="max-width: 100%"></a></p><p><a href="https://arnaudbuchholz.github.io/blog/post/Release%200.2.8/coverage.png" target="_blank">coverage<br><img src="https://arnaudbuchholz.github.io/blog/post/Release%200.2.8/coverage.png" alt="coverage" title="coverage" border="0" style="max-width: 100%"></a></p><p><a href="https://arnaudbuchholz.github.io/blog/post/Release%200.2.8/tests.png" target="_blank">tests<br><img src="https://arnaudbuchholz.github.io/blog/post/Release%200.2.8/tests.png" alt="tests" title="tests" border="0" style="max-width: 100%"></a></p><h3>Documentation of ESLint rules</h3><h4>Automated documentation</h4><p><a href="https://en.wikipedia.org/wiki/Lint_%28software%29" target="_blank">Linting</a> is used to <strong>statically validate the source code</strong> since the beginning of the project. The set of <a href="https://eslint.org/" target="_blank">eslint</a> rules has been <strong>refined over the releases</strong> and <strong>critical settings</strong> framed the way the sources look like.</p><p>Furthermore, the linter also evolves with time (and feedback) and <strong>some rules become obsolete as new ones are introduced</strong>.</p><p>In the end, it is really challenging to <strong>stay up-to-date</strong> and provide <strong>clear and complete explanations</strong> on the different rules that are configured (and why they are configured this way).</p><p>These are the problems that were addressed with the <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/280" target="_blank">task #280</a>.</p><p>As a result, a <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/doc/linting.js" target="_blank">script</a> leverages eslint's <a href="https://eslint.org/docs/rules/" target="_blank">rules documentation</a> to <strong>extract and validate</strong> the library settings. When needed, some <a href="https://github.com/ArnaudBuchholz/gpf-js/tree/master/doc/linting" target="_blank">details</a> are provided.</p><p>The final result appears in the documentation in the <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-LINTING.html" target="_blank">Tutorials\Linting</a> menu</p><h4>no-magic-numbers</h4><p>While documenting the rules, the <a href="https://eslint.org/docs/rules/no-magic-numbers" target="_blank">no-magic-numbers</a> one stood out.</p><p>I wanted to understand how this rule would (could?) <strong>improve the code</strong>. It was enabled to see how many magic numbers existed. Realizing that this generates a <strong>huge amount of errors</strong>, the check was turned off for test filesto start with)<em>.</em></p><div class="note"><p> Some people like to distinguish warnings and errors. However warnings <strong>do not call for action</strong>. As a result, they tend to last forever leading to the <a href="https://en.wikipedia.org/wiki/Broken_windows_theory" target="_blank">broken window</a> effect. I prefer a binary approach meaning it is either OK or not OK.</p></div><p>It took almost one month of refactoring to remove them but, in the end, it did improve the code and lessons were learned.</p><div class="note"><p> This also demonstrated the value of having 100% of test coverage.</p></div><h2>Lessons learned</h2><h3>Library + application</h3><p>This may sound obvious but using the library as a support for an application gives <strong>immediate feedback</strong> on how the <strong>API is appropriate</strong>. It helps to keep the focus on how practical the methods are.</p><p>For instance, the helper <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.serial.html#.get__anchor" target="_blank">gpf.serial.get</a> was integrated in the library because its <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.8/src/serial/get.js#L21" target="_blank">10 little lines of code</a> were repeated in the application.</p><h3>Refactoring</h3><p>It is not the first time that the whole library requires refactoring. And I actually <strong>like the exercise</strong> because it gives the opportunity to <strong>come back on old code</strong> that hasn't been touched in a while. Since the project started several years ago, my <strong>knowledge and skill have evolved</strong> and it gives a new look on the sources. Furthermore, the code being fully tested, there are very <strong>little risks</strong>.</p><p>When dealing with <a href="https://eslint.org/docs/rules/no-magic-numbers" target="_blank">magic numbers</a>, I realized that some patterns were <strong>obsolete</strong> because of JavaScript methods I was not used to. As the library offers a <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-COMPATIBILITY.html" target="_blank">compatibility layer</a>, it has been enriched with these new methods and the code modified consequently.</p><p>For instance: <code class="javascript gpf-blog"><span class="keyword">if</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">string</span><span class="symbol">.</span><span class="identifier">indexOf</span><span class="symbol">(</span><span class="identifier">otherString</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="number">0</span><span class="symbol">)</span></code> is better replaced with: <code class="javascript gpf-blog"><span class="keyword">if</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">string</span><span class="symbol">.</span><span class="identifier">startsWith</span><span class="symbol">(</span><span class="identifier">otherString</span><span class="symbol">)</span><span class="symbol">)</span></code></p><p>The same way: <code class="javascript gpf-blog"><span class="keyword">if</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">string</span><span class="symbol">.</span><span class="identifier">indexOf</span><span class="symbol">(</span><span class="identifier">otherString</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">!==</span><span class="space"> </span><span class="symbol">-</span><span class="number">1</span><span class="symbol">)</span></code> should be using: <code class="javascript gpf-blog"><span class="keyword">if</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">string</span><span class="symbol">.</span><span class="identifier">includes</span><span class="symbol">(</span><span class="identifier">otherString</span><span class="symbol">)</span><span class="symbol">)</span></code></p><p>Last example, <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp" target="_blank">regular expressions</a> are widely used with <strong>capturing groups</strong>. Their value is available in the array-like result through indexes. <strong>Using constants rather than numbers to get these values improves the code readability</strong>.</p><h2>Next release</h2><p>The <a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/17" target="_blank">next release content</a> is not completely defined. There are plans to expand <strong>the use of attributes to ES6 classes</strong> and to integrate <a href="https://github.com/graalvm/graaljs" target="_blank">graaljs</a>.</p><p>For the rest, it will depend on the side project since <strong>it needs all my attention</strong>.</p></div>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-48512957881729573432018-08-07T18:46:00.001+02:002018-12-07T19:50:37.247+01:00Release 0.2.7: Quality and XML<br>
<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/release.png" align="left">
This small release focuses on quality by integrating hosted automated code review services and introduces XML
serialization.
</div>
<div class="code markdown"><h2>Release 0.2.7: Quality and XML</h2><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/15?closed=1" target="_blank">Stories and bugs</a> implemented</li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/tree/v0.2.7" target="_blank">Sources</a></li><li><a href="https://arnaudbuchholz.github.io/gpf/doc/index.html" target="_blank">Documentation</a></li><li><a href="https://www.npmjs.com/package/gpf-js" target="_blank">NPM package</a></li></ul><h2>Release content</h2><h3>A smaller release</h3><p>As announced during the release of <a href="http://gpf-js.blogspot.com/2018/05/release-026-gpfrequirejs.html" target="_blank">version 0.2.6</a>, the month of June was busy developing a <a href="https://github.com/ArnaudBuchholz/training-ui5con18-opa" target="_blank">sample application</a> to support the <a href="https://youtu.be/HiZq-kuIbt0" target="_blank">UICon'18 conference</a>.</p><p>Unexpectedly, another <strong>interesting project</strong> emerged from this development but this will be detailed later on the blog.</p><p>In the end, the <strong>bandwidth was limited</strong> to work on this release.</p><h3>XML Serialization</h3><p>This version introduces the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.interfaces.IXmlContentHandler.html" target="_blank">IXmlContentHandler</a> interface as well as the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.xml.Writer.html" target="_blank">gpf.xml.Writer</a> class to <strong>enable <a href="https://en.wikipedia.org/wiki/XML" target="_blank">XML</a> writing</strong>.</p><p>If you are not familiar with the <a href="https://en.wikipedia.org/wiki/Simple_API_for_XML" target="_blank">Simple API for XML</a>, there are tons of existing implementation in different languages. The <a href="https://docs.oracle.com/javase/tutorial/jaxp/sax/parsing.html" target="_blank">Java</a> one is considered to be <strong>normative</strong>.</p><p>To put it in a nutshell, <strong>SAX proposes an interface to parse and generate XML</strong>.</p><div class="note"><p> The parsing part might be implemented later, only the generation one is required today.</p></div><p>Here is an example of an XML generation piped to a string buffer:</p><p><code class="javascript gpf-blog"><span class="identifier">const</span><span class="space">
</span><span class="identifier">writer</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">xml</span><span class="symbol">.</span><span class="identifier">Writer</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">output</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">WritableString</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">pipe</span><span class="symbol">(</span><span class="identifier">writer</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">output</span><span class="symbol">)</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">console</span><span class="symbol">.</span><span class="identifier">log</span><span class="symbol">(</span><span class="identifier">output</span><span class="symbol">.</span><span class="identifier">toString</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">writer</span><span class="symbol">.</span><span class="identifier">startDocument</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="identifier">writer</span><span class="symbol">.</span><span class="identifier">startElement</span><span class="symbol">(</span><span class="string">"document"</span><span class="symbol">)</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="identifier">writer</span><span class="symbol">.</span><span class="identifier">startElement</span><span class="symbol">(</span><span class="string">"a"</span><span class="symbol">)</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="identifier">writer</span><span class="symbol">.</span><span class="identifier">startElement</span><span class="symbol">(</span><span class="string">"b"</span><span class="symbol">)</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="identifier">writer</span><span class="symbol">.</span><span class="identifier">endElement</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="identifier">writer</span><span class="symbol">.</span><span class="identifier">endElement</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="identifier">writer</span><span class="symbol">.</span><span class="identifier">startElement</span><span class="symbol">(</span><span class="string">"c"</span><span class="symbol">)</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="identifier">writer</span><span class="symbol">.</span><span class="identifier">endElement</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="identifier">writer</span><span class="symbol">.</span><span class="identifier">endElement</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="identifier">writer</span><span class="symbol">.</span><span class="identifier">endDocument</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">)</span><span class="symbol">;</span></code></p><p>Which leads to the following output:</p></div><pre><document><a><b/></a><c/></document></pre><div class="code markdown"><p>Representing the following structure:</p></div><pre>document
|
+- a
| |
| +- b
|
+- c</pre><div class="code markdown"><p>Since all the methods returns a promise, the syntax is quite <strong>tedious</strong>. When writing the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/f506c20d2ad16bffc33f188d94a1151aa5705f50/test/xml/writer.js" target="_blank">first tests</a>, it quickly became clear that its complexity could be greatly reduced by augmenting the result promise with the interface methods.</p><p>As a result, a <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/6478d94ebfe547c3df5a4e4c4ab83966591781f8/test/xml/writer.js#L6-L26" target="_blank">wrapper</a> was designed to <strong>simplify the tests</strong> leading to this improved syntax:</p><p><code class="javascript gpf-blog"><span class="identifier">const</span><span class="space">
</span><span class="identifier">writer</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">xml</span><span class="symbol">.</span><span class="identifier">Writer</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">output</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">WritableString</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">pipe</span><span class="symbol">(</span><span class="identifier">writer</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">output</span><span class="symbol">)</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="symbol">></span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">console</span><span class="symbol">.</span><span class="identifier">log</span><span class="symbol">(</span><span class="identifier">output</span><span class="symbol">.</span><span class="identifier">toString</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">wrap</span><span class="symbol">(</span><span class="identifier">writer</span><span class="symbol">)</span><span class="symbol">.</span><span class="identifier">startDocument</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">startElement</span><span class="symbol">(</span><span class="string">"document"</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">startElement</span><span class="symbol">(</span><span class="string">"a"</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">startElement</span><span class="symbol">(</span><span class="string">"b"</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">endElement</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">endElement</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">startElement</span><span class="symbol">(</span><span class="string">"c"</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">endElement</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">endElement</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">endDocument</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span></code></p><p>This will <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/272" target="_blank">surely</a> be standardized in a future version.</p><h3>Improved gpf.require</h3><h4>Preloading</h4><p>The goal of the library is to support application development. As explained in the article <a href="http://gpf-js.blogspot.com/2018/01/my-own-require-implementation.html" target="_blank">My own require implementation</a>, splitting the code into <strong>modules enforces better code</strong>. However, at some point, all these modules must be consolidated to <strong>speed up the application loading</strong>.</p><p>This version offers the possibility to <strong>preload</strong> the sources by passing a dictionary mapping the resources path to their textual content. As a result, when the resource is required, it does not need to be loaded.</p><p>Here is the proposed bootstrap implementation:</p><p><code class="javascript gpf-blog"><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">http</span><span class="symbol">.</span><span class="identifier">get</span><span class="symbol">(</span><span class="string">"preload.json"</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">response</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">if</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">response</span><span class="symbol">.</span><span class="identifier">status</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="number">200</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">JSON</span><span class="symbol">.</span><span class="identifier">parse</span><span class="symbol">(</span><span class="identifier">response</span><span class="symbol">.</span><span class="identifier">responseText</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">Promise</span><span class="symbol">.</span><span class="identifier">reject</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">preload</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">require</span><span class="symbol">.</span><span class="identifier">configure</span><span class="symbol">(</span><span class="symbol">{</span><span class="space">
</span><span class="identifier">preload</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">preload</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="keyword">catch</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">reason</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">// Document and/or absorb errors</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">require</span><span class="symbol">.</span><span class="identifier">define</span><span class="symbol">(</span><span class="symbol">{</span><span class="space">
</span><span class="identifier">app</span><span class="symbol">:</span><span class="space"> </span><span class="string">"app.js"</span><span class="space"> </span><span class="comment">// Might be preloaded</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">,</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">require</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">require</span><span class="symbol">.</span><span class="identifier">app</span><span class="symbol">.</span><span class="identifier">start</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span></code></p><h4>Modern browsers</h4><p>One of the challenges of building a feature-specific version of the library (a.k.a. flavor) is to test it with <strong>modern browsers</strong> only. The <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-COMPATIBILITY.html" target="_blank">compatibility layer</a> of the library takes a <strong>significant part</strong> of it and is useless if the flavor's target is NodeJS or any recent browser.</p><p>Worst, while building the release, the tests were failing when 'old' browsers were configured.</p><p>So, the concurrent task was <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.7/grunt/concurrent.js#L12-L15" target="_blank">modified</a> to include a <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.7/grunt/concurrent.js#L75" target="_blank">condition</a> on modern browsers.</p><p>These are considered modern:</p><ul><li>Chrome</li><li>Firefox</li><li>Safari (if on Mac)</li></ul><h3>Quality improvement</h3><h4>Abstract classes</h4><p>Quality is also about providing tools to make sure that developers don't make mistake. Abstract classes concept is one of them. This version offers the possibility to create <strong>abstract classes</strong> by adding <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.7/test/define.js#L313-L373" target="_blank">$abstract</a> in their definition.</p><div class="note"><p> If one wants to deal with abstract methods, they can be defined with gpf.Error.abstractMethod. However, this won't prevent class instantiation.</p></div><h4>Debugging with sources</h4><p>Debugging the library can be laborious. I am more familiar with Chrome development tools and I sometimes use them with <a href="https://nodejs.org/en/docs/guides/debugging-getting-started/" target="_blank">NodeJS</a>. Because the sources are loading through the <a href="http://linterrors.com/js/eval-is-evil" target="_blank">evil-ish</a> use of <a href="https://developer.mozilla.org/fr/docs/Web/JavaScript/Reference/Objets_globaux/eval" target="_blank">eval</a>, they <strong>don't appear in the debugger sources tab</strong>.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/Release%200.2.7/before.png" target="_blank">No sources<br><img src="https://arnaudbuchholz.github.io/blog/post/Release%200.2.7/before.png" alt="No sources" title="No sources" border="0" style="max-width: 100%"></a></p><p>To solve that problem, <a href="https://www.html5rocks.com/en/tutorials/developertools/sourcemaps/" target="_blank">source maps</a> were applied.</p><p>To put it in a nutshell:</p><ul><li>Test files loading <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.7/test/host/web_loader.js#L48" target="_blank">injects
sourceURL</a> before evaluation</li><li>A special version of the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.7/test/host/web_loader.js#L207" target="_blank">boot.js</a> file is handled by a <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.7/grunt/connect-middleware/sources.js" target="_blank">new
middleware</a>. It generates a unique source file (sources.js) with its corresponding source map thanks to <a href="https://github.com/mishoo/UglifyJS2" target="_blank">uglify-js</a>.</li></ul><p>As a result, sources appear:</p><p><a href="https://arnaudbuchholz.github.io/blog/post/Release%200.2.7/after.png" target="_blank">With sources<br><img src="https://arnaudbuchholz.github.io/blog/post/Release%200.2.7/after.png" alt="With sources" title="With sources" border="0" style="max-width: 100%"></a></p><h4>Hosted automated code review</h4><p><a href="https://github.com/" target="_blank">GitHub</a> is a huge source of information. While browsing some repositories, I discovered two code review services that integrates nicely.</p><ul><li><a href="https://codeclimate.com/github/ArnaudBuchholz/gpf-js" target="_blank">Code Climate</a></li><li><a href="https://app.codacy.com/project/ArnaudBuchholz/gpf-js/dashboard" target="_blank">Codacy</a></li></ul><p>They both focus on <strong>code quality</strong> (based on static checks) and propose exhaustive report on potential issues or code smells found in your code.</p><p>Today, <strong>only the <a href="https://github.com/ArnaudBuchholz/gpf-js/tree/master/src" target="_blank">src</a> folder of the repository is submitted for review</strong>.</p><p>It revealed some interesting issues such as:</p><ul><li>Code similarities, i.e. opportunity for code refactoring</li></ul><ul><li>Code complexities:</li></ul><p>Some were already known and have been addressed in this version (in particular <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.7/src/compatibility/promise.js" target="_blank">src/compatibility/promise.js</a> where plato was giving a little <a href="https://arnaudbuchholz.github.io/gpf/plato/files/src_compatibility_promise_js/index.html" target="_blank">74.46</a>).</p><p>The surprise came from a <strong>class definition with more than 20 methods</strong> as it was considered an issue (<a href="https://github.com/ArnaudBuchholz/gpf-js/blob/ebee2a06417401cbbaabd89f8fa65a43bb7ec943/src/xml/writer.js" target="_blank">src/xml/writer.js</a>). After having diligently improved the code, by <strong>isolating the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.7/src/xml/check.js" target="_blank">XML validation helpers</a></strong>, one must admit that it makes things more <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.7/src/xml/writer.js" target="_blank">readable</a> !</p><p>Finally, these tools <strong>rank the overall quality</strong> with a score that can be inserted in the project readme.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/Release%200.2.7/quality.png" target="_blank">Quality scores<br><img src="https://arnaudbuchholz.github.io/blog/post/Release%200.2.7/quality.png" alt="Quality scores" title="Quality scores" border="0" style="max-width: 100%"></a></p><h2>Lessons learned</h2><p>From a pure development prospective, <strong>a lot was done in a very limited time</strong>. Since the quality of the code is <strong>enforced</strong> by the usual best practices (TDD, static code validation) but also <strong>measured</strong> (with plato), modifications are safe and immediately validated.</p><p>A lot was learned on <a href="https://www.html5rocks.com/en/tutorials/developertools/sourcemaps/" target="_blank">JavaScript source mappings</a> since it was required to enable debugging in the browser.</p><p>The relevance of the problems raised by the Code Climate tool was quit surprising: the <strong>overall project quality benefited from this integration</strong>.</p><h2>Next release</h2><p>The <a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/16" target="_blank">next release content</a> is not even defined. For the next months, I will focus on a side project that requires all my attention.</p></div>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-78736646836469250292018-05-12T15:25:00.001+02:002018-12-07T19:50:26.268+01:00Release 0.2.6: gpf.require.js<br>
<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/release.png" align="left">
This release fixes sporadic Travis issues, improves the modularization helper and finalizes the flavor mechanism to
deliver the first reduced version of the library (gpf.require.js).
</div>
<div class="code markdown"><h2>New version</h2><p>Here comes the new version:</p><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/14?closed=1" target="_blank">Stories and bugs</a> implemented</li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/tree/v0.2.6" target="_blank">Sources</a></li><li><a href="https://arnaudbuchholz.github.io/gpf/doc/index.html" target="_blank">Documentation</a></li><li><a href="https://www.npmjs.com/package/gpf-js" target="_blank">NPM package</a></li></ul><h2>Release content</h2><h3>Sporadic Travis issues</h3><p>From time to time, the <strong>Travis continuous integration build was failing</strong> with a message indicating that the <strong>coverage information was missing for the browser</strong>. Here is <a href="https://travis-ci.org/ArnaudBuchholz/gpf-js/builds/367614025#L693" target="_blank">a recent example</a>.</p><p>I was first suspecting the <strong>concurrent execution of quality checks</strong> to be the root cause of the issue. I disabled it but, still, the problem persisted.</p><p>So, I took a closer look on how the coverage information was generated while executing tests in the browser.</p><p>In the Travis environment, <strong>chrome is spawned as a command line</strong> with options to disable user interface. When the tests are completed, an AJAX request is triggered to <strong>save the tests results</strong> through the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/grunt/connect-middleware/cache.js" target="_blank">cache middleware</a>. In the meantime a command line, responsible of spawning the browser, <strong><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.6/test/host/browser.js#L22-L48" target="_blank">waits for the cache to be updated before closing
the process</a></strong>.</p><p>Also, the test page is responsible of <strong>saving the coverage information</strong> through another AJAX request that goes to <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/grunt/connect-middleware/fs.js" target="_blank">fs
middleware</a>.</p><p>I realized that the sequence was incorrect: the tests results were sent <strong>before</strong> the coverage information. The two steps are now executed in the <a href="https://github.com/ArnaudBuchholz/gpf-js/commit/accc6830d5b29af93c734df5b63af082e2cef8ad#diff-1db347236fe2e62038c9d9829be1b967" target="_blank">correct order</a>.</p><h3>Improved gpf.require namespace</h3><p>The <a href="http://gpf-js.blogspot.ca/2018/01/my-own-require-implementation.html" target="_blank">modularization helper</a> was <strong>improved</strong> following the feedback obtained after the first use:</p><ul><li>Errors are <strong>better <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/241" target="_blank">documented</a></strong> to help the developer figure what went wrong and where</li><li>Loaded files now <strong><a href="https://github.com/ArnaudBuchholz/gpf-js/issues/244" target="_blank">appear inside the browser debugger</a></strong>. The trick consisted in augmenting the file content with a <a href="https://www.html5rocks.com/en/tutorials/developertools/sourcemaps/" target="_blank">SourceURL information</a></li></ul><h3>gpf.require.js</h3><h4>Flavor specification</h4><p>In the previous version, the <a href="https://github.com/fzaninotto/DependencyWheel" target="_blank">dependency wheel</a> was added to the source tile to give a visual representation of the <strong>dependencies</strong>. Also, each source file has been documented with 'tags' qualifying the <strong>feature</strong> or the <strong>host</strong> the source relates to.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/Release%200.2.6/sources%20before%20flavor.png" target="_blank">Sources without any flavor selection<br><img src="https://arnaudbuchholz.github.io/blog/post/Release%200.2.6/sources%20before%20flavor.png" alt="Sources without any flavor selection" title="Sources without any flavor selection" border="0" style="max-width: 100%"></a></p><p>Also, in the previous version, a <strong>syntax was initiated</strong> to instruct, in a readable way, which sources should be kept for a given <strong>flavor</strong>. Combining the features list, hosts specification and dependencies, an <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.6/make/flavor.js" target="_blank">algorithm - not my proudest one -</a> is capable of generating an array of booleans <strong>filtering the list of sources</strong>.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/Release%200.2.6/flavor.png" target="_blank">Example of flavor syntax<br><img src="https://arnaudbuchholz.github.io/blog/post/Release%200.2.6/flavor.png" alt="Example of flavor syntax" title="Example of flavor syntax" border="0" style="max-width: 100%"></a></p><p>As a result, the list of sources is <strong>reduced to meet the flavor specification</strong>.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/Release%200.2.6/sources%20after%20flavor.png" target="_blank">Sources with a flavor selection<br><img src="https://arnaudbuchholz.github.io/blog/post/Release%200.2.6/sources%20after%20flavor.png" alt="Sources with a flavor selection" title="Sources with a flavor selection" border="0" style="max-width: 100%"></a></p><p>Everything was ready to setup the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.6/make/flavor/require.json" target="_blank">require flavor specification</a>, it contains:</p><ul><li>The <strong>version in which the flavor was introduced</strong>: it is required to build the versions table in the <a href="https://github.com/ArnaudBuchholz/gpf-js#versions" target="_blank">readme page</a></li><li>The <strong>flavor filter string</strong></li><li>The tests required to <strong>validate the flavor</strong></li><li>A functional description of the flavor: it will be used to document flavors <em>(not yet implemented)</em>
</li><li>A technical description of the exposed API: the goal is to narrow down the list of namespaces / methods that are <strong>exposed</strong> by the flavor <em>(not yet implemented)</em>
</li></ul><h4>Reducing flavor size</h4><p>The very first results were <strong>quite disappointing</strong>:</p><ul><li>NodeJS implementation of require depended on <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.fs.html" target="_blank">gpf.fs</a>, <strong>almost the whole library was included</strong> (<a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.html#.define__anchor" target="_blank">gpf.define</a>, <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.interfaces.html" target="_blank">gpf.interfaces</a>, <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.stream.html" target="_blank">gpf.stream</a>, ...). So I decided to <strong>isolate</strong> <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.fs.html#.read__anchor" target="_blank">gpf.fs.read</a> into a <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.html#.read__anchor" target="_blank">smaller read feature</a>.</li></ul><ul><li>The same way, Browser implementation of require depends on <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.fs.html" target="_blank">gpf.http</a> to load the resource content. This namespace offers a <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.http.html#.mock__anchor" target="_blank">mocking helper</a> that is <strong>not needed</strong> for require. The code had to be <strong>drastically changed</strong> to successfully unplug this source.</li></ul><h4>Testing the flavor</h4><p>There is no way the flavor could be officially released without making sure <strong>it works as expected</strong>. Luckily, the whole library is already <strong>100% tested</strong>. Since the flavor description lists the test files to run, the development framework was altered to <strong>support a flavor parameter</strong> that:</p><ul><li>loads the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.6/build/gpf-require.js" target="_blank">flavor output file</a></li><li>loads and executes the necessary test files</li></ul><p>Mostly, the following files were modified:</p><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.6/test/host/web_loader.js#L168-L185" target="_blank">test/host/web_loader.js</a>: as it takes care of browsers tests execution</li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.6/test/host/loader.js#L383-L398" target="_blank">test/host/loader.js</a>: as it takes care of the other - command line type - hosts</li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.6/grunt/exec.js" target="_blank">grunt/exec.js</a>: it declares all the <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-TESTME.html" target="_blank">exec:test</a> tasks</li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.6/grunt/concurrent.js" target="_blank">grunt/concurrent.js</a>: it creates tasks to run all tests for a given version or flavor</li></ul><p>Obviously, the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.6/grunt/tasks/make.js#L24-L28" target="_blank">grunt make task</a> that schedules all the tasks required to build the version was also modified.</p><h2>Lessons learned</h2><p>Creating this first flavor was quite an <strong>interesting challenge</strong> and it took <strong>longer than expected</strong>. It forced me to <strong>rethink the way the code is articulated</strong>, especially with regards to host specific implementations.</p><p>The original pattern consisted in <strong>dictionaries</strong> containing operations indexed by host name. However, this had a major drawback: every time it was accessed, it was uselessly <strong>impacting the performances</strong> and <strong>producing complex code</strong>.</p><p>Now, when appropriate, a new helper is introduced to define the proper implementation depending on the host.</p><p>Here is the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.fs.html" target="_blank">gpf.http</a> use case:</p><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.6/src/http.js#L46-L60" target="_blank">_gpfHttpSetRequestImplIf</a> sets the http request method if the provided host matches the detected one</li><li>NodeJS' implementation <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.6/src/http/nodejs.js#L79" target="_blank">call</a></li></ul><p>Regarding <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.http.html#.mock__anchor" target="_blank">gpf.http.mock</a>, I decided to <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.6/src/http/mock.js#L160-L171" target="_blank">plug it in the gpf.http implementation</a> only when loaded, improving its <strong>modularity</strong>.</p><p>All this work paid, the <a href="https://arnaudbuchholz.github.io/gpf/plato/index.html" target="_blank">maintainability ratio</a> <strong>increased</strong> to 82.22.</p><p>One last thing, the require flavor supports only <strong>'modern' browsers</strong> meaning the ones where the <strong>compatibility layer is not required</strong>. However, as of now, the development framework does not distinguish the configured browsers.</p><p>A configuration <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/261" target="_blank">will be defined</a>.</p><h2>Next release</h2><p>The <a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/15" target="_blank">next release content</a> is not yet clearly defined. Since I will participate to <a href="https://openui5.org/ui5con/" target="_blank">UICon'18</a> as a presenter, I will first focus on delivering a good presentation.</p><p>I expect to work again on the library <strong>after</strong> June.</p></div>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-16892674671962922002018-04-18T01:31:00.000+02:002018-04-18T01:31:46.366+02:00JS Challenge<div dir="ltr" style="text-align: left;" trbidi="on">
I created some challenges to test your JavaScript skills, give it a try !<br />
<br />
<ul style="text-align: left;">
<li><a href="http://arnaudbuchholz.github.io/blog/post/JS%20Challenge/global%20scope.html" target="_blank">Accessing the global scope</a></li>
<li><a href="http://arnaudbuchholz.github.io/blog/post/JS%20Challenge/clone.html" target="_blank">Easy cloning</a></li>
</ul>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<img border="0" data-original-height="512" data-original-width="512" height="320" src="https://arnaudbuchholz.github.io/blog/post/JS%20Challenge/challenge.png" width="320" /></div>
<div>
<br /></div>
</div>
Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-22747436063784239582018-04-06T00:52:00.001+02:002018-12-07T19:49:58.044+01:00Release 0.2.5: Flavors<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/release.png" align="left">
This release finalizes WScript simulation required for Travis, it improves the development environment
and it introduces a mechanism to deliver smaller feature-centric versions of the library.
</div>
<div class="code markdown"><h2>New version</h2><p>Here comes the new version:</p><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/13?closed=1" target="_blank">Stories and bugs</a> implemented</li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/tree/v0.2.5" target="_blank">Sources</a></li><li><a href="https://arnaudbuchholz.github.io/gpf/doc/index.html" target="_blank">Documentation</a></li><li><a href="https://www.npmjs.com/package/gpf-js" target="_blank">NPM package</a></li></ul><h2>Release content</h2><h3>Finalized WScript simulation</h3><p>In the previous <a href="http://gpf-js.blogspot.ca/2018/03/release-024-attributes.html" target="_blank">release</a>, Travis continuous integration platform was <a href="http://gpf-js.blogspot.ca/2018/03/travis-integration.html" target="_blank">integrated</a>. To assess that <strong>all hosts are working fine</strong>, WScript had to be <strong>simulated</strong> because not supported on Linux environments.</p><p>At that time, <strong>the result was not perfect</strong> and the coverage was reaching only <a href="https://coveralls.io/github/ArnaudBuchholz/gpf-js?branch=master" target="_blank">99%</a> (including the <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-COVERAGE.html" target="_blank">documented exceptions</a>).</p><p>To achieve 100%, the environment is now <strong><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/test/host/wscript/WScript.js#L15-L27" target="_blank">tweaked to disable standard
objects</a></strong> and let the <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-COMPATIBILITY.html" target="_blank">compatibility layer</a> replace them with <a href="https://developer.mozilla.org/en-US/docs/Glossary/Polyfill" target="_blank">polyfills</a>.</p><h4>An unexpected test case</h4><p>The simulation is based on the <a href="https://www.npmjs.com/package/sync-request" target="_blank">sync-request</a> external library that implements <strong>synchronous HTTP requests in NodeJS</strong>. After enabling the JSON polyfills, <strong>the tests were failing</strong>... because of this library.</p><p>It revealed two problems:</p><ul><li>The <strong>JSON emulation was incomplete</strong></li><li>The <strong>compatibility tests</strong> were not covering all features of the JSON object</li></ul><h4>Universal Module Definition</h4><p>After fixing the polyfills problems (see below), the final validation consisted in <strong>running the debug and release versions</strong> of the library inside the WScript simulator.</p><p>However, those versions are based on the concatenation of all sources with the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/make/UMD.js" target="_blank">Universal Module
Loader</a>.</p><p>This mechanism <strong>detects the current host</strong> and defines the gpf symbol accordingly.</p><p>When NodeJS is detected, which is the case here, the loader assigns the gpf namespace to <a href="https://www.sitepoint.com/understanding-module-exports-exports-node-js/" target="_blank">module.exports</a> to be the result of a <a href="https://nodejs.org/api/modules.html#modules_modules" target="_blank">require</a> call.</p><p>But in WScript, the library is <strong>supposed to define the gpf symbol <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-LOADING.html" target="_blank">globally</a></strong>.</p><p>A <strong>workaround was implemented</strong> to simulate this behavior, it is done by <a href="https://github.com/ArnaudBuchholz/gpf-js/commit/7b85dca0f8af8008405b1cb6c7a82deacd9b8415" target="_blank">leveraging the AMD
syntax</a>.</p><h3>Improved JSON polyfill</h3><p>The <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/parse" target="_blank">JSON.parse method</a> accepts a reviver parameter that is used to <strong>transform parsed values</strong> before the final value is returned.</p><p><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.5/test/compatibility/json.js#L209-L233" target="_blank">New tests</a> were added and the missing part was implemented.</p><p>The same way, the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify" target="_blank">JSON.stringify
method</a> accepts two additional parameters:</p><ul><li>One, documented as replacer, that can be either a function to <strong>transform values before the stringification process</strong> or an array of string that serves as a <strong>whitelist of properties</strong> to be included in the final string.</li><li>A formatting parameter, documented as space, that is used to <strong>beautify the output</strong> of the method.</li></ul><p><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.5/test/compatibility/json.js#L121-L184" target="_blank">New tests</a> were added and the missing parts were implemented.</p><h3>Improved development environment</h3><p>In this release, a <strong>significant effort</strong> was put in the development environment.</p><h4>sources.json</h4><p>The <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.5/src/sources.json" target="_blank">sources.json file</a> is the <strong>spinal column</strong> of the library. It defines <strong>which sources are loaded</strong> and <strong>additional properties</strong> can be associated to them.</p><p>For instance, the "test" attribute defines when the associated test file (same name but under <a href="https://github.com/ArnaudBuchholz/gpf-js/tree/master/test" target="_blank">test</a> instead of <a href="https://github.com/ArnaudBuchholz/gpf-js/tree/master/src" target="_blank">src</a>) should be loaded. The default value being true, only the sources with no test file are flagged with false (for <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/src/sources.json#L12" target="_blank">example</a>).</p><p>The sources list was <strong><a href="http://gpf-js.blogspot.ca/2018/03/release-024-attributes.html" target="_blank">recently</a> cleaned up</strong> for <a href="https://deepscan.io/dashboard#view=project&pid=1923&bid=8681" target="_blank">DeepScan</a>. As a result, the remaining sources were all containing documentation to extract. Consequently, the <strong>"doc" attribute has been removed</strong>.</p><p>Also, <strong>a space separated list of tags</strong> was added, where:</p><ul><li><strong>core</strong> means the source is a core feature and it must always be loaded</li><li><strong>host:hostname</strong> means the source is specific to the host hostname</li><li>any other tag is interpreted as a <strong>feature name</strong> (for instance: define, require, fs...)</li></ul><p>This is used for the flavors development that will be detailed right after.</p><h4>GitHub tile</h4><p>A new tile was added to the development dashboard.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/Release%200.2.5/GitHub%20tile.png" target="_blank">The GitHub tile<br><img src="https://arnaudbuchholz.github.io/blog/post/Release%200.2.5/GitHub%20tile.png" alt="The GitHub tile" title="The GitHub tile" border="0" style="max-width: 100%"></a></p><p>It connects to the <a href="https://github.com/" target="_blank">GitHub API</a> to <strong>fetch the current release progress</strong> and <strong>links directly</strong> to it.</p><p>This was also the opportunity to <strong>redesign the whole dashboard</strong>. All the <strong>HTML is now generated</strong> and <a href="https://github.com/ArnaudBuchholz/gpf-js/tree/v0.2.5/res/tiles" target="_blank">modules</a> are organized thanks to the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.require.html#.define__anchor" target="_blank">gpf.require.define</a> feature.</p><h4>Documentation validation</h4><p>The only way to make sure that all links contained in the documentation are actually pointing to something is to try them. A new step in the build process was added to <strong><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.5/doc/validate.js" target="_blank">validate</a> the documentation by checking all the links</strong>.</p><p>Here again, it was a nice opportunity to test the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.http.html" target="_blank">gpf.http</a> feature.</p><h4>Dependency viewer</h4><p>The sources page was redesigned to <strong>visually show dependencies</strong>. It was done thanks to the awesome <a href="https://github.com/fzaninotto/DependencyWheel" target="_blank">dependency wheel</a> from Francois Zaninotto.</p></div><iframe width="560" height="315" src="https://www.youtube.com/embed/KRS4_lrG-kU" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen=""></iframe><div class="code markdown"><h4>Simplified release</h4><p>Last but not least, the library is now completely released through a <strong>single command line</strong>.</p></div><iframe width="560" height="315" src="https://www.youtube.com/embed/9H0QisPMCgg" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen=""></iframe><div class="code markdown"><h3>Flavors</h3><p>The library starts to <strong>generate interest</strong> but people often complain about the fact that it handles too many hosts and, consequently, <strong>it's too big</strong>.</p><p>Furthermore, some internal mechanics generate troubles:</p><ul><li>The use of <a href="https://docs.microsoft.com/en-us/scripting/javascript/reference/activexobject-object-javascript" target="_blank">ActiveXObject</a> may generate security issues on Internet Explorer</li><li>require("js") produces extra work with <a href="https://webpack.js.org/" target="_blank">webpack</a></li><li>...</li></ul><p>Hence, some time was invested to study the ability to build <strong>smaller - more dedicated - versions</strong> by having a way to specify <strong>which parts to consolidate</strong> in the library.</p><p>The idea is to rely on feature tags.</p><p>For instance, if one wants to use <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-REQUIRE.html" target="_blank">require</a> on <a href="https://nodejs.org/en/" target="_blank">NodeJS</a> only, the flavor would be "require host:nodejs". From there, an <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.5/make/flavor.js" target="_blank">algorithm</a> is capable of <strong>listing all the sources that must be included</strong> by:</p><ul><li>filtering sources from tags</li><li>adding selected sources' dependencies</li></ul><h3>Mapping stream</h3><p>This release delivers a <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.stream.Map.html" target="_blank">mapping stream</a> that completes the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.stream.Filter.html" target="_blank">filtering stream</a> introduced in the last version.</p><p>Here is a modified version of the previous release's sample that includes mapping:</p><p><code class="javascript gpf-blog"><span class="comment">// Reading a CSV file and keep only some records</span><span class="space">
</span><span class="comment">/*global Record*/</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">csvFile</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">fs</span><span class="symbol">.</span><span class="identifier">getFileStorage</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">openTextStream</span><span class="symbol">(</span><span class="string">"file.csv"</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">fs</span><span class="symbol">.</span><span class="identifier">openFor</span><span class="symbol">.</span><span class="identifier">reading</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">lineAdapter</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">LineAdapter</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">csvParser</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">csv</span><span class="symbol">.</span><span class="identifier">Parser</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">filter</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">Filter</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">record</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">record</span><span class="symbol">.</span><span class="identifier">FIELD</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="string">"expected value"</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">map</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">Map</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">record</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">Record</span><span class="symbol">(</span><span class="identifier">record</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="space">
</span><span class="identifier">output</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">WritableArray</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// csvFile -> lineAdapter -> csvParser -> filter -> map -> output</span><span class="space">
</span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">pipe</span><span class="symbol">(</span><span class="identifier">csvFile</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">lineAdapter</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">csvParser</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">filter</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">map</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">output</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">output</span><span class="symbol">.</span><span class="identifier">toArray</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">records</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">// process records</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span></code></p><p>See how you can <strong>easily swap the streams to refactor the code</strong>: let say that the Record class has a method named isObsolete which gives the filtering condition. You don't need to rely on the CSV literal object properties to reproduce the logic:</p><p><code class="javascript gpf-blog"><span class="comment">// Reading a CSV file and keep only some records</span><span class="space">
</span><span class="comment">/*global Record*/</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">csvFile</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">fs</span><span class="symbol">.</span><span class="identifier">getFileStorage</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">openTextStream</span><span class="symbol">(</span><span class="string">"file.csv"</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">fs</span><span class="symbol">.</span><span class="identifier">openFor</span><span class="symbol">.</span><span class="identifier">reading</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">lineAdapter</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">LineAdapter</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">csvParser</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">csv</span><span class="symbol">.</span><span class="identifier">Parser</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">filter</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">Filter</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">record</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="symbol">!</span><span class="identifier">record</span><span class="symbol">.</span><span class="identifier">isObsolete</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">map</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">Map</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">record</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">Record</span><span class="symbol">(</span><span class="identifier">record</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="space">
</span><span class="identifier">output</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">WritableArray</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// csvFile -> lineAdapter -> csvParser -> map -> filter -> output</span><span class="space">
</span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">pipe</span><span class="symbol">(</span><span class="identifier">csvFile</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">lineAdapter</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">csvParser</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">map</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">filter</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">output</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">output</span><span class="symbol">.</span><span class="identifier">toArray</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">records</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">// process records</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span></code></p><h2>Lessons learned</h2><p>This release enabled a 'real' <strong>productive use of the library</strong>. And, naturally, several weaknesses were identified.</p><p>For instance, requests to HTTPS websites were <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/246" target="_blank">not working</a> with NodeJS.</p><p>The same way, the usability of the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.require.html#.define__anchor" target="_blank">gpf.require.define</a> feature has to be improved <strong>whenever something goes wrong</strong>.</p><p>If loading fails because the resource file does not exist or its evaluation generates an error, the resulting exception must help the developer to quickly find and fix the problem:</p><ul><li><strong>Which</strong> resource is concerned?</li><li>Through which <strong>intermediate</strong> resources it was loaded?</li><li><strong>What</strong> is the problem?</li><li><strong>Where</strong> is the problem (line number)?</li></ul><p>Also, debugging loaded module might become a challenge since the evaluation model prevents the browser to <strong>map the file in the debugger</strong>.</p><p>But, in the end, this exercise <strong>validated the concepts</strong>: the tiles were quickly redesigned and common code put in modules that are shared by all of them:</p><ul><li>a <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.5/res/tiles/tile.js" target="_blank">tile base class</a></li><li>targeted helpers (<a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.5/res/dom.js" target="_blank">dom</a> and <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.5/res/dialogs.js" target="_blank">dialogs</a>)</li></ul><h2>Next release</h2><p>The <a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/14" target="_blank">next release content</a> will mostly focus on:</p><ul><li>Taking care of improving gpf.define.require</li><li>Releasing a standalone version of the gpf.define.require feature</li><li>Offering Object Oriented concepts (abstract classes, singletons, final classes)</li><li>Documenting JSON compatibility</li></ul></div>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-47828090735889666052018-03-02T05:58:00.001+01:002018-12-07T19:49:45.927+01:00Release 0.2.4: Attributes<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/release.png" align="left">
This release increases quality, it provides a new streaming tool and it introduces attributes.
Last but not least, a new scriping host was added to the supported list: Nashorn.
</div>
<div class="code markdown"><h2>New version</h2><p>Released right on time, here comes the new version:</p><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/12?closed=1" target="_blank">Stories and bugs</a> implemented</li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/tree/v0.2.4" target="_blank">Sources</a></li><li><a href="https://arnaudbuchholz.github.io/gpf/doc/index.html" target="_blank">Documentation</a></li><li><a href="https://www.npmjs.com/package/gpf-js" target="_blank">NPM package</a></li></ul><h2>Release content</h2><h3>Leftovers on gpf.require.define</h3><p>When writing the article <a href="http://gpf-js.blogspot.ca/2018/01/my-own-require-implementation.html" target="_blank">My own require implementation</a>, the handling of JavaScript modules was split in two distinct parts:</p><ul><li><a href="https://en.wikipedia.org/wiki/CommonJS" target="_blank">CommonJS</a> handling <em>(mostly because of synchronous requires)</em>
</li><li><a href="https://en.wikipedia.org/wiki/Asynchronous_module_definition" target="_blank">AMD</a> / GPF handling <em>(because asynchronous)</em>
</li></ul><p>Furthermore, 'simple' CommonJS modules (i.e. no require with just exports) were not supported.</p><p>Those problems were addressed by fixing the issue <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/219" target="_blank">#219</a>.</p><h3>Quality focus</h3><p>A significant effort was put in quality:</p><ul><li><a href="https://travis-ci.org/" target="_blank">Travis-CI</a>, a continuous integration platform, is enabled</li><li>This leaded to Linux testing and <a href="https://github.com/ArnaudBuchholz/gpf-js/tree/master/test/host/wscript" target="_blank">WScript simulation</a> for non-Windows environments</li><li>A <a href="https://coveralls.io/github/ArnaudBuchholz/gpf-js?branch=master" target="_blank">code coverage report</a> is updated on each build</li><li>A <a href="https://deepscan.io/dashboard/#view=project&pid=1923&bid=8681" target="_blank">code quality tool</a> is monitoring the sources</li><li>Project dependencies are <a href="https://david-dm.org/" target="_blank">verified</a></li></ul><p>This is all explained in the article <a href="http://gpf-js.blogspot.ca/2018/03/travis-integration.html" target="_blank">Travis integration</a>.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/Release%200.2.4/GPF%20Quality.png" target="_blank">GPF-JS quality<br><img src="https://arnaudbuchholz.github.io/blog/post/Release%200.2.4/GPF%20Quality.png" alt="GPF-JS quality" title="GPF-JS quality" border="0" style="max-width: 100%"></a></p><h3>Nashorn support</h3><p><a href="https://en.wikipedia.org/wiki/Nashorn_%28JavaScript_engine%29" target="_blank">Nashorn</a> is another JavaScript engine implemented in Java. It's like <a href="https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Rhino" target="_blank">Rhino</a>'s little brother: it's faster and delivered with <a href="http://www.oracle.com/technetwork/java/javase/downloads/index.html" target="_blank">Java Runtime Environment</a>.</p><p>This new Java host made me realize that the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.rhino.html" target="_blank">gpf.rhino</a> namespace was badly named. it has been deprecated it to the benefit of the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.java.html" target="_blank">gpf.java</a> namespace.</p><h3>Filtering stream</h3><p><a href="http://gpf-js.blogspot.ca/2017/12/release-023-streams-and-pipes.html" target="_blank">Previous release</a> introduced <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.stream.html#.pipe__anchor" target="_blank">gpf.stream.pipe</a> to chain streams together.</p><p>This release delivers a filtering processor that forwards or blocks data based on a filtering function. Here is a modified version of the previous release's sample that includes filtering:</p><p><code class="javascript gpf-blog"><span class="comment">// Reading a CSV file and keep only some records</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">csvFile</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">fs</span><span class="symbol">.</span><span class="identifier">getFileStorage</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">openTextStream</span><span class="symbol">(</span><span class="string">"file.csv"</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">fs</span><span class="symbol">.</span><span class="identifier">openFor</span><span class="symbol">.</span><span class="identifier">reading</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">lineAdapter</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">LineAdapter</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">csvParser</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">csv</span><span class="symbol">.</span><span class="identifier">Parser</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">filter</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">Filter</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">record</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">record</span><span class="symbol">.</span><span class="identifier">FIELD</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="string">"expected value"</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">output</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">WritableArray</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// csvFile -> lineAdapter -> csvParser -> filter -> output</span><span class="space">
</span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">pipe</span><span class="symbol">(</span><span class="identifier">csvFile</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">lineAdapter</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">csvParser</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">filter</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">output</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">output</span><span class="symbol">.</span><span class="identifier">toArray</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">records</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">// process records</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span></code></p><h3>Attributes</h3><p>Back to 2016, an article to <a href="http://gpf-js.blogspot.ca/2016/01/gpf-js-presentation.html" target="_blank">introduce GPF-JS</a> was written and it explains the concepts that were expected in the library: classes, interfaces and attributes.</p><p>The last pillar is introduced in this release.</p><p>It all starts by its simpler aspects:</p><ul><li>The <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.attributes.Attribute.html" target="_blank">gpf.attributes.Attribute</a> base class</li><li>The possibility to specify attributes in a class definition (not yet documented but check the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/test/attributes.js" target="_blank">tests</a>)</li><li>An <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.attributes.html#.get__anchor" target="_blank">API to extract attributes</a> from a class</li></ul><p>More features will come soon but this rewriting is very exciting as it fixes all the issues of the initial draft <em>(developed in 2015)</em>.</p><h3>Improved development environment</h3><p>It is now possible to use an authentication token to connect to the GitHub platform (previously user and password were required). It opens the door for a new tile in the dashboard to speed up development.</p><p>This is already enabled in the release process which has also been improved to prepare the following version in its final step. Unfortunately, the command line failed when testing it on this release and the remaining steps were finished manually</p><p>...which demonstrates how much value automation brings in this procedure...</p><h2>Lessons learned</h2><p>If it's not tested, it doesn't work. This was confirmed with Travis integration where the tools were run for the first time on a non-Windows environment.</p><p>The build process is taking longer, for two reasons:</p><ul><li>There is a new host to test (Nashorn)</li><li>The library now has 9 legacy test files</li></ul><p>A <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/238" target="_blank">strategy will be decided</a> to reduce the number of tests being run.</p><p>With the integration of <a href="https://deepscan.io/" target="_blank">Deepscan.io</a>, a lot of cleaning has been applied because there is no easy way to exclude specific files from the tool (other than naming them one by one). All the unused files were moved from src & test to the new <a href="https://github.com/ArnaudBuchholz/gpf-js/tree/master/lost%2Bfound" target="_blank">lost+found</a> folder.</p><h2>Next release</h2><p>The library becoming more complex (and bigger), it might evolve to propose several 'flavors'. The goal would be to provide smaller / dedicated versions that cut in the feature set.</p><p>For instance, a modern browser version could see the file system management removed as well as the compatibility layer.</p><p>Today, the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/build/dependencies.json" target="_blank">dependencies.json</a> file enumerates module dependencies. To be able to isolate them, a more accurate view is required and some modifications must be done inside the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/src/sources.json" target="_blank">sources.json</a> file.</p><p>The <a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/13" target="_blank">next release content</a> will mostly focus on:</p><ul><li>Documenting attributes</li><li>Refining the WScript simulation</li><li>Improving the development framework (illustrate the versions)</li><li>Adding a new stream helper to transform records</li></ul></div>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-67215804794613989922018-03-01T14:36:00.001+01:002018-12-07T19:57:37.696+01:00Travis integration<br>
<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/coding.png" align="left">
This article summarizes how Travis CI was plugged into the GPF-JS GitHub repository
to assess quality on every push.
</div>
<div class="code markdown"><h2>Quality control</h2><p>Since the beginning of the <a href="https://github.com/ArnaudBuchholz/gpf-js" target="_blank">GPF-JS</a> project, <a href="https://github.com/ArnaudBuchholz/gpf-js#metrics" target="_blank">quality</a> is a high priority topic.</p><p>Over the years, a complete development and testing framework has been built:</p><ul><li>Development follows <strong>TDD</strong> principles</li><li>The <strong>code is linted</strong> with eslint <em>(also with jshint, even if it is redundant)</em>
</li><li>The <strong>maintainability ratio</strong> is computed using <a href="https://www.npmjs.com/package/plato" target="_blank">plato</a> and <strong>verified through custom threshold</strong> values</li><li><strong>Testing is done with all supported hosts</strong></li><li>The <strong>coverage level</strong> is measured and controlled with custom threshold values</li><li>A <strong>web dashboard</strong> displays the metrics and gives access to the tools</li></ul><p>However, these validations were done locally and they had to be <strong>run manually before pushing the code</strong>.</p><p>To <strong>enable collaboration</strong> and ensure that all pull requests would go to the same <strong>validation process</strong>, an <a href="https://en.wikipedia.org/wiki/Continuous_integration" target="_blank">continuous integration</a> platform was required.</p><p>Another approach consists in creating <a href="https://git-scm.com/book/gr/v2/Customizing-Git-Git-Hooks" target="_blank">git hooks</a> to force the quality assessment before pushing. However, it requires the submitting platform to be properly configured. Having a central place where all the tests are being run <strong>guarantees the correct and unbiased execution of the tests</strong>.</p><h2>Travis CI</h2><p><a href="https://travis-ci.org/" target="_blank">Travis-CI</a> is a <strong>free solution</strong> that integrates smoothly with <a href="https://github.com/" target="_blank">GitHub</a>. Actually, the <strong>GitHub credentials are used to login</strong> to the platform.</p><p>After authentication, all the repositories are made available to Travis and each one can be enabled <strong>individually</strong>.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/Travis%20integration/steps.png" target="_blank">Documented steps<br><img src="https://arnaudbuchholz.github.io/blog/post/Travis%20integration/steps.png" alt="Documented steps" title="Documented steps" border="0" style="max-width: 100%"></a></p><p>The selected repositories must be <strong><a href="https://docs.travis-ci.com/user/getting-started" target="_blank">configured</a></strong> to instruct Travis on <strong>how the builds are done</strong>. To put it in a nutshell, a new file <em>(.travis.yml)</em> must be added.</p><p>Once everything is put together, the platform will <strong>monitor pushes and trigger builds automatically</strong>. Eventually, it documents the push in GitHub with a flag indicating if the build succeeded.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/Travis%20integration/push%20validation.png" target="_blank">Successful builds<br><img src="https://arnaudbuchholz.github.io/blog/post/Travis%20integration/push%20validation.png" alt="Successful builds" title="Successful builds" border="0" style="max-width: 100%"></a></p><p>When opening the Travis website, the user is welcomed with a nice <strong>dashboard listing the enabled repositories with their respective status</strong>.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/Travis%20integration/dashboard.png" target="_blank">Travis dashboard<br><img src="https://arnaudbuchholz.github.io/blog/post/Travis%20integration/dashboard.png" alt="Travis dashboard" title="Travis dashboard" border="0" style="max-width: 100%"></a>.</p><p>The <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/README.md" target="_blank">project README</a> can be modified to show the <a href="https://github.com/ArnaudBuchholz/gpf-js#gpf-library" target="_blank">last build result</a> using an image delivered by the Travis infrastructure:</p></div><a href="https://travis-ci.org/ArnaudBuchholz/gpf-js#" target="_blank">
<img src="https://travis-ci.org/ArnaudBuchholz/gpf-js.svg?branch=master" title="Travis-CI">
</a><div class="code markdown"></div><p><div class="code markdown"><h2>Changes in GPF-JS</h2><h3>.travis.yml</h3><p>The <em>.yml</em> extension stands for <strong><a href="https://en.wikipedia.org/wiki/YAML" target="_blank">YAML file</a></strong>: a structured format that is commonly used for configuration files. In the context of Travis, it describes the <strong>requirements and steps to build and validate</strong> the project.</p><h4>Requirements</h4><p>One obvious requirement of the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/.travis.yml" target="_blank">GPF-JS</a> development environment is <a href="https://nodejs.org/" target="_blank">NodeJS</a>. This is configured by defining the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/.travis.yml#L1" target="_blank">language setting</a> and selecting the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/.travis.yml#L2-L3" target="_blank">Long Term Support</a> version.</p><p>Chrome is also <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/.travis.yml#L5" target="_blank">required</a> to enable browser testing.</p><p><a href="https://gruntjs.com/" target="_blank">Grunt</a> is installed using the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/.travis.yml#L6" target="_blank">proper command line</a>.</p><p>The remaining requirements are described by the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/f0e1d5a5cd48820d65754f21864e2e0a642cf3be/package.json#L24-L63" target="_blank">project dependencies</a> and are <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/.travis.yml#L7" target="_blank">installed</a> through <a href="https://www.npmjs.com/" target="_blank">NPM</a>.</p><h4>Steps</h4><p>The default step being executed by Travis is <code class="javascript gpf-blog"><span class="identifier">npm</span><span class="space"> </span><span class="identifier">test</span></code></p><p>The resulting command line is specified in the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/package.json#L11" target="_blank">package.json</a> file; it executes <strong>grunt</strong> with the following tasks:</p><ul><li><strong>connectIf</strong> to setup a local web server necessary for browser and <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.http.html" target="_blank">gpf.http</a> testing</li><li><strong>concurrent:source</strong> to execute all tests with the source version (see <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-TESTME.html" target="_blank">library testing
tutorial</a>)</li></ul><p>But, first, <strong>the GPF-JS development environment requires a configuration file</strong> to be generated: it contains the list of detected hosts, the metrics thresholds and the HTTP port to be used to serve files.</p><h3>Configuration</h3><p>The configuration file is <strong>interactively created the very first time the grunt command is being executed</strong>. Indeed, the user input is expected to approve or change some values (such as the HTTP port).</p><p><a href="https://arnaudbuchholz.github.io/blog/post/Travis%20integration/grunt%20config.png" target="_blank">Grunt config<br><img src="https://arnaudbuchholz.github.io/blog/post/Travis%20integration/grunt%20config.png" alt="Grunt config" title="Grunt config" border="0" style="max-width: 100%"></a></p><p>However, on Travis, no user can answer the questions. Hence, the tool must be fully automatic.</p><p>When grunt is triggered, the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.3/gruntfile.js#L17" target="_blank">gruntfile.js</a> first checks if any configuration file already exist. If not, a <strong>default task takes place to run the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/make/config.js" target="_blank">configuration tool</a></strong>.</p><p>To support Travis, this default task processing was isolated in a folder of <a href="https://github.com/ArnaudBuchholz/gpf-js/tree/master/grunt/config" target="_blank">'first-launch' configuration
tasks</a>. The name of the file is used to name the task.</p><p>The configuration tool was then modified to handle a so-called <strong>quiet mode</strong> and the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/grunt/config/travis.js" target="_blank">Travis configuration task</a> leverages this option. Additionally, <strong>the task also executes the quality tasks</strong> (linters, plato & coverage).</p><p>So, the command <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/.travis.yml#L8" target="_blank">grunt travis</a> is executed before the testing step to ensure the existence of the configuration file.</p><h3>Challenges</h3><p>After figuring out the different steps, a long debugging session - <em>mostly composed of try and fail cycles</em> - was required to figure out the specificities of the platform.</p><p>The history of pushes on the <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/218" target="_blank">Travis integration issue</a> illustrates all the attempts that were made.</p><h4>Non-Windows environment</h4><p><a href="https://yakking.branchable.com/posts/truism-4-if-it-is-not-tested/" target="_blank">"If it's not tested, it doesn't work"</a></p><p>The very first issue was related to the environment itself. The development being made on Windows, the library has never been tested with a different operating system before.</p><p>Travis offers the possibility to <a href="https://docs.travis-ci.com/user/multi-os/" target="_blank">use different flavors</a> but the <strong>default one is Linux</strong>.</p><p>As a consequence, some scripts were adapted to handle the differences in the <strong>path separator</strong> (for <a href="https://github.com/ArnaudBuchholz/gpf-js/commit/3e6bab7b65e7606f4ef507539ad468167f643d7d" target="_blank">instance</a>).</p><div class="note"><p> NodeJS can absorb those differences. This is the reason why the library uses the / as a path separator but translates back to \ when running on Windows-specific host.</p></div><h4>Browser testing</h4><p>Another issue was related to browser testing. As a matter of fact, <strong>the Travis environment does not provide any graphical user interface</strong> by default. This being said, it means that you can't start a browser.</p><p>Fortunately, this is explained in the <a href="https://docs.travis-ci.com/user/gui-and-headless-browsers/" target="_blank">Travis documentation</a> where several options are proposed.</p><p>And, luckily, the <strong>GPF-JS development environment provides an <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/doc/tutorials/TESTME.md#command-line-browser-testing" target="_blank">alternate way</a> to test browsers</strong>.</p><p>Consequently, the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/f8771cba9b121d221983bf236840a014c98430f1/make/config.js#L161-L165" target="_blank">headless mode of chrome is used</a>.</p><h4>Wscript</h4><p>The operating system being Linux, the <a href="https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-xp/bb490887%28v=technet.10%29" target="_blank">Microsoft Scripting Host</a> is not available.</p><p>Unlike path separators, it is a normal situation as the <strong>configuration file includes a flag that is set upon detection of this scripting host</strong>.</p><p>However, one unexpected effect is bound to the code coverage measurement: a <strong>big part of the library is dedicated to bring older hosts to a <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-COMPATIBILITY.html" target="_blank">common feature set</a></strong> and WScript was running most of it. Thus, the coverage minimum was not reached and the build was failing.</p><p>This was temporarily addressed by <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/3bbbdfe177a4b96b61367cd5fd9d92292e38a84a/grunt/config/travis.js#L20" target="_blank">disabling the coverage threshold errors</a>.</p><p>Also, thanks to the <a href="https://github.com/mrpapercut/wscript" target="_blank">Wscript emulator project from Mischa Rodermond</a>, a <strong><a href="https://github.com/ArnaudBuchholz/gpf-js/tree/master/test/host/wscript" target="_blank">simple simulator</a> implemented with node</strong> validates the WScript specifics.</p><p>There are still some uncovered part, but they will be fixed in a <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/228" target="_blank">future release</a>.</p><h2>Other goodies</h2><p>After digging through GitHub projects, I also discovered and integrated the <a href="https://www.npmjs.com/package/coveralls" target="_blank">coveralls package</a> which can be used in conjunction with <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/.travis.yml#L9" target="_blank">Travis</a> so that coverage statistics are uploaded on a <a href="https://coveralls.io/github/ArnaudBuchholz/gpf-js?branch=master" target="_blank">dedicated board</a>.</p><p>I also added a <a href="https://david-dm.org/" target="_blank">dependency checker</a> as well as a <a href="https://deepscan.io/dashboard/#view=project&pid=1923&bid=8681" target="_blank">code quality checker</a> (which required <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/229" target="_blank">some cleaning</a>).</p><p>Everything is visible from the project readme.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/Release%200.2.4/GPF%20Quality.png" target="_blank">GPF-JS quality<br><img src="https://arnaudbuchholz.github.io/blog/post/Release%200.2.4/GPF%20Quality.png" alt="GPF-JS quality" title="GPF-JS quality" border="0" style="max-width: 100%"></a></p></div>
</p>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-47554086671509008502018-01-26T20:38:00.000+01:002018-12-07T19:41:54.012+01:00My own require implementation<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/student.png" align="left">
The release 0.2.2 of GPF-JS delivers a polymorphic modularization mechanism that mimics RequireJS and CommonJS
implementation of NodeJS. It was surprisingly easy to make it happen on all supported hosts now that the library
offers the basic services. This API combines lots of technologies, here are the implementation details.
</div>
<div class="code markdown"><h2>Introduction</h2><p>The <a href="https://github.com/ArnaudBuchholz/gpf-js/tree/v0.2.2" target="_blank">GPF-JS release 0.2.2</a> is finally <a href="https://gpf-js.blogspot.ca/2017/11/release-022-gpfrequire.html" target="_blank">out</a> and it contains several improvements:</p><ul><li>Better code quality</li><li>Better documentation</li><li>Some tests were rewritten</li></ul><div class="note"><p> Actually, the GPF-JS is already in <a href="https://github.com/ArnaudBuchholz/gpf-js/tree/v0.2.3" target="_blank">version 0.2.3</a> and it <a href="http://gpf-js.blogspot.ca/2017/12/release-023-streams-and-pipes.html" target="_blank">introduces stream piping</a>. But this article took me a very long time to finalize!</p></div><p>But the exciting part of it is the new <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.require.html" target="_blank">gpf.require</a> namespace that exposes a <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-REQUIRE.html" target="_blank">modularization helper</a>.</p><p>To give a bit of context, the article will start by explaining how modularity helps developers create better code. Then, a rapid overview of some existing modularization solutions will be covered. Finally, the implementation as well as the future of gpf.require will be explored.</p><h2>Modularity</h2><h3>One file to rule them all</h3><p><a href="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/ring.png" target="_blank">One File to bring them all and in the darkness bind them<br><img src="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/ring.png" alt="One File to bring them all and in the darkness bind them" title="One File to bring them all and in the darkness bind them" border="0" style="max-width: 100%"></a></p><p><a href="https://en.wikipedia.org/wiki/One_Ring" target="_blank">In the Land of Mordor where the Shadows lie.</a></p><p>To demonstrate the value of modularity, we will start with an extreme <a href="https://en.wikipedia.org/wiki/Edge_case" target="_blank">edge case</a>: an application which source code stands in <strong>one single file</strong> and a development team composed of <strong>several developers</strong>. We will assume that <strong>they all work simultaneously</strong> and a version control system is used to store the file.</p></div><svg width="100%" height="256" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<defs>
<marker id="head" orient="auto" markerWidth="2" markerHeight="4" refX="0.1" refY="2">
<path d="M0,0 V4 L2,2 Z" fill="red"></path>
</marker>
</defs>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/coding.png" x="32" y="0" width="64px" height="64px"></image>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/albert.png" x="0" y="0" width="64px" height="64px"></image>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/document edit.png" x="96" y="16" width="32px" height="32px"></image>
<text x="106" y="34" text-anchor="middle">Z</text>
<path marker-end="url(#head)" stroke-width="5" fill="none" stroke="red" d="M128,32 L232,104"></path>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/coding.png" x="32" y="64" width="64px" height="64px"></image>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/britney.png" x="0" y="64" width="64px" height="64px"></image>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/document edit.png" x="96" y="80" width="32px" height="32px"></image>
<text x="106" y="98" text-anchor="middle">Z</text>
<path marker-end="url(#head)" stroke-width="5" fill="none" stroke="red" d="M128,96 L232,120"></path>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/coding.png" x="32" y="128" width="64px" height="64px"></image>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/charles.png" x="0" y="128" width="64px" height="64px"></image>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/document edit.png" x="96" y="144" width="32px" height="32px"></image>
<text x="106" y="162" text-anchor="middle">Z</text>
<path marker-end="url(#head)" stroke-width="5" fill="none" stroke="red" d="M128,160 L232,136"></path>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/coding.png" x="32" y="192" width="64px" height="64px"></image>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/denise.png" x="0" y="192" width="64px" height="64px"></image>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/document edit.png" x="96" y="208" width="32px" height="32px"></image>
<text x="106" y="226" text-anchor="middle">Z</text>
<path marker-end="url(#head)" stroke-width="5" fill="none" stroke="red" d="M128,224 L232,152"></path>
<rect x="256" y="64" width="128px" height="128px" strike="black" fill="silver"></rect>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/cloud database.png" x="256" y="64" width="128px" height="128px"></image>
<path marker-end="url(#head)" stroke-width="5" fill="none" stroke="red" d="M400,128 L496,128"></path>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/document locked.png" x="512" y="96" width="64px" height="64px"></image>
<text x="534" y="116" text-anchor="middle">Z</text>
</svg><div class="code markdown"><p>(<em>Each developer works on a local copy of the source file and pushes to the file control system</em>)</p><p>The first obvious concern is the resulting file size. Depending on the application complexity, and assuming no library is used, <strong>the source file will be big</strong>.</p><p>And with size comes additional problems, leading to <a href="https://en.wikipedia.org/wiki/Maintainability" target="_blank">maintainability</a> issues. Even if guidelines are well established between the team members and comments are used, <strong>it will be hard to navigate through the lines of code</strong>.</p><p>JavaScript offer <a href="https://developer.mozilla.org/en-US/docs/Glossary/Hoisting" target="_blank">hoisting</a> that basically allows the developer to use a function before it is being declared.</p><p><code class="javascript gpf-blog"><span class="identifier">catName</span><span class="symbol">(</span><span class="string">"Chloe"</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">function</span><span class="space"> </span><span class="identifier">catName</span><span class="symbol">(</span><span class="identifier">name</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">console</span><span class="symbol">.</span><span class="identifier">log</span><span class="symbol">(</span><span class="string">"My cat's name is "</span><span class="space"> </span><span class="symbol">+</span><span class="space"> </span><span class="identifier">name</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="comment">// The result of the code above is: "My cat's name is Chloe"</span></code></p><p>But most linters will raise <a href="https://eslint.org/docs/rules/no-use-before-define" target="_blank">an error</a> for the above code. Indeed, it is a best practice to declare functions and variables before using them.</p><p>So, the team will end up <strong>cluttering the source file</strong> to ensure declarations are made before use.</p><p>Not being able to navigate easily in the code also generates a subtler problem: <strong>some logic might be repeated because it was hard to locate</strong>.</p><p>Finally, having all developers work on a single file will generate <strong><a href="https://betterexplained.com/articles/a-visual-guide-to-version-control/" target="_blank">conflicts</a></strong> when pushing it to the <a href="https://en.wikipedia.org/wiki/Version_control" target="_blank">version control system</a>. Fortunately, most of those systems have mechanism to solve the conflicts either automatically or with the help of the developer. But <strong>this takes developer time and it is not risk free</strong>.</p><p>Obviously, all these problems may appear on smaller files too but, in general, <strong>the larger the file the more problems</strong>.</p><p>Two questions are remaining:</p><ul><li>What could be the advantages of having a single source file?</li><li>What is the maximum file size?</li></ul><p>In the context of web applications, <strong>a single source file makes the application load faster</strong> as it reduces the number of requests. It does not mean that it must be written manually, there are many tools capable of <strong>building</strong> it.</p><p>For instance:</p><ul><li><a href="https://github.com/mishoo/UglifyJS2" target="_blank">UglifyJS</a> is a command line tool that concatenates and <a href="https://en.wikipedia.org/wiki/Minification_%28programming%29" target="_blank">minifies</a> sources. It also exists as a <a href="https://github.com/gruntjs/grunt-contrib-uglify" target="_blank">grunt task</a>.</li><li><a href="https://webpack.js.org/" target="_blank">Webpack</a> goes even beyond JavaScript concatenation by handling resources and <a href="https://www.stevefenton.co.uk/2012/11/compiling-vs-transpiling/" target="_blank">transpiling</a> ES6 code.</li></ul><p>Answering the second question is way more difficult. From my experience, <strong>any JavaScript source file bigger than 1000 lines is a problem</strong>. There might be good reasons to have a such a big file but it always comes with a cost.</p><p>In <a href="https://github.com/ArnaudBuchholz/gpf-js#metrics" target="_blank">GPF-JS</a>, the average number of lines per source file is a little under 100. But it has not <a href="https://arnaudbuchholz.github.io/gpf/plato/index.html" target="_blank">always been like this!</a></p><p><a href="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/plato%20average%20lines.png" target="_blank">Plato report on GPF-JS<br><img src="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/plato%20average%20lines.png" alt="Plato report on GPF-JS" title="Plato report on GPF-JS" border="0" style="max-width: 100%"></a></p><h3>Divide and rule</h3><p><a href="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/divide-and-conquer1.jpg" target="_blank">Or divide and conquer<br><img src="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/divide-and-conquer1.jpg" alt="Or divide and conquer" title="Or divide and conquer" border="0" style="max-width: 100%"></a></p><p>Code splitting is a good illustration of the <a href="https://en.wikipedia.org/wiki/Divide_and_rule" target="_blank">divide and rule</a> principle. Indeed, <strong>by slicing the application into smaller chunks, the developers have a better control over the changes</strong>.</p></div><svg width="100%" height="256" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<defs>
<marker id="head" orient="auto" markerWidth="2" markerHeight="4" refX="0.1" refY="2">
<path d="M0,0 V4 L2,2 Z" fill="red"></path>
</marker>
</defs>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/coding.png" x="32" y="0" width="64px" height="64px"></image>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/albert.png" x="0" y="0" width="64px" height="64px"></image>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/document edit.png" x="96" y="16" width="32px" height="32px"></image>
<text x="106" y="34" text-anchor="middle">A</text>
<path marker-end="url(#head)" stroke-width="5" fill="none" stroke="red" d="M128,32 L232,104"></path>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/coding.png" x="32" y="64" width="64px" height="64px"></image>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/britney.png" x="0" y="64" width="64px" height="64px"></image>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/document edit.png" x="96" y="80" width="32px" height="32px"></image>
<text x="106" y="98" text-anchor="middle">B</text>
<path marker-end="url(#head)" stroke-width="5" fill="none" stroke="red" d="M128,96 L232,120"></path>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/coding.png" x="32" y="128" width="64px" height="64px"></image>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/charles.png" x="0" y="128" width="64px" height="64px"></image>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/document edit.png" x="96" y="144" width="32px" height="32px"></image>
<text x="106" y="162" text-anchor="middle">C</text>
<path marker-end="url(#head)" stroke-width="5" fill="none" stroke="red" d="M128,160 L232,136"></path>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/coding.png" x="32" y="192" width="64px" height="64px"></image>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/denise.png" x="0" y="192" width="64px" height="64px"></image>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/document edit.png" x="96" y="208" width="32px" height="32px"></image>
<text x="106" y="226" text-anchor="middle">D</text>
<path marker-end="url(#head)" stroke-width="5" fill="none" stroke="red" d="M128,224 L232,152"></path>
<rect x="256" y="64" width="128px" height="128px" strike="black" fill="silver"></rect>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/cloud database.png" x="256" y="64" width="128px" height="128px"></image>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/document locked.png" x="422" y="48" width="64px" height="64px"></image>
<text x="444" y="68" text-anchor="middle">A</text>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/document locked.png" x="512" y="48" width="64px" height="64px"></image>
<text x="534" y="68" text-anchor="middle">B</text>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/document locked.png" x="422" y="144" width="64px" height="64px"></image>
<text x="444" y="164" text-anchor="middle">C</text>
<image xlink:href="https://ArnaudBuchholz.github.io/blog/document locked.png" x="512" y="144" width="64px" height="64px"></image>
<text x="534" y="164" text-anchor="middle">D</text>
</svg><div class="code markdown"><p>(<em>Each developer works on separate - smaller - source files</em>)</p><p>It sounds easy but it is not.</p><p>When it comes to deciding how to slice the application and organize the sources, there must be <strong>rules and discipline</strong>.</p><p>For example, maximizing the <strong>level of granularity</strong> by having one function per source will generate an overwhelming number of files. Also, <strong>not having a clear file structure will make the architecture obscure</strong> and will slow down the team.</p><h4>Files organization</h4><p>There are many guidelines and how-to on the web depending on the type of project or technology.</p><p>For instance:</p><ul><li><a href="https://www.terlici.com/2014/08/25/best-practices-express-structure.html" target="_blank">Best practices for Express app structure</a></li><li><a href="https://scotch.io/tutorials/angularjs-best-practices-directory-structure" target="_blank">AngularJS Best Practices: Directory Structure</a></li><li><a href="http://www.sapui5tutors.com/2016/03/sapui5-application-project-structuring.html" target="_blank">SAPUI5 application project structuring</a></li><li><a href="https://docs.sencha.com/extjs/6.0.0/guides/application_architecture/application_architecture.html" target="_blank">Introduction to ExtJS Application Architecture</a></li><li><a href="https://daveceddia.com/react-project-structure/" target="_blank">How to Structure Your React Project</a></li></ul><p>On the other hands, some tools offer the possibility to instantiate new projects with a predefined structure. Such as <a href="http://yeoman.io/" target="_blank">Yeoman</a> which contains <a href="http://yeoman.io/generators/" target="_blank">thousands of project generators</a>.</p></div><iframe width="560" height="315" src="https://www.youtube.com/embed/zBt2g9ekiug" frameborder="0" allowfullscreen="">
</iframe><div class="code markdown"><p>Yet, once the basic structure is in place, developers are still <strong>confronted with choices when new files must be created.</strong></p><p>So, regarding files organization, here are some basic principles (in no particular order):</p><ul><li>Document the structure and make it known: ask people to <a href="https://en.wikipedia.org/wiki/RTFM" target="_blank">Read The Fabulous Manual</a></li></ul><ul><li>Folder names are used to qualify the files they contain. Indeed, if a folder is named "controller", it is more than expected to find only <a href="https://en.wikipedia.org/wiki/Model–view–controller" target="_blank">controllers</a> in it. The same way, for a web application, a "public" folder usually indicates that its content is exposed</li></ul><ul><li>The folder qualification can be technical ("sources", "tests", "public") or functional ("controllers", "views", "dialogs", "stream") but mixing at the same level should be avoided. For one technology stack, if dialogs are implemented with controllers: it makes sense to see "dialogs" below "controllers" but having both in the same folder will be confusing</li></ul><ul><li>Try to stick to widely accepted (and understood) names: use "public" instead "www", use "dist" or "release" instead of "shipping"...</li></ul><ul><li>Try to avoid names that are too generic: "folder" <em>(true story)</em>, "misc", "util", "helpers", "data"...</li></ul><ul><li>Stick to one language <em>(don't mix French and English)</em>
</li></ul><ul><li>Select and stick to a naming formalism, such as <a href="https://en.wikipedia.org/wiki/Camel_case" target="_blank">Camel Case</a></li></ul><ul><li>Forget about the <a href="https://en.wikipedia.org/wiki/8.3_filename" target="_blank">8.3</a> or <a href="https://support.microsoft.com/fr-ca/help/177665/path-too-long-error-message-when-exceeding-max-path" target="_blank">MAX_PATH</a> limitations, names can be as long as necessary</li></ul><p><a href="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/back-in-my-day-funny-5.jpg" target="_blank">... and 640K is enough for everyone :-)<br><img src="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/back-in-my-day-funny-5.jpg" alt="... and 640K is enough for everyone :-)" title="... and 640K is enough for everyone :-)" border="0" style="max-width: 100%"></a></p><h4>Level of granularity</h4><p>Once the structure is clearly defined, the only difficulty remaining is to figure out what to put inside the files. Obviously, their names must be self-explanatory about purpose and content.</p><strong>Struggling to choose the right file name is usually a good sign that it contains more than necessary</strong>: splitting may<p>help.</p><p>In case of doubts, just associate:</p><ul><li>folders to <a href="https://en.wikipedia.org/wiki/Namespace" target="_blank">namespaces</a></li><li>files to <a href="https://en.wikipedia.org/wiki/Class_%28computer_programming%29" target="_blank">classes</a></li></ul><p>Then, try to stick to the <a href="https://en.wikipedia.org/wiki/SOLID_%28object-oriented_design%29" target="_blank">SOLID</a> <a href="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/SOLID.jpg" target="_blank">principles<br><img src="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/SOLID.jpg" alt="principles" title="principles" border="0" style="max-width: 100%"></a></p><p>In particular, <strong>the <a href="https://en.wikipedia.org/wiki/Single_responsibility_principle" target="_blank">Single Responsibility principle</a> should drive the choices when creating a new file</strong>.</p><p>Here is a shamelessly modified copy of the Wikipedia definition as it summarizes the idea:<em>
"The single responsibility principle is a computer programming principle that states that every module should have responsibility over a single part of the functionality provided by the software, and that responsibility should be entirely encapsulated by the module. All its services should be narrowly aligned with that responsibility."</em>
</p><div class="note"><p> Referring to the <a href="https://en.wikipedia.org/wiki/Liskov_substitution_principle" target="_blank">Liskov substitution principle</a> and the <a href="https://en.wikipedia.org/wiki/Dependency_inversion_principle" target="_blank">Dependency inversion principle</a> may look weird when considering source files but the following sections (in particular interface and mocking) will shed some light on the analysis behind this statement.</p></div><p>Basically, because of this definition, a source file is no more a senseless bunch of lines of codes but rather a <strong>self-contained feature</strong>. Hence, it will be referred as a <strong>module</strong>.</p><h4>Modules</h4><p>The advantages of modules are multiple:</p><ul><li><strong>Testability</strong>: it is sometimes difficult to draw the line between <a href="https://en.wikipedia.org/wiki/Unit_testing" target="_blank">unit testing</a> and <a href="https://en.wikipedia.org/wiki/Integration_testing" target="_blank">integration testing</a>. Long story short, if several modules are needed to run a test, it sounds like integration. On the other hand, if the module can be tested with no or very few dependencies (maybe through mocking), this is unit testing. So, ideally, <strong>a module can be easily isolated to achieve unit testing</strong></li></ul><ul><li><strong>Maintainability</strong>: on top of the testability, the file size should be relatively small. Both facts greatly increase the module's <a href="https://en.wikipedia.org/wiki/Maintainability" target="_blank">maintainability</a>. This means that <strong>it is easier (in terms of complexity and effort) to improve or fix</strong> the module.</li></ul><ul><li><strong>Reusability</strong>: no finger pointing here. Every developer starts his career by learning the 'virtues' of copy & paste. It takes time and experience to realize that <strong><a href="https://hethmonster.wordpress.com/2010/09/21/duplicate-code-is-evil/" target="_blank">code duplication is evil</a></strong>. Modules are an indisputable alternative to copy & paste because <strong>they are designed to be reusable</strong>.</li></ul><h3>Loading</h3><p>The way modules are loaded <strong>varies for each host</strong> and some examples will be shown in the 'Existing implementations' section. This is where <strong>GPF-JS brings value by offering one uniform solution</strong> for all supported hosts.</p><p>The purpose of this part is not to explain each mechanism but rather describe the overall philosophy when it comes to loading the sources of an application.</p><h4>Everything at once</h4><p><a href="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/X-All-The-Y.jpg" target="_blank">Load all the files!<br><img src="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/X-All-The-Y.jpg" alt="Load all the files!" title="Load all the files!" border="0" style="max-width: 100%"></a></p><p>As already mentioned before, there are tools designed to concatenate and minify all sources to generate <strong>a single application file</strong>. The resulting file is consequently <strong>self-sufficient</strong> and can be easily loaded.</p><p>Dependencies will be addressed later but, to generate this file, the tool must know the list of files to concatenate (or, usually, the folder where to find the files).</p><p>An alternative to load the application is to maintain a <strong>list of sources to load</strong>... in the correct order so that <strong>dependencies are loaded before their dependent modules</strong>.</p><p>As a result, in an HTML page, a big list of <a href="https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script" target="_blank">script</a> tags is required to include all sources.</p><p>For example, let consider the following modules: A, B, C, D, E and F</p><ul><li>A requires B and C to be loaded</li><li>B requires D to be loaded</li><li>E requires F to be loaded</li><li>F can be loaded separately</li></ul><p>The resulting ordered lists can be:</p><ul><li>D, B, C, A, F, E</li><li>C, D, F, B, A, E</li><li>F, D, C, B, A, E</li><li>F, E, D, B, C, A</li><li>...</li></ul><p>Advantages are:</p><ul><li>Sequence being managed by a list, it gives a <strong>clear understanding of the loading process</strong> and allows the developers to fully control it</li><li>Unlike the unique concatenated file that is regenerated for each release, some files might not even change between versions. Consequently, <strong>the loading process can benefit from caching mechanism</strong> and the application starts faster.</li></ul><p>Disadvantages are:</p><ul><li>Every time a new source is created, <strong>it must be added to the list in the correct order</strong> to be loaded</li><li><strong>The more files, the bigger the list</strong> (and the more complex to maintain)</li><li>When the list becomes too big, one may lose track of which files are really required. In the worst situation, <strong>some may be loaded even if they are not used anymore</strong></li><li><strong>Any change in dependencies implies a re-ordering of the list</strong></li></ul><div class="note"><p> This is not the preferred solution but this is how the source version of GPF-JS is <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/src/sources.json" target="_blank">handled</a>. The lazy me has created some tools to maintain this list in the dashboard.</p></div><h4>Lazy loading</h4><p>Instead of loading all the files at once, another method consists in loading what is needed to <strong>boot the application</strong> and then load <strong>additional files when required</strong>. This is known as <a href="https://en.wikipedia.org/wiki/Lazy_loading" target="_blank">lazy loading</a>.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/lazy.jpg" target="_blank">Or the energy saving mode...<br><img src="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/lazy.jpg" alt="Or the energy saving mode..." title="Or the energy saving mode..." border="0" style="max-width: 100%"></a></p><p>For each part of the loading process, the list of files to load must be maintained. Hence the benefits are mostly focused on the <strong>time needed to start the application</strong>.</p><p>Back to the previous example, E and F are big files requiring lot of time to load and evaluate.</p><ul><li>Loading the application is done with the booting list: D, B, C, A</li><li>And, when required, the feature implemented by E is loaded with: F, E</li></ul><p>Advantages are:</p><ul><li>Faster startup of the application, it also implies <strong>smaller initial <a href="https://en.wikipedia.org/wiki/Application_footprint" target="_blank">application footprint</a></strong></li><li>Loading instructions are split into smaller lists which are consequently <strong>easier to maintain</strong></li></ul><p>Disadvantages are:</p><ul><li>Once the application started, accessing a part that has not been previously loaded will <strong>generate delay</strong></li><li>If any trouble occurs during the additional loading phase (such as no network) <strong>the application breaks</strong></li></ul><h4>Dependency management</h4><p>Whether the application is based on static or lazy loading, both mechanism require lists of files to load. However, <strong>maintaining these lists is time consuming and error prone</strong>.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/nolist.jpg" target="_blank">Is there any way to get rid of these lists?<br><img src="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/nolist.jpg" alt="Is there any way to get rid of these lists?" title="Is there any way to get rid of these lists?" border="0" style="max-width: 100%"></a></p><p>It's almost impossible to get rid of the lists but <strong>changing the scope</strong> simplifies their management. Instead of maintaining global lists of files to load, <strong>each module can declare which subsequent modules it depends on</strong>. This way, when loading modules, <strong>the loader must check dependencies and process them recursively</strong>.</p><p>Hence, the whole application can be started by loading one module which depends on everything else. Also, it means that, on top of implementing a feature, a module contains information about the way it should be loaded.</p><p>Back to the previous example:</p><ul><li>A contains information that B and C must be loaded</li><li>B contains information that D must be loaded</li><li>E contains information that F must be loaded Then:</li><li>Loading the application is done by loading A</li><li>And, when required, the feature implemented by E is loaded with E</li></ul><p>Advantages are:</p><ul><li>No big lists to maintain, <strong>each module declare its dependencies</strong> separately</li><li>Better <strong>visibility on module dependencies</strong></li><li>Simplified <strong>reusability</strong> of modules</li><li><strong>New files</strong> are loaded if declared as dependencies</li><li>Only <strong>required files</strong> are loaded</li></ul><p>Disadvantages are:</p><ul><li><strong>Loading a module is more complex</strong>: dependencies must be extracted and resolved recursively</li><li>Having such a fine level of granularity increases the <strong>risk of tangled dependencies</strong> and, in the worst case, deadlocks. For instance, three modules: A, B and C. If A depends on B, B depends on C and C depends on A, it is impossible to load any of them.</li><li>If one module is a common dependency to several other modules, <strong>it should be loaded only once</strong>. If not, the loading performances will be decreased because of multiple loading of the same module</li></ul><div class="note"><p> Obviously, for the last part, the loading mechanism must handle some kind of caching.</p></div><h3>Interface</h3><p>Assuming modules have information about their dependencies, how do they access the features being exposed?</p><h4>Global variables</h4><p><a href="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/js-global-variables-everywhere.jpg" target="_blank">Look Woody...<br><img src="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/js-global-variables-everywhere.jpg" alt="Look Woody..." title="Look Woody..." border="0" style="max-width: 100%"></a></p><p>There are many mechanism but the simplest one is to alter the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Grammar_and_types#Variable_scope" target="_blank">global scope</a> to export functions and variables.</p><p>Indeed, in a flat module, declaring functions and variables is enough to make them <strong>available globally</strong> in the application.</p><p><code class="javascript gpf-blog"><span class="comment">// No dependencies</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">ANY_CONSTANT</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="string">"value"</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">function</span><span class="space"> </span><span class="identifier">myExportedFunction</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">/* ...implementation... */</span><span class="space">
</span><span class="symbol">}</span></code></p><div class="note"><p> To be precise, this depends on the host and how the module is loaded. Let's stay simple for now.</p></div><p>The main advantage is <strong>simplicity</strong>.</p><p>However, problems occur as soon as the complexity of the module grows.</p><p>In fact, the module may also declare functions and variables that are used internally and those <strong>implementation details should not be exposed</strong>. With the above mechanism, it is almost impossible to guarantee that they won't be accessedor, worst, altered)<em> and, consequently, it creates <strong>tight <a href="https://en.wikipedia.org/wiki/Loose_coupling#In_programming" target="_blank">coupling</a></strong> with the rest of the application.</em></p><p>This means that it gives less flexibility when it comes to maintaining the module.</p><p>One easy <a href="https://en.wikipedia.org/wiki/Workaround" target="_blank">workaround</a> consists in encapsulating these implementation details inside a <strong>private scope</strong> using an <a href="https://en.wikipedia.org/wiki/Immediately-invoked_function_expression" target="_blank">Immediately Invoked Function Expression</a>.</p><p><code class="javascript gpf-blog"><span class="comment">// No dependencies</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">ANY_CONSTANT</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="string">"value"</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">myExportedFunction</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// IIFE</span><span class="space">
</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="string">"use strict"</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// Begin of private scope</span><span class="space">
</span><span class="comment">// Any functions or variables declared here are 'private'</span><span class="space">
</span><span class="comment">// Reveal function</span><span class="space">
</span><span class="identifier">myExportedFunction</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">/* ...implementation... */</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="comment">// End of private scope</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">)</span><span class="symbol">;</span></code></p><p>Another aspect that should be considered when dealing with global symbols is <strong><a href="https://en.wikipedia.org/wiki/Name_collision" target="_blank">name collisions</a></strong>. This can be quickly addressed with <strong><a href="https://en.wikipedia.org/wiki/Namespace" target="_blank">namespaces</a></strong> but here comes a challenging example.</p><h4>Module interface</h4><p><a href="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/UseTheAPILuke.jpg" target="_blank">Do. Or do not. There is no try.<br><img src="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/UseTheAPILuke.jpg" alt="Do. Or do not. There is no try." title="Do. Or do not. There is no try." border="0" style="max-width: 100%"></a></p><p>Let's consider an application capable of loading and saving documents. Developers were smart enough to isolate the serialization part inside a module which exposes two methods: save and load. The application grows in functionality and, for some reasons, saving and loading evolve to a new format. Still, for backward compatibility reasons, the application should be able to load older documents.</p><p>At that point, there are several possibilities:</p><ul><li>Changing the code to support both formats in one function is not an option: for the sake of <strong>maintainability</strong>, there must be a wrapper function to detect the format and then switch to the proper implementation.</li></ul><ul><li><strong>Rename</strong> the save and load methods to include a version number (for instance: saveV1, loadV1, saveV2 and loadV2). Then the code <em>(production and test)</em> must be modified to use the proper method depending on the format that needs to be serialized.</li></ul><ul><li><strong>Create a namespace</strong> for each version and have the methods being set inside the proper namespace: app.io.v1.save, app.io.v1.load, app.io.v2.save and app.io.v2.load. Again, the code must be adapted but provided the <strong>namespace can be a parameter</strong> (thanks to JavaScript where <a href="https://javascriptweblog.wordpress.com/2010/12/07/namespacing-in-javascript/" target="_blank">namespaces are objects</a>), this reduces the cost.</li></ul><ul><li><strong>Define an interface</strong> and have one module per version that exposes this interface. It's almost like with namespacing but accessing the proper object does not depend on global naming but rather on <strong>loading the right module</strong>.</li></ul><p>For instance: <code class="javascript gpf-blog"><span class="keyword">var</span><span class="space"> </span><span class="identifier">currentVersion</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="number">2</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">io</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">loadModule</span><span class="symbol">(</span><span class="string">"app/io/v"</span><span class="space"> </span><span class="symbol">+</span><span class="space"> </span><span class="identifier">currentVersion</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// io exposes save and load methods</span><span class="space">
</span><span class="keyword">function</span><span class="space"> </span><span class="identifier">save</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">fileName</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">// Always save with the lastest version</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">io</span><span class="symbol">.</span><span class="identifier">save</span><span class="symbol">(</span><span class="identifier">fileName</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="keyword">function</span><span class="space"> </span><span class="identifier">load</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">fileName</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">version</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">detect</span><span class="symbol">(</span><span class="identifier">fileName</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">if</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">version</span><span class="space"> </span><span class="symbol">!==</span><span class="space"> </span><span class="identifier">currentVersion</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">loadModule</span><span class="symbol">(</span><span class="string">"app/io/v"</span><span class="space"> </span><span class="symbol">+</span><span class="space"> </span><span class="identifier">version</span><span class="symbol">)</span><span class="symbol">.</span><span class="identifier">load</span><span class="symbol">(</span><span class="identifier">fileName</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">io</span><span class="symbol">.</span><span class="identifier">load</span><span class="symbol">(</span><span class="identifier">fileName</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span></code></p><p>The impact of the next version is limited to changing the name of the module. Remember lazy loading? This is an example where the oldest versions are loaded only when needed (reducing the application footprint on startup).</p><p>So, ideally, on top of providing dependency information, the module also <strong>exposes a known <a href="https://en.wikipedia.org/wiki/Application_programming_interface" target="_blank">interface</a></strong>. It defines the expectations regarding the methods and variables that will be available and simplifies its integration and reusability.</p><h3>Module loader</h3><p>To summarize what has been presented so far:</p><ul><li>An application is composed of modules, they are <strong>smaller parts encapsulating single functionalities</strong></li><li>To simplify the loading of the application, <strong>modules declares their list of dependencies</strong></li><li>Loading the module gives access to a <strong>given set of API a.k.a. interface</strong></li></ul><p>Consequently, the ultimate module loader is capable of extracting and resolving dependencies. Then it uses <a href="https://en.wikipedia.org/wiki/Dependency_injection" target="_blank">dependency injection</a> to give access to the interfaces exposed by its dependencies.</p><h3>Mocking</h3><p><a href="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/mocking.png" target="_blank">Test double<br><img src="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/mocking.png" alt="Test double" title="Test double" border="0" style="max-width: 100%"></a></p><p>Another subtle benefit of modularity is related to unit testing.</p><p>Coming back to the example of module B requiring module D to be loaded: what if module D can't be loaded or used in a test environment? This module might require critical resources. For instance, it could access a database and alter its content: nobody wants the tests to mess with real data.</p><p>Does this mean that module B can't be tested because module D is not testable?</p><p>Well, if module D is exposing a well-defined interface and if the modularization system offers a way to substitute a module by another one, it is possible to create a <strong>fake module that exposes the same interface</strong> but <strong>mocking the functionalities</strong>.</p><h2>Existing implementations</h2><h3>Browser includes</h3><p><a href="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/browsers.jpg" target="_blank">We are... browsers!<br><img src="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/browsers.jpg" alt="We are... browsers!" title="We are... browsers!" border="0" style="max-width: 100%"></a></p><p>The simplest way to include a JavaScript source in an HTML page is to add a <a href="https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script" target="_blank">script tag</a>.</p><p>Even though <strong>loading is asynchronous</strong>, and unless the attributes <em>async</em> or <em>defer</em> are specified, <strong>scripts are evaluated in the same order as they are declared</strong> in the page.</p><p>By default, the scope of evaluation is global. Consequently, <strong>any variable declaration is globally available</strong> as if it was a member of the <a href="https://developer.mozilla.org/en-US/docs/Web/API/Window" target="_blank">window object</a>.</p><p>As explained before, it is possible to create a <strong>private scope using an IIFE</strong>. Exposing the module interface can be done through <strong>assigning new members to the window object</strong> or using <em>this</em> in the IIFE.</p><p>Here is a small variation to make the exported API more explicit:</p><p><code class="javascript gpf-blog"><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">exports</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="string">"use strict"</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// Begin of private scope</span><span class="space">
</span><span class="comment">// Any functions or variables declared here are 'private'</span><span class="space">
</span><span class="comment">// Reveal function</span><span class="space">
</span><span class="identifier">exports</span><span class="symbol">.</span><span class="identifier">myExportedFunction</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">/* ...implementation... */</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="comment">// End of private scope</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">(</span><span class="keyword">this</span><span class="symbol">)</span><span class="symbol">)</span><span class="symbol">;</span></code></p><p>In this context, declaring or loading dependencies inside a module becomes quite complex. Indeed, it is possible to generate script tags using JavaScript (or use AJAX requests) but the code must wait for the subsequent script to be loaded.</p><h3>RequireJS</h3><p>The <a href="http://requirejs.org/" target="_blank">RequireJS</a> library was originally created for browsers but also works with other hosts (Rhino).</p><p>By offering a unique mechanism to <strong>load and inject modules</strong>, this helper solves all the problems of modularization. The <strong>factory function</strong> generates a private scope and receives the injected dependencies as parameters.</p><p><code class="javascript gpf-blog"><span class="identifier">define</span><span class="symbol">(</span><span class="symbol">[</span><span class="string">"dependency"</span><span class="symbol">]</span><span class="symbol">,</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">dependency</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="string">"use strict"</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// Begin of private scope</span><span class="space">
</span><span class="comment">// Any functions or variables declared here are 'private'</span><span class="space">
</span><span class="comment">// Reveal function</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">myExportedFunction</span><span class="symbol">:</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">/* ...implementation... */</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// End of private scope</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span></code></p><p>The article <a href="https://www.sitepoint.com/understanding-requirejs-for-effective-javascript-module-loading/" target="_blank">"Understanding RequireJS for Effective JavaScript Module Loading"</a> covers all the steps to start with the library.</p><h3>NodeJs</h3><p><a href="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/this-is-javascript.jpg" target="_blank">Front end and back end...<br><img src="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/this-is-javascript.jpg" alt="Front end and back end..." title="Front end and back end..." border="0" style="max-width: 100%"></a></p><p><a href="https://en.wikipedia.org/wiki/Node.js" target="_blank">NodeJS</a> offers a modularization helper initially inspired from <a href="https://en.wikipedia.org/wiki/CommonJS" target="_blank">CommonJS</a>.</p><p>Loading a module is as easy as calling the <a href="https://nodejs.org/api/modules.html#modules_require" target="_blank">require</a> function: <strong>loading is synchronous</strong>. The same way, the module can load its dependencies using require.</p><strong>Modules have a private scope</strong>. If the global scope must be altered <em>(which is not recommended)</em>, the <strong>global symbol</strong> is available.<p>But the normal way to expose the module API is to assign members to the <strong>module.exports</strong> object (the shortcut exports can also be used).</p><p>A typical module would look like:</p><p><code class="javascript gpf-blog"><span class="string">"use strict"</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// Begin of private scope</span><span class="space">
</span><span class="comment">// Any functions or variables declared here are 'private'</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">dependency</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">require</span><span class="symbol">(</span><span class="string">"dependency"</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// Reveal function</span><span class="space">
</span><span class="identifier">module</span><span class="symbol">.</span><span class="identifier">exports</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">myExportedFunction</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">/* ...implementation... */</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// End of private scope</span></code></p><p>Actually, there are two types of modules.</p><h4>NPM modules</h4><p>The <a href="https://www.npmjs.com/" target="_blank">NPM repository</a> stores a huge collection of modules which can be <strong>downloaded and updated</strong> using the <a href="https://en.wikipedia.org/wiki/Npm_%28software%29" target="_blank">NPM</a> command line (installed with NodeJS). These modules are usually defined as <strong>project dependencies</strong> through the <a href="https://docs.npmjs.com/files/package.json" target="_blank">package.json</a> file or they can be globally installed.</p><p>When loading these modules, <strong>they are referenced by an absolute name</strong>.</p><p>For instance, the <a href="https://www.npmjs.com/package/gpf-js" target="_blank">GPF-JS NPM module</a>, once installed, can be loaded with: <code class="javascript gpf-blog"><span class="keyword">var</span><span class="space"> </span><span class="identifier">gpf</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">require</span><span class="symbol">(</span><span class="string">"gpf-js"</span><span class="symbol">)</span><span class="symbol">;</span></code></p><p>You can <a href="https://runkit.com/npm/gpf-js" target="_blank">experiment it</a> yourself.</p><h4>Local modules</h4><p>On the other hand, the project may contain its own set of modules: they don't need to be published to be located. To load a local module, <strong>the path to the source file must be provided</strong>, it is <strong>relative to the current module</strong>.</p><p><code class="javascript gpf-blog"><span class="keyword">var</span><span class="space"> </span><span class="identifier">CONST</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">require</span><span class="symbol">(</span><span class="string">"./const.js"</span><span class="symbol">)</span><span class="symbol">;</span></code></p><h3>PhantomJS</h3><p><a href="http://phantomjs.org/" target="_blank">PhantomJS</a> proposes a similar approach to NodeJS when it comes to loading modules: a require function. However, it is a custom and limited implementation. See the following <a href="https://stackoverflow.com/questions/11873853/phantomjs-require-a-relative-path" target="_blank">question thread</a> for details.</p><p>Furthermore, this host is a hybrid between a command line and a browser. Hence, for most of the needs, it might be simpler to <strong>consider the browser approach</strong>.</p><h3>Bundlers</h3><p>A <a href="https://medium.com/@gimenete/how-javascript-bundlers-work-1fc0d0caf2da" target="_blank">JavaScript bundler</a> is a tool designed to concatenate an application composed of modules (and resources) into a single self-sufficient and easy-to-load file.</p><p>The good parts are:</p><ul><li>it enforces code reuse by <strong>enabling modularization when developing</strong></li><li>it <strong>speeds up the application loading</strong> by limiting the number of files</li><li>by relying on the dependence tree, it also <strong>trims files which are never loaded</strong></li></ul><p>The bad parts are:</p><ul><li>The generated code is usually <strong>obfuscated</strong>, debugging might be complicated</li></ul><div class="note"><p> Some bundlers are capable of generating the <a href="https://developer.mozilla.org/en-US/docs/Tools/Debugger/How_to/Use_a_source_map" target="_blank">source map</a> information. This helps a lot when debugging but the source code must be exposed.</p></div><ul><li>The application file must be <strong>generated before running the application</strong>. Depending on the number of files it might take some time</li></ul><p><a href="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/you-were-so-busy-wondering-if-you-could-do-it-you-never-stopped-to-ask-if-you-should.jpg" target="_blank">The lack of humility before vanilla JavaScript that's being displayed here, uh... staggers me.<br><img src="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/you-were-so-busy-wondering-if-you-could-do-it-you-never-stopped-to-ask-if-you-should.jpg" alt="The lack of humility before vanilla JavaScript that's being displayed here, uh... staggers me." title="The lack of humility before vanilla JavaScript that's being displayed here, uh... staggers me." border="0" style="max-width: 100%"></a></p><h4>Browserify</h4><p><a href="http://browserify.org/" target="_blank">Browserify</a> is a bundler understanding the <strong>NodeJS require syntax</strong> and capable of collecting all JavaScript dependencies to generate <strong>one single file loadable in a browser</strong> (hence the name).</p><h4>WebPack</h4><p><a href="https://webpack.js.org/" target="_blank">WebPack</a> is like browserify on steroids as it does more than just collecting and packing JavaScript dependencies. It is also capable of embedding <strong>resource files</strong> such as images, sounds...</p><p>The <a href="https://github.com/ArnaudBuchholz/bubu-timer" target="_blank">bubu-timer project</a> is based on WebPack.</p><h4>Transpilers</h4><p>Most <strong><a href="https://scotch.io/tutorials/javascript-transpilers-what-they-are-why-we-need-them" target="_blank">transpilers</a> are also bundlers</strong>:</p><ul><li><a href="https://babeljs.io/" target="_blank">Babel</a> which understands <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/New_in_JavaScript/ECMAScript_2015_support_in_Mozilla" target="_blank">ES6</a></li><li><a href="http://coffeescript.org/" target="_blank">CoffeeScript</a>, a sort of intermediate language compiled into JavaScript</li><li><a href="http://www.typescriptlang.org/" target="_blank">Typescript</a>, which can be <em>extremely summarized</em> by a JavaScript superset with type checking</li></ul><p>If one plan to use one or the other, the following articles might help:</p><ul><li><a href="http://blog.namangoel.com/browserify-vs-webpack-js-drama" target="_blank">Browserify VS Webpack - JS Drama</a></li><li><a href="https://www.sitepoint.com/javascript-modules-bundling-transpiling/" target="_blank">Understanding JavaScript Modules: Bundling & Transpiling</a></li></ul><h3>Windows Scripting File</h3><p>Microsoft's <a href="https://technet.microsoft.com/en-us/library/bb490887.aspx" target="_blank">cscript</a> and wscript command lines offer a simple and a bit obsolete but yet interesting scripting host enabling the use of <a href="https://msdn.microsoft.com/fr-fr/library/7sw4ddf8%28v=vs.94%29.aspx" target="_blank">ActiveX objects</a>.</p><div class="note"><p> I plan to document some valuable examples coming from past experiences demonstrating how one can send an outlook meeting request or extract host information using <a href="https://msdn.microsoft.com/en-us/library/aa394582%28v=vs.85%29.aspx" target="_blank">Windows Management Instrumentation</a></p></div><p>It can run any JavaScript file (extension .js) provided the system is properly configured or by setting the engine with the <a href="https://technet.microsoft.com/en-us/library/bb490816.aspx" target="_blank">engine selector option /E</a>. In <a href="https://github.com/ArnaudBuchholz/gpf-js/" target="_blank">GPF-JS</a>, <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.html#.hosts" target="_blank">wscript</a> testing is <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/bd25aadb73e83c0dd62940dbe7caae8138ca0a2f/grunt/exec.js#L94" target="_blank">configured that way</a>.</p><h4>Manual loading</h4><p>This <strong>host does not provide any standardized modularization helper for JavaScript files</strong>. One common workaround consists in <strong>loading and evaluating</strong> a file with the help of the <a href="https://msdn.microsoft.com/en-us/library/aa242706%28v=vs.60%29.aspx" target="_blank">FileSystem object</a> and eval (which is <a href="https://javascriptweblog.wordpress.com/2010/04/19/how-evil-is-eval/" target="_blank">evil</a>):</p><p><code class="javascript gpf-blog"><span class="keyword">var</span><span class="space"> </span><span class="identifier">fso</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">ActiveXObject</span><span class="symbol">(</span><span class="string">"Scripting.FileSystemObject"</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">srcFile</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">OpenTextFile</span><span class="symbol">(</span><span class="string">"dependency.js"</span><span class="symbol">,</span><span class="number">1</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">srcContent</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">srcFile</span><span class="symbol">.</span><span class="identifier">ReadAll</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">srcFile</span><span class="symbol">.</span><span class="identifier">Close</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">eval</span><span class="symbol">(</span><span class="identifier">srcContent</span><span class="symbol">)</span><span class="symbol">;</span></code></p><p>Which can be put in a function or summarized in one ugly line: <code class="javascript gpf-blog"><span class="identifier">eval</span><span class="symbol">(</span><span class="symbol">(</span><span class="keyword">new</span><span class="space"> </span><span class="identifier">ActiveXObject</span><span class="symbol">(</span><span class="string">"Scripting.FileSystemObject"</span><span class="symbol">)</span><span class="symbol">)</span><span class="symbol">.</span><span class="identifier">OpenTextFile</span><span class="symbol">(</span><span class="string">"dependency.js"</span><span class="symbol">,</span><span class="number">1</span><span class="symbol">)</span><span class="symbol">.</span><span class="identifier">ReadAll</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">)</span><span class="symbol">;</span></code></p><p>With this pattern, <strong>loading is synchronous and scope is global</strong> <em>(because of eval)</em>.</p><h4>Windows Scripting File</h4><p>Microsoft designed the <a href="https://en.wikipedia.org/wiki/Windows_Script_File" target="_blank">Windows Scripting File</a> format that allows:</p><ul><li>Language mixing, for instance VBA with JavaScript <em>(but we are only interested by JavaScript, aren't we?)</em>
</li><li>References to component or external files</li></ul><p>This is illustrated in <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-LOADING.html" target="_blank">Loading GPF Library</a> tutorial.</p><h3>Rhino</h3><p>Almost like Microsoft's scripting hosts, <a href="https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Rhino" target="_blank">Rhino (Java's JavaScript host)</a> does not provide any modularization helper. However, the shell offers the <a href="https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Rhino/Shell" target="_blank">load method</a> which <strong>loads and evaluates</strong> JavaScript files. In that context too, <strong>loading is synchronous and scope is global</strong>.</p><h3>Javascript's native import</h3><p><a href="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/if-the-javascript-language-could-stop-inventing-new-keywords-thatd-be-great.jpg" target="_blank">Here comes a new keyword<br><img src="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/if-the-javascript-language-could-stop-inventing-new-keywords-thatd-be-great.jpg" alt="Here comes a new keyword" title="Here comes a new keyword" border="0" style="max-width: 100%"></a></p><p>There is no way this article could talk about JavaScript modularization without mentioning the latest EcmaScript's <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import" target="_blank">import feature</a>.</p><h4>Static import</h4><p>When the name of the imported module is static (i.e. a string), the mechanism is called <strong>static import</strong>. It is <strong>synchronous</strong> and the developer decides what is imported. It also means that the module <strong>exposes an interface</strong> from a <strong>private scope</strong>.</p><p>For instance: <code class="javascript gpf-blog"><span class="identifier">import</span><span class="space"> </span><span class="symbol">{</span><span class="space"> </span><span class="identifier">myExportedFunction</span><span class="space"> </span><span class="symbol">}</span><span class="space"> </span><span class="identifier">from</span><span class="space"> </span><span class="string">"dependency"</span><span class="symbol">;</span></code></p><p>When the file <em>dependency.js</em> contains: <code class="javascript gpf-blog"><span class="string">"use strict"</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// Begin of private scope</span><span class="space">
</span><span class="comment">// Any functions or variables declared here are 'private'</span><span class="space">
</span><span class="comment">// Reveal function</span><span class="space">
</span><span class="identifier">export</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="identifier">myExportedFunction</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">/* ...implementation... */</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="comment">// End of private scope</span></code></p><p>On one hand, <strong>static import is fully supported by <a href="https://nodejs.org/api/esm.html#esm_notable_differences_between_import_and_require" target="_blank">NodeJS</a>, transpilers and bundlers</strong>. Regarding browsers, <strong>native implementation</strong> are on <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import#AutoCompatibilityTable" target="_blank">their way</a>.</p><p>On the other hand, Rhino and WScript do not support it. PhantomJS may support it in <a href="https://github.com/ariya/phantomjs/issues/14506" target="_blank">version 2.5</a>.</p><h4>Dynamic import</h4><p>When the name of the imported module is built from variables or function calls, the mechanism is called <strong>dynamic import</strong>. As the name of the module to import is <strong>known at execution time</strong>, this generates some interesting challenges for the hosts and tools.</p><p>There are several proposals for implementation:</p><ul><li><a href="http://2ality.com/2017/01/import-operator.html" target="_blank">ES proposal: import() – dynamically importing ES modules</a></li><li><a href="https://github.com/tc39/proposal-dynamic-import" target="_blank">tc39/proposal-dynamic-import</a></li></ul><p>And it has recently become a <a href="https://developers.google.com/web/updates/2017/11/dynamic-import" target="_blank">reality in Chrome and Safari</a>.</p><h2>GPF-JS Implementation</h2><p><a href="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/mine%20mine.jpg" target="_blank">My own implementation<br><img src="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/mine%20mine.jpg" alt="My own implementation" title="My own implementation" border="0" style="max-width: 100%"></a></p><h3>API</h3><p>The new namespace <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.require.html" target="_blank">gpf.require</a> exposes a <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-REQUIRE.html" target="_blank">modularization helper</a> which entry point is <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.require.html#.define__anchor" target="_blank">gpf.require.define</a>.</p><p>The main characteristics are:</p><ul><li>It is working the same way on <strong><a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.html#.hosts" target="_blank">all hosts</a></strong></li><li>Loading is <strong>asynchronous</strong></li><li>Module scope is <strong>private</strong> (because of the pattern leveraging factory functions)</li><li><strong>API exposure</strong> is done by <strong>returning an object</strong></li></ul><p>But the API also offers:</p><ul><li><strong>caching</strong> of loaded modules</li><li><strong>cache manipulation</strong> to simplify testing by injecting mocked dependencies</li><li><strong>synchronization</strong> through a <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise" target="_blank">Promise</a> resolved when all dependencies are loaded and the module is evaluated (or rejected if any issue occurred)</li><li><strong>lazy loading</strong> as it may be called anytime</li><li><strong>resource types</strong>, dependent files may be <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON" target="_blank">JSON files</a> or JavaScript modules</li></ul><p>When writing a JavaScript module, several syntaxes are supported:</p><ul><li>The 'GPF' way:</li></ul><p><code class="javascript gpf-blog"><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">require</span><span class="symbol">.</span><span class="identifier">define</span><span class="symbol">(</span><span class="symbol">{</span><span class="space">
</span><span class="identifier">name1</span><span class="symbol">:</span><span class="space"> </span><span class="string">"dependency1.js"</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">name2</span><span class="symbol">:</span><span class="space"> </span><span class="string">"dependency2.json"</span><span class="symbol">,</span><span class="space">
</span><span class="comment">// ...</span><span class="space">
</span><span class="identifier">nameN</span><span class="symbol">:</span><span class="space"> </span><span class="string">"dependencyN.js"</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">,</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">require</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="string">"use strict"</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// Private scope</span><span class="space">
</span><span class="identifier">require</span><span class="symbol">.</span><span class="identifier">name1</span><span class="symbol">.</span><span class="identifier">api1</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// Implementation</span><span class="space">
</span><span class="comment">// Interface</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">api1</span><span class="symbol">:</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="comment">/*...*/</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">/*...*/</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">,</span><span class="space">
</span><span class="comment">// ...</span><span class="space">
</span><span class="identifier">apiN</span><span class="symbol">:</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="comment">/*...*/</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">/*...*/</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span></code></p><ul><li>Using <a href="https://en.wikipedia.org/wiki/Asynchronous_module_definition" target="_blank">AMD</a> format:</li></ul><p><code class="javascript gpf-blog"><span class="identifier">define</span><span class="symbol">(</span><span class="symbol">[</span><span class="space">
</span><span class="string">"dependency1.js"</span><span class="symbol">,</span><span class="space">
</span><span class="string">"dependency2.json"</span><span class="symbol">,</span><span class="space">
</span><span class="comment">/*...*/</span><span class="symbol">,</span><span class="space">
</span><span class="string">"dependencyN.js"</span><span class="space">
</span><span class="symbol">]</span><span class="symbol">,</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">name1</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">name2</span><span class="symbol">,</span><span class="space"> </span><span class="comment">/*...*/</span><span class="space"> </span><span class="identifier">nameN</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="string">"use strict"</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// Private scope</span><span class="space">
</span><span class="identifier">name1</span><span class="symbol">.</span><span class="identifier">api1</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// Implementation</span><span class="space">
</span><span class="comment">// Interface</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">api1</span><span class="symbol">:</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="comment">/*...*/</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">/*...*/</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">,</span><span class="space">
</span><span class="comment">// ...</span><span class="space">
</span><span class="identifier">apiN</span><span class="symbol">:</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="comment">/*...*/</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">/*...*/</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span></code></p><ul><li>Using <a href="https://en.wikipedia.org/wiki/CommonJS" target="_blank">CommonJS</a> format:</li></ul><p><code class="javascript gpf-blog"><span class="string">"use strict"</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// Private scope</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">name1</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">require</span><span class="symbol">(</span><span class="string">"dependency1.js"</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">name2</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">require</span><span class="symbol">(</span><span class="string">"dependency2.json"</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="comment">// ...</span><span class="space">
</span><span class="identifier">nameN</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">require</span><span class="symbol">(</span><span class="string">"dependencyN.js"</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">name1</span><span class="symbol">.</span><span class="identifier">api1</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// Implementation</span><span class="space">
</span><span class="comment">// Interface</span><span class="space">
</span><span class="identifier">module</span><span class="symbol">.</span><span class="identifier">exports</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">api1</span><span class="symbol">:</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="comment">/*...*/</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">/*...*/</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">,</span><span class="space">
</span><span class="comment">// ...</span><span class="space">
</span><span class="identifier">apiN</span><span class="symbol">:</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="comment">/*...*/</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">/*...*/</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">;</span></code></p><p>For this last format, the module names must be static and modules are always evaluated <em>(reasons below)</em>.</p><div class="note"><p> Support of these formats was implemented for two reasons: it was an interesting challenge to handle all of them but it also offers interoperability between hosts. Indeed, some NodeJS modules are simple enough to be reused in Rhino, WScript or browsers without additional cost.</p></div><h3>Implementation details</h3><p>The implementation is composed of several files:</p><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require.js" target="_blank">src/require.js</a>: it contains the namespace declaration, the main entry points as well as the cache handling</li></ul><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require/load.js" target="_blank">src/require/load.js</a>: it manages resource loading</li></ul><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require/json.js" target="_blank">src/require/json.js</a>: it handles JSON resources</li></ul><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require/javascript.js" target="_blank">src/require/javascript.js</a>: it handles JavaScript resources</li></ul><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require/wrap.js" target="_blank">src/require/wrap.js</a>: it contains a special mechanism that was required to synchronize loading of subsequent JavaScript modules <em>(will be detailed later)</em>
</li></ul><p>But instead of going through each file one by one, the algorithm will be explained with examples.</p><h4>A simple example</h4><p>Let consider the following pair of files <em>(in the same folder)</em>:</p><ul><li>sample1.js</li></ul><p><code class="javascript gpf-blog"><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">require</span><span class="symbol">.</span><span class="identifier">define</span><span class="symbol">(</span><span class="symbol">{</span><span class="space">
</span><span class="identifier">dep1</span><span class="symbol">:</span><span class="space"> </span><span class="string">"dependency1.json"</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">,</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">require</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">alert</span><span class="symbol">(</span><span class="identifier">dep1</span><span class="symbol">.</span><span class="identifier">value</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span></code></p><ul><li>dependency1.json</li></ul><p><code class="javascript gpf-blog"><span class="symbol">{</span><span class="space">
</span><span class="string">"value"</span><span class="symbol">:</span><span class="space"> </span><span class="string">"Hello World!"</span><span class="space">
</span><span class="symbol">}</span></code></p><p>During the library startup, the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.require.html#.define__anchor" target="_blank">gpf.require.define</a> method is <strong><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require.js#L199" target="_blank">allocated</a></strong> using an <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require.js#L187" target="_blank">internal function</a>.</p><p>Indeed, the three main entry point (<a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.require.html#.define__anchor" target="_blank">define</a>, <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.require.html#.resolve__anchor" target="_blank">resolve</a> and <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.require.html#.configure__anchor" target="_blank">configure</a>) are internally <strong>bound to an hidden context</strong> object storing the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.typedef.html#.requireOptions" target="_blank">current configuration</a>.</p><p>When calling gpf.require.define, the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require.js#L155" target="_blank">implementation</a> starts by enumerating the dependencies and it builds <strong>an array of promises</strong>. Each promise will eventually be <strong>resolved to the corresponding resource's content</strong>:</p><ul><li>The exposed API for a JavaScript module</li><li>The JavaScript object for a JSON file</li></ul><p>Hence, this function:</p><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require.js#L118" target="_blank">Resolves</a> each dependency absolute path. This is done by simply <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.path.html#.join__anchor" target="_blank">concatenating</a> the <strong>resource name with the contextual base</strong>.</li></ul><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require.js#L129" target="_blank">Gets or associates</a> a promise with the resource. Using the <strong>absolute resource path as a key</strong>, it checks if the resource's promise is already in the cache. If not, the resource is <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require.js#L135" target="_blank">loaded</a>. <em>This will be detailed later</em>.</li></ul><p>In the above example, the array will contain only one promise corresponding to the resource "dependency1.json".</p><p>Once <strong>all promises are resolved</strong>, if the second parameter of gpf.require.define is a function (also called <strong>factory function</strong>), it is <strong><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require.js#L171" target="_blank">executed</a></strong> with a dictionary indexing <strong>all dependencies by their name</strong>.</p><div class="note"><p> <a href="https://en.wikipedia.org/wiki/Asynchronous_module_definition" target="_blank">AMD</a> format passes all dependencies as separate parameters. As a consequence, the factory function often bypass the maximum number of parameters that is specified in the linter. A dictionary has two advantages: it needs only one parameter and the API remains open for future additional parameters.</p></div><h4>Loading resources</h4><p>Whatever its type, loading a resource <strong>always starts by reading the file content</strong>. To make it <strong>host independent</strong>, <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require/load.js#L38" target="_blank">the implementation</a> switches between:</p><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/http.js#L54" target="_blank">gpf.http.request's internal implementation</a> when the host is a browser</li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/fs.js#L100" target="_blank">an internal method</a> which reads any file content for other hosts (implementations: <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/fs/nodejs.js#L217" target="_blank">NodeJS</a>, <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/fs/phantomjs.js#L12" target="_blank">PhantomJS</a>, <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/fs/rhino.js#L14" target="_blank">Rhino</a>, <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/fs/wscript.js#L198" target="_blank">WScript</a>)</li></ul><p>Then, based on the resource extension, a <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require/load.js#L63" target="_blank">content processor</a> is triggered. It is reponsible of converting the textual content into a result that will resolve the resource promise.</p><p>For the above example, the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require/json.js#L10" target="_blank">JSON file is parsed</a> and the promise is resolved to the resulting object.</p><div class="note"><p> Note how error processing is simplified by having this code encapsulated in promises.</p></div><h4>Subsequent dependencies</h4><p>Now let consider the following files <em>(sharing the same base folder)</em>:</p><ul><li>sample2.js</li></ul><p><code class="javascript gpf-blog"><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">require</span><span class="symbol">.</span><span class="identifier">define</span><span class="symbol">(</span><span class="symbol">{</span><span class="space">
</span><span class="identifier">dep2</span><span class="symbol">:</span><span class="space"> </span><span class="string">"sub/dependency2.js"</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">,</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">require</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">alert</span><span class="symbol">(</span><span class="identifier">dep2</span><span class="symbol">.</span><span class="identifier">value</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span></code></p><ul><li>sub/dependency2.js</li></ul><p><code class="javascript gpf-blog"><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">require</span><span class="symbol">.</span><span class="identifier">define</span><span class="symbol">(</span><span class="symbol">{</span><span class="space">
</span><span class="identifier">dep2</span><span class="symbol">:</span><span class="space"> </span><span class="string">"dependency2.json"</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">,</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">require</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">value</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">dep2</span><span class="symbol">.</span><span class="identifier">value</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span></code></p><ul><li>sub/dependency2.json</li></ul><p><code class="javascript gpf-blog"><span class="symbol">{</span><span class="space">
</span><span class="string">"value"</span><span class="symbol">:</span><span class="space"> </span><span class="string">"Hello World!"</span><span class="space">
</span><span class="symbol">}</span></code></p><div class="note"><p> When writing this article, I realized that 'simple' JavaScript modules exposing APIs (i.e. CommonJS syntax with no require but module.exports) are not properly handled. An <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/219" target="_blank">incident</a> was created in the next <a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/12" target="_blank">release</a>.</p></div><p>This last example illustrates several challenges:</p><ul><li>JavaScript modules loading</li><li>Relative path management</li><li>Subsequent dependencies must be loaded before resolving the promise associated to the module</li></ul><p>Let consider the first point: how does the API loads JavaScript modules?</p><p>As explained previously, the resource extension is used to determine which content processor is applied. In that case, the <strong><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require/javascript.js" target="_blank">JavaScript
processor</a></strong> is selected and executed.</p><p>To distinguish which syntax is being used in the module, the <em>v0.2.2</em> algorithm <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require/javascript.js#L114-L115" target="_blank">looks for the
CommonJS' require keyword</a>. Indeed, CommonJS modules are synchronous and, because of that, the dependencies must be <strong>preloaded before evaluating the module</strong>.</p><div class="note"><p> Back to the previous comment, this distinction might not be necessary anymore, the code will probably evolve in a future version. That's why no additional explanation will be given in this article.</p></div><p>In the above example, no require keyword can be found, hence the generic JavaScript processor applies.</p><p>Before <strong>evaluating</strong> the loaded JavaScript module, the content is <strong><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require/javascript.js#L106" target="_blank">wrapped</a> in a new function</strong> taking two parameters:</p><p><code class="javascript gpf-blog"><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">gpf</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">define</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">require</span><span class="symbol">.</span><span class="identifier">define</span><span class="symbol">(</span><span class="symbol">{</span><span class="space">
</span><span class="identifier">dep2</span><span class="symbol">:</span><span class="space"> </span><span class="string">"dependency2.json"</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">,</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">require</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">value</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">dep2</span><span class="symbol">.</span><span class="identifier">value</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span></code></p><p>This mechanism has several advantages:</p><ul><li>The process of creating this function <strong>compiles the JavaScript content</strong>. This won't detect runtime errors, such as the use of unknown symbols, but it validates the syntax.</li></ul><ul><li>The <strong>function wrapping pattern generates a private scope</strong> in which the code will be evaluated. Hence, variable and function declarations remain <strong>local to the module</strong>. Indeed, unlike <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/eval" target="_blank">eval</a> which can alter the global context, the <strong>function evaluation has a limited footprint</strong>.</li></ul><div class="note"><p> It is still possible to declare global symbols by accessing the global context (such as the <a href="https://developer.mozilla.org/en-US/docs/Web/API/Window" target="_blank">window
object</a> for browsers) but a simple way to avoid this is to redefine the window symbol as a parameter of the function and pass an empty object when executing it.</p></div><ul><li>It allows the <strong>definition (or override) and injection of 'global' symbols</strong>.</li></ul><p>In our case, two of them are being defined:</p><ul><li><strong>define</strong> is an <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require/javascript.js#L94-L103" target="_blank">encapsulation of the
gpf.require.define API</a> adapted to the <a href="https://en.wikipedia.org/wiki/Asynchronous_module_definition" target="_blank">AMD syntax</a></li></ul><ul><li><strong>gpf</strong> is a modified clone of the gpf namespace where the <strong>require member is <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require/javascript.js#L112" target="_blank">re-allocated</a></strong>.</li></ul><p>This last point is important to solve the two remaining questions:</p><ul><li>How does the library handles relative path?</li><li>How does it know when the subsequent dependencies are loaded?</li></ul><p>Everything is done inside the last <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require/wrap.js" target="_blank">uncovered source
file</a>.</p><p>A so-called wrapper is created to:</p><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require/wrap.js#L53-L55" target="_blank">Change</a> the settings of the require API by taking into account <strong>the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require/wrap.js#L54" target="_blank">current
path</a> of the resource being loaded</strong></li></ul><ul><li><strong><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require/wrap.js#L27-L35" target="_blank">Capture</a> the result promise of gpf.require.define</strong> if used.</li></ul><p>As a consequence, the API can <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.2/src/require/javascript.js#L126" target="_blank">return a
promise</a> that is either immediately resolved or the one <strong>corresponding to the inner gpf.require.define use</strong>, ensuring the module is <strong>properly loaded</strong>.</p><h2>Future of gpf.require</h2><p><a href="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/the%20future.jpg" target="_blank">A bright future<br><img src="https://arnaudbuchholz.github.io/blog/post/My%20own%20require%20implementation/the%20future.jpg" alt="A bright future" title="A bright future" border="0" style="max-width: 100%"></a></p><p>I would not pretend that the implementation is perfect but, from an API standpoint, it opens the door for interesting evolutions.</p><h3>Custom resource processor</h3><p>Right now, only json and js files are being handled. Everything else is considered text resources (i.e. the loaded content is returned without any additional processing).</p><p>Offering the possibility to <strong>map an extension to a custom resource processor</strong> (such as yml or jsx) would add significant value to the library.</p><h3>Handling cross references</h3><p>What happens when <strong>resources are inter-dependent</strong>? It should never be the case but, when building the modules, the developer may fall in the trap of cross references.</p><p>For instance, module A depends on module B, B depends on C, C depends on D and D depends on A!</p><p>There are easy patterns to work around this situation but, at least, the library <strong>must detect and deliver the proper information to the developer to help him solve the issue</strong>.</p><p>Also, the concept of <strong>weak references</strong> could be implemented: these are <strong>dependencies that are resolved later than the module itself</strong>.</p><p>For example <em>(and this is just an idea)</em>, the dependency is flagged with weak=true. This means that it is not required to execute the factory function. To know when the dependency is available (and access it), the member is no more the resource result but rather a promise resolved to the resource result once it has been loaded and processed.</p><p><code class="javascript gpf-blog"><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">require</span><span class="symbol">.</span><span class="identifier">define</span><span class="symbol">(</span><span class="symbol">{</span><span class="space">
</span><span class="identifier">dependency</span><span class="symbol">:</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">path</span><span class="symbol">:</span><span class="space"> </span><span class="string">"dependency.js"</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">weak</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">true</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">,</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">require</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">exposedApi</span><span class="symbol">:</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">require</span><span class="symbol">.</span><span class="identifier">dependency</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">resolvedDependency</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">resolvedDependency</span><span class="symbol">.</span><span class="identifier">api</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span></code></p><h3>Module object</h3><p>NodeJS proposes a lot of <a href="https://nodejs.org/docs/latest/api/modules.html" target="_blank">additional properties</a> on the module symbol. Some of them could be valuable in the library too.</p><h2>Conclusion</h2><p>If you have reached this conclusion: congratulations and thank you for your patience!</p><p>This article is probably one of the longest I ever wrote on this blog: almost two month of work was required. It clearly delayed the library development but, as usual, it made me realize some weaknesses of the current implementation.</p><p>I highly recommend this exercise to anybody who is developing some code: write an article on how it works and why it works that way.</p><p>It was important to illustrate how modularization helps building better applications before digging into the API proposition. The same way, because the GPF-JS library is compatible with several host, it was important to highlight existing solutions as they inspired me.</p><p>I am looking for feedback so don't hesitate to drop a comment or send me an <a href="mailto://arnaud.buchholz@free.fr?subject=[GPF-JS]%20My%20own%20require%20implementation" target="_blank">email</a>. I would be more than happy to discover your thoughts and discuss the implementation.</p></div>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-73993459870588944422017-12-21T05:58:00.001+01:002018-12-07T19:49:28.436+01:00Release 0.2.3: Streams and Pipes<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/release.png" align="left">
This new release focuses on streams by offering two new tools: a way to pipe streams together (and simplify their
use) and a CSV parser.
</div>
<div class="code markdown"><h2>New version</h2><p>Just before Christmas vacations, here comes the new version:</p><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/11?closed=1" target="_blank">Stories and bugs</a> implemented</li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/tree/v0.2.3" target="_blank">Sources</a></li><li><a href="https://arnaudbuchholz.github.io/gpf/doc/index.html" target="_blank">Documentation</a></li><li><a href="https://www.npmjs.com/package/gpf-js" target="_blank">NPM package</a></li></ul><h2>Release content</h2><h3>Cleaning</h3><p>A significant effort was put in cleaning the content of the <a href="https://github.com/ArnaudBuchholz/ArnaudBuchholz.github.io/tree/master/gpf" target="_blank">publication repository</a> as it previously contained <a href="https://arnaudbuchholz.github.io/gpf/plato/index.html" target="_blank">one plato report folder</a> as well as <a href="https://arnaudbuchholz.github.io/gpf/doc/index.html" target="_blank">one documentation folder</a> for each version.</p><p>Now everything is consolidated in a single folder and the plato report also includes file history (for instance: <a href="https://arnaudbuchholz.github.io/gpf/plato/files/src_regexp_js/index.html" target="_blank">regexp.js</a>).</p><p>All existing links are adjusted.</p><h3>gpf.stream.pipe</h3><p>Combining streams can become tedious when more than two streams must be plugged together. This is greatly simplified using the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.stream.html#.pipe__anchor" target="_blank">gpf.stream.pipe</a> helper.</p><p>It takes care of chaining the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.interfaces.IReadableStream.html" target="_blank">read</a> / <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.interfaces.IWritableStream.html" target="_blank">write</a> methods and returns a <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise" target="_blank">Promise</a> that is resolved when all the data has been processed (including <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.interfaces.IFlushableStream.html" target="_blank">flushable</a> streams).</p><p>For example:</p><p><code class="javascript gpf-blog"><span class="comment">// Reading a CSV file</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">csvFile</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">fs</span><span class="symbol">.</span><span class="identifier">getFileStorage</span><span class="symbol">(</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">openTextStream</span><span class="symbol">(</span><span class="string">"file.csv"</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">fs</span><span class="symbol">.</span><span class="identifier">openFor</span><span class="symbol">.</span><span class="identifier">reading</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">lineAdapter</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">LineAdapter</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">csvParser</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">csv</span><span class="symbol">.</span><span class="identifier">Parser</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">output</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">WritableArray</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// csvFile -> lineAdapter -> csvParser -> output</span><span class="space">
</span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">pipe</span><span class="symbol">(</span><span class="identifier">csvFile</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">lineAdapter</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">csvParser</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">output</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">output</span><span class="symbol">.</span><span class="identifier">toArray</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">records</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">// process records</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span></code></p><h3>gpf.stream.csv.Parser</h3><p>As demonstrated in the previous example, CSV parsing is now possible using the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.stream.csv.Parser.html" target="_blank">gpf.stream.csv.Parser</a> class.</p><h2>Lessons learned</h2><p>There are still some doubts regarding the mechanics of the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.3/src/stream/pipe.js" target="_blank">gpf.stream.pipe</a> implementation.</p><p>In particular, if an intermediate read completes before the whole stream has been processed, a new read must be triggered. Also, error handling is unclear even if <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.3/test/stream/pipe.js" target="_blank">fully tested</a>. Finally, flushing a stream does not necessarily means its end. Hence it should be possible to continue writing after a flush.</p><p>As these behaviors are not yet clearly defined, <strong>the code might require adjustments in the future</strong>.</p><p>In any case, it demonstrates that some 'real life' examples are required to see how this could simplify (or not) the development.</p><h2>Next release</h2><p>The <a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/12" target="_blank">next release content</a> will mostly focus on:</p><ul><li>Finishing the release process improvements</li><li>Introducing attributes that will be later used to simplify CSV generation</li><li>Adding new stream helpers to filter / page batch of records</li></ul><p>In parallel, the article on <a href="https://gpf-js.blogspot.ca/2017/11/release-022-gpfrequire.html" target="_blank">gpf.require.define</a> implementation is still in progress and some additional side projects are popping up.</p></div>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-85270150791264084232017-11-03T23:00:00.000+01:002018-12-07T19:49:06.893+01:00Release 0.2.2: gpf.require<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/release.png" align="left">
This new release includes a modularization helper as well as feedback from a side project.
It also includes fixes for some annoying bugs and quality improvement.
</div>
<div class="code markdown"><h2>New version</h2><p>Here comes the new version:</p><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/10?closed=1" target="_blank">Stories and bugs</a> implemented</li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/tree/v0.2.2" target="_blank">Sources</a></li><li><a href="https://arnaudbuchholz.github.io/gpf/doc/index.html" target="_blank">Documentation</a></li><li><a href="https://www.npmjs.com/package/gpf-js" target="_blank">NPM package</a></li></ul><h3>Why did it take so long to get a new version?</h3><p>First, no timeline is clearly defined for the project. Usually, a release spans over one month and an implicit commitment is made. However, the necessary time is spent to make sure that a release contains everything needed.</p><p>In that case, there are three main reasons to explain the delay.</p><ul><li>The first one is <a href="https://arnaudbuchholz.github.io/blog/post/Release%200.2.2/vacations.jpg" target="_blank">vacations<br><img src="https://arnaudbuchholz.github.io/blog/post/Release%200.2.2/vacations.jpg" alt="vacations" title="vacations" border="0" style="max-width: 100%"></a></li></ul><ul><li>The second reason is that a <a href="https://github.com/ArnaudBuchholz/bubu-timer" target="_blank">side project</a> dragged a lot of my time. I will later write an article about it as it was my first HTML mobile app experience with a tough customer... my wife.</li></ul></div><div align="center">
<iframe src="https://arnaudbuchholz.github.io/bubu-timer/index.html#about" width="400" height="400"></iframe>
</div><div class="code markdown"><ul><li>Finally, this release addresses an issue that appeared from time to time and that was never considered seriously. Some tests were failing on a regular basis but not frequently enough to represent a real threat. This problem was prioritized and tackled once for all but it brought many time-consuming challenges.</li></ul><p>Oh, and I also designed a <a href="https://arnaudbuchholz.github.io/gpf/gpf_320x200.svg" target="_blank">logo<br><img src="https://arnaudbuchholz.github.io/gpf/gpf_320x200.svg" alt="logo" title="logo" border="0" style="max-width: 100%"></a></p><h2>Release content</h2><h3>gpf.require</h3><p>This is clearly the new feature coming with this version. It allows the developer to modularize its project by creating separate source files that depend from each other. A separate article will be written to detail the concept (and implementation) but, in the meantime, you may read the <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-REQUIRE.html" target="_blank">tutorial</a> on how to use it.</p><p>To put it in a nutshell:</p><p><code class="javascript gpf-blog"><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">require</span><span class="symbol">.</span><span class="identifier">define</span><span class="symbol">(</span><span class="symbol">{</span><span class="space">
</span><span class="identifier">hello</span><span class="symbol">:</span><span class="space"> </span><span class="string">"hello.js"</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">,</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">require</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="string">"use strict"</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">console</span><span class="symbol">.</span><span class="identifier">log</span><span class="symbol">(</span><span class="identifier">require</span><span class="symbol">.</span><span class="identifier">hello</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">)</span><span class="space"> </span><span class="comment">// Output "World!"</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span></code></p><p>Provided the hello.js file is in the same folder and contains:</p><p><code class="javascript gpf-blog"><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">require</span><span class="symbol">.</span><span class="identifier">define</span><span class="symbol">(</span><span class="symbol">{</span><span class="symbol">}</span><span class="symbol">,</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="string">"use strict"</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="string">"World!"</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span></code></p><p>Or (using CommonJS syntax)... <code class="javascript gpf-blog"><span class="string">"use strict"</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">module</span><span class="symbol">.</span><span class="identifier">exports</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="string">"World!"</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">;</span></code></p><p>This mechanism was envisioned for a while but it was hard to figure out how (and where) to start. After reading some documentation about NodeJS and now that the library is mature enough, it appeared to be surprisingly easy to implement.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/Release%200.2.2/SANDCASTLES.png" target="_blank">I did it<br><img src="https://arnaudbuchholz.github.io/blog/post/Release%200.2.2/SANDCASTLES.png" alt="I did it" title="I did it" border="0" style="max-width: 100%"></a></p><p>Some of you may object that, indeed, it already exists in NodeJS and browsers (natively or through different libraries such as <a href="http://requirejs.org/" target="_blank">RequireJS</a> or <a href="http://browserify.org/" target="_blank">browserify</a>). But this new mechanism brings one uniform feature to all supported platforms.</p><p>And it comes with a way to alter the require cache to simplify testing by injecting <a href="https://en.wikipedia.org/wiki/Mock_object" target="_blank">mocked</a> dependencies.</p><h3>Improved gpf.web.createTagFunction</h3><p>To develop the <a href="https://arnaudbuchholz.github.io/bubu-timer/index.html#about" target="_blank">mobile app</a>, <a href="https://developer.mozilla.org/en-US/docs/Web/SVG" target="_blank">Scalable Vector Graphics</a> were extensively leveraged. In order to create the appropriate markup, some specific DOM APIs must be used (such as <a href="https://developer.mozilla.org/en-US/docs/Web/API/Document/createElementNS" target="_blank">createElementNS</a>). That's why the support of the predefined svg namespace was added to <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.web.html#.createTagFunction__anchor" target="_blank">gpf.web.createTagFunction</a>.</p><div class="note"><p> ... But the use of namespaces is not yet documented... an <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/210" target="_blank">incident</a> is assigned to the next release.</p></div><p>The same way, this method allocates a function that generates tags exposing methods (toString and appendTo). The documentation was missing but now this is <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.typedef.Tag.html" target="_blank">resolved</a>.</p><h3>Improved gpf.http layer</h3><p>The <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/HEAD" target="_blank">HEAD verb</a> was not supported by the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.http.html" target="_blank">HTTP layer</a>. It is now <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.http.html#.head__anchor" target="_blank">possible</a>.</p><p>Another improvement is the possibility to mock any HTTP requests by adding a <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.typedef.html#.mockedRequest" target="_blank">custom handler</a>. Using this feature, any code that involves HTTP communication can be tested independently from the web server.</p><p>For instance:</p><p><code class="javascript gpf-blog"><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">http</span><span class="symbol">.</span><span class="identifier">mock</span><span class="symbol">(</span><span class="symbol">{</span><span class="space">
</span><span class="identifier">method</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">http</span><span class="symbol">.</span><span class="identifier">methods</span><span class="symbol">.</span><span class="identifier">get</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">url</span><span class="symbol">:</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">RegExp</span><span class="symbol">(</span><span class="string">"echo\\?status=([0-9]+)"</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">response</span><span class="symbol">:</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">request</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">status</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">if</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">status</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="string">"400"</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="symbol">;</span><span class="space"> </span><span class="comment">// Don't mock</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">status</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">parseInt</span><span class="symbol">(</span><span class="identifier">status</span><span class="symbol">,</span><span class="space"> </span><span class="number">10</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">+</span><span class="space"> </span><span class="number">1</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">headers</span><span class="symbol">:</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="string">"x-mock"</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">true</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">responseText</span><span class="symbol">:</span><span class="space"> </span><span class="string">"It works"</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// No HTTP request will be sent to any server</span><span class="space">
</span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">http</span><span class="symbol">.</span><span class="identifier">get</span><span class="symbol">(</span><span class="string">"/echo?status=200"</span><span class="symbol">)</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">response</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">assert</span><span class="symbol">(</span><span class="identifier">response</span><span class="symbol">.</span><span class="identifier">status</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="number">201</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">assert</span><span class="symbol">(</span><span class="identifier">response</span><span class="symbol">.</span><span class="identifier">headers</span><span class="symbol">[</span><span class="string">"x-mock"</span><span class="symbol">]</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="identifier">true</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">assert</span><span class="symbol">(</span><span class="identifier">response</span><span class="symbol">.</span><span class="identifier">responseText</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="string">"It works"</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="comment">/*...*/</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// But this one will...</span><span class="space">
</span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">http</span><span class="symbol">.</span><span class="identifier">get</span><span class="symbol">(</span><span class="string">"/echo?status=400"</span><span class="symbol">)</span><span class="symbol">;</span></code></p><h3>Improved Timeout & Promise tests</h3><p>From time to time, with a ratio of 1 of 1000 executions, the WScript tests were failing. It took ages to narrow the problem down to the place where the assertion failed:</p><ul><li>An 'infinite performance' mode was added to the testing command line: it loops forever and shows mean as well as deviation on the execution time</li><li>Error occurring in the testing command line are now documented to help locating the issue</li></ul><p><a href="https://ArnaudBuchholz.github.io/blog/post/Release%200.2.2/timeout1.png" target="_blank">Failure example 1<br><img src="https://ArnaudBuchholz.github.io/blog/post/Release%200.2.2/timeout1.png" alt="Failure example 1" title="Failure example 1" border="0" style="max-width: 100%"></a></p><p><a href="https://ArnaudBuchholz.github.io/blog/post/Release%200.2.2/timeout2.png" target="_blank">Failure example 2<br><img src="https://ArnaudBuchholz.github.io/blog/post/Release%200.2.2/timeout2.png" alt="Failure example 2" title="Failure example 2" border="0" style="max-width: 100%"></a></p><div class="note"><p> This highlights the fact that tests should be small and clear enough to immediately spot a problem when it happens.</p></div><p>Upon qualification, it appeared that two test suites were badly designed:</p><ul><li>Promises testing was using timeouts for no good reasons. Removing them from the test suite was easy: <a href="https://github.com/ArnaudBuchholz/gpf-js/commit/5678f579b136eed6e17f8e5a24dd0dc710512e8f#diff-0efebaf31544407a2e23ae2c22c42478" target="_blank">diff</a></li></ul><ul><li>Timeout testing was based on global variables to assess several timeout executions and the data got corrupted depending on the executions sequences. Changing the test suite to secure it was more challenging: <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/dc472d0714d7e25e19ce975481962602e305a6ad/test/compatibility/timeout.js" target="_blank">before</a> <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/d20a1ea1239474aa37e09918ad077c393c39aaee/test/compatibility/timeout.js" target="_blank">after</a></li></ul><p>Once the whole thing was figured out and fixed, the problem of <a href="https://github.com/ArnaudBuchholz/gpf-js/tree/master/test/legacy" target="_blank">legacy tests</a> remained. Indeed, the test suites are saved after each release and they should remain untouched to ensure backward compatibility.</p><p>But then, how do you handle tests that were badly designed? You can't just drop the legacy test suites (more than 600 tests) just because some of them are invalid.</p><p><a href="https://ArnaudBuchholz.github.io/blog/post/Release%200.2.2/legacy.png" target="_blank">Legacy code<br><img src="https://ArnaudBuchholz.github.io/blog/post/Release%200.2.2/legacy.png" alt="Legacy code" title="Legacy code" border="0" style="max-width: 100%"></a></p><p>That's why the idea of a legacy management came out. With the help of the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/test/legacy/legacy.json" target="_blank">legacy.json</a> file, the library offers a way to disable some tests based on their name and version.</p><h3>Improved quality and tooling</h3><p>The minimum <a href="https://blogs.msdn.microsoft.com/codeanalysis/2007/11/20/maintainability-index-range-and-meaning/" target="_blank">maintainability ratio</a> has been <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/make/configFile.js#L26" target="_blank">increased to 70</a>, raising the overall quality requirement for source files.</p><p>The grunt connect middleware was <a href="https://github.com/ArnaudBuchholz/gpf-js/commit/1ad9f27e0a86231700a0546ad8165971acb80052#diff-9e8c92d72950a0def094554de6cd09efR54" target="_blank">modified</a> to automatically scroll the page content. This allowed the recording of this joyful video.</p></div><iframe width="560" height="315" src="https://www.youtube.com/embed/3qU_-wFNdYk" frameborder="0" allowfullscreen="">
</iframe><div class="code markdown"><h2>Lessons learned</h2><ul><li>Almost everything can be implemented in oldest JavaScript hosts. This demonstrates the power and flexibility of the language.</li></ul><ul><li>The way tests are written today significantly impacts tomorrow versions. This is something to keep in mind.</li></ul><ul><li>Documentation must be reviewed before releasing. However, it becomes more and more complex and I am not necessarily the best person to review my own writing.</li></ul><h2>Next release</h2><p>The <a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/11" target="_blank">next release content</a> will mostly focus on:</p><ul><li>Improving the streams formalism and implement an CSV from/to records engine</li><li>Improving the release process</li><li>Doing some cleaning</li></ul></div>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-50119971017523130242017-11-01T22:54:00.001+01:002017-11-01T22:55:27.733+01:00Released 0.2.2<div dir="ltr" style="text-align: left;" trbidi="on">
Just a quick note to announce that <a href="https://github.com/ArnaudBuchholz/gpf-js/releases/tag/v0.2.2" target="_blank">version 0.2.2</a> is released.<br />
I am now working on release notes and next release preparation.<br />
<br />
I captured the operation to illustrate how easy (and safe) the release process is.<br />
<br />
<iframe allowfullscreen="" frameborder="0" height="315" src="https://www.youtube.com/embed/77alWUGaPjg" width="560"></iframe>
</div>
Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-64375152258202754242017-10-14T21:25:00.001+02:002018-12-07T19:24:29.703+01:000.2.2 in progress<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/release.png" align="left">
This version 0.2.2 is still in progress. Here is a sneak peek of the build process.
</div>
<div class="code markdown"><p>When building the <a href="https://github.com/ArnaudBuchholz/gpf-js" target="_blank">GPF library</a>, it is tested with 4 command line hosts:</p><ul><li><a href="http://nodejs.org/" target="_blank">NodeJs</a></li></ul><ul><li><a href="http://technet.microsoft.com/en-us/library/bb490887.aspx" target="_blank">cscript/wscript</a></li></ul><ul><li><a href="https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Rhino" target="_blank">Rhino</a></li></ul><ul><li><a href="http://phantomjs.org/" target="_blank">PhantomJS</a></li></ul><p>And at least 3 browsers:</p><ul><li><a href="https://www.google.com/chrome/index.html" target="_blank">Chrome</a></li></ul><ul><li><a href="http://appldnld.apple.com/Safari5/041-5487.20120509.INU8B/SafariSetup.exe" target="_blank">Safari</a></li></ul><ul><li>Internet Explorer</li></ul><p>Backward compatibility with all previous versions is checked as well as maintainability ratio (using <a href="https://github.com/es-analysis/plato" target="_blank">Plato</a>) and coverage (using <a href="https://jasmine.github.io/" target="_blank">Jasmine</a>).</p><p>The library is published in three flavors:</p><ul><li>source (development version)</li></ul><ul><li>debug (concatenated sources)</li></ul><ul><li>release (an optimized and <a href="https://github.com/mishoo/UglifyJS2" target="_blank">uglify</a>-ed version of debug)</li></ul><p>The test library contains more than 600 unit tests covering almost 100% of the code (exceptions are <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-COVERAGE.html" target="_blank">documented</a>).</p><p>Last but not least, the code is <a href="https://eslint.org/" target="_blank">linted</a> and documentation is generated with <a href="http://usejsdoc.org/" target="_blank">jsdoc</a>.</p><p>This represents a total of 55 test executions.</p><p>Obviously, I am too lazy to do it all by myself: everything is automated with <a href="https://gruntjs.com/" target="_blank">Grunt</a>.</p><p>Today was a big day working on <a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/10" target="_blank">version 0.2.2</a> (still in progress): I am finalizing a feature that will help me better <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/168" target="_blank">modularize my code</a>.</p><p>I wanted to share those 2 minutes of pure development joy that concluded my work <em>(full screen recommended)</em>.</p></div><iframe width="560" height="315" src="https://www.youtube.com/embed/QmyPHTzuzYQ?rel=0" frameborder="0" allowfullscreen="">
</iframe><div class="code markdown"></div>
Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-86355326157494842612017-09-16T16:50:00.001+02:002018-12-07T19:56:46.151+01:00Sneaky JavaScript Technics IV<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/ninja.png" align="left">
A ninja is a lazy fighter. As Sun Tzu stated "Every battle is won or lost before it's ever fought".
Here are quick tips to simplify your development when dealing with optional parameters.
</div>
<div class="code markdown"><h3>Handling default parameters</h3><p>The are many way to default parameters when they are not passed during <a href="http://yehudakatz.com/2011/08/11/understanding-javascript-function-invocation-and-this/" target="_blank">function invocation</a>.</p><p><a href="http://es6-features.org/" target="_blank">ES6</a> proposes <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/Default_parameters" target="_blank">a syntax</a> to formalize them, you can <a href="https://babeljs.io/repl/" target="_blank">try it by yourself</a>.</p><p>The following example: <code class="javascript gpf-blog"><span class="keyword">function</span><span class="space"> </span><span class="identifier">test</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">firstParam</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="string">"default value"</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">firstParam</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span></code></p><p>is <a href="https://scotch.io/tutorials/javascript-transpilers-what-they-are-why-we-need-them" target="_blank">transpiled</a> into: <code class="javascript gpf-blog"><span class="string">"use strict"</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">function</span><span class="space"> </span><span class="identifier">test</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">firstParam</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">arguments</span><span class="symbol">.</span><span class="identifier">length</span><span class="space"> </span><span class="symbol">></span><span class="space"> </span><span class="number">0</span><span class="space"> </span><span class="symbol">&&</span><span class="space"> </span><span class="identifier">arguments</span><span class="symbol">[</span><span class="number">0</span><span class="symbol">]</span><span class="space"> </span><span class="symbol">!==</span><span class="space"> </span><span class="identifier">undefined</span><span class="space"> </span><span class="symbol">?</span><span class="space"> </span><span class="identifier">arguments</span><span class="symbol">[</span><span class="number">0</span><span class="symbol">]</span><span class="space"> </span><span class="symbol">:</span><span class="space"> </span><span class="string">"default value"</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">firstParam</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span></code></p><p>Indeed, the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/arguments" target="_blank">arguments</a> object contains the list of parameters that were passed during function invocation. It looks like an array (but it is not) and exposes length as well as all passed parameters.</p><p>This is quite complex and I usually go with:</p><p><code class="javascript gpf-blog"><span class="keyword">function</span><span class="space"> </span><span class="identifier">test</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">firstParam</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">if</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">firstParam</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="identifier">undefined</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">firstParam</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="string">"default value"</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">firstParam</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span></code></p><p>Or, shorter,</p><p><code class="javascript gpf-blog"><span class="keyword">function</span><span class="space"> </span><span class="identifier">test</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">firstParam</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">firstParam</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">firstParam</span><span class="space"> </span><span class="symbol">||</span><span class="space"> </span><span class="string">"default value"</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">firstParam</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span></code></p><div class="note"><p> This last syntax may lead to errors when dealing with <a href="https://developer.mozilla.org/en-US/docs/Glossary/Falsy" target="_blank">falsy values</a></p></div><p>All these non ES6 syntaxes are working fine but they come with two drawbacks:</p><ul><li>You have to manually handle the missing parameters</li><li>The condition adds <a href="https://en.wikipedia.org/wiki/Cyclomatic_complexity" target="_blank">cyclomatic complexity</a> to your function</li></ul><h3>The lazy solution</h3><p>Wouldn't it be nice to have a way to wrap a function so that default parameters would be handled without having to code anything?</p><p>Considering that optional parameters are usually at the end of a function, let's introduce a new method to the function object that will default its last parameters.</p><p>So considering this function: <code class="javascript gpf-blog"><span class="keyword">function</span><span class="space"> </span><span class="identifier">add</span><span class="symbol">(</span><span class="identifier">value</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">increment</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">value</span><span class="space"> </span><span class="symbol">+</span><span class="space"> </span><span class="identifier">increment</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span></code></p><p>We could define inc as: <code class="javascript gpf-blog"><span class="keyword">var</span><span class="space"> </span><span class="identifier">inc</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">add</span><span class="symbol">.</span><span class="identifier">withDefaults</span><span class="symbol">(</span><span class="number">1</span><span class="symbol">)</span><span class="symbol">;</span></code></p><p>That would be equivalent to the ES6 version of: <code class="javascript gpf-blog"><span class="keyword">function</span><span class="space"> </span><span class="identifier">inc</span><span class="symbol">(</span><span class="identifier">value</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">increment</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="number">1</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">add</span><span class="symbol">(</span><span class="identifier">value</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">increment</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span></code></p><h4>Forewords</h4><p>Before going straight to the proposal, you need to understand the following concepts:</p><ul><li>A function exposes the size of its signature through the property <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Function/length" target="_blank">length</a>: it allows any developer to know the number of expected parameters</li></ul><ul><li>The arguments object is not an array but it can be easily converted into one using the following pattern: <code class="javascript gpf-blog"><span class="symbol">[</span><span class="symbol">]</span><span class="symbol">.</span><span class="identifier">slice</span><span class="symbol">.</span><span class="identifier">call</span><span class="symbol">(</span><span class="identifier">arguments</span><span class="symbol">)</span></code></li></ul><ul><li>You can create an array of any size using <code class="javascript gpf-blog"><span class="keyword">new</span><span class="space"> </span><span class="identifier">Array</span><span class="symbol">(</span><span class="identifier">size</span><span class="symbol">)</span></code> It will be filled with <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/undefined" target="_blank">undefined</a> values</li></ul><ul><li>If the provided default parameters are not enough to set missing ones, they are replaced with <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/undefined" target="_blank">undefined</a> (as expected)</li></ul><ul><li>If you have read my other articles, you already know <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Function/apply" target="_blank">Fuction.prototype.apply</a>.</li></ul><p>Now you are ready.</p><h4>Proposal</h4><p>Here is the proposal to handle default values:</p><p><code class="javascript gpf-blog"><span class="identifier">Function</span><span class="symbol">.</span><span class="identifier">prototype</span><span class="symbol">.</span><span class="identifier">withDefaults</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">defaultParameters</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="symbol">[</span><span class="symbol">]</span><span class="symbol">.</span><span class="identifier">slice</span><span class="symbol">.</span><span class="identifier">call</span><span class="symbol">(</span><span class="identifier">arguments</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">wrappedFunction</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">this</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">receivedParameters</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="symbol">[</span><span class="symbol">]</span><span class="symbol">.</span><span class="identifier">slice</span><span class="symbol">.</span><span class="identifier">call</span><span class="symbol">(</span><span class="identifier">arguments</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">missingCount</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">wrappedFunction</span><span class="symbol">.</span><span class="identifier">length</span><span class="space"> </span><span class="symbol">-</span><span class="space"> </span><span class="identifier">receivedParameters</span><span class="symbol">.</span><span class="identifier">length</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">actualParameters</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">sliceFrom</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">if</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">missingCount</span><span class="space"> </span><span class="symbol">></span><span class="space"> </span><span class="number">0</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">sliceFrom</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">defaultParameters</span><span class="symbol">.</span><span class="identifier">length</span><span class="space"> </span><span class="symbol">-</span><span class="space"> </span><span class="identifier">missingCount</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">actualParameters</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">receivedParameters</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">concat</span><span class="symbol">(</span><span class="space">
</span><span class="keyword">new</span><span class="space"> </span><span class="identifier">Array</span><span class="symbol">(</span><span class="identifier">Math</span><span class="symbol">.</span><span class="identifier">max</span><span class="symbol">(</span><span class="symbol">-</span><span class="identifier">sliceFrom</span><span class="symbol">,</span><span class="space"> </span><span class="number">0</span><span class="symbol">)</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">defaultParameters</span><span class="symbol">.</span><span class="identifier">slice</span><span class="symbol">(</span><span class="identifier">Math</span><span class="symbol">.</span><span class="identifier">max</span><span class="symbol">(</span><span class="identifier">sliceFrom</span><span class="symbol">,</span><span class="space"> </span><span class="number">0</span><span class="symbol">)</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space"> </span><span class="keyword">else</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">actualParameters</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">receivedParameters</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">wrappedFunction</span><span class="symbol">.</span><span class="identifier">apply</span><span class="symbol">(</span><span class="keyword">this</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">actualParameters</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="symbol">}</span></code></p><p>As well as the associated <a href="https://jsfiddle.net/ArnaudBuchholz/4mdfn6qg/1/" target="_blank">test case</a>.</p><h4>Improvements</h4><p>This solution is not perfect. Indeed, the resulting function has a signature length of 0. Consequently, you can't chain with another call to default parameters of such a wrapped function.</p><div class="note"><p> There are several ways to work around this limitation but I wanted to keep this article short and simple</p></div></div>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-35990549120919152892017-06-21T03:22:00.001+02:002018-12-07T19:27:41.786+01:005 ways to make an http request<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/student.png" align="left">
The version 0.2.1 of the GPF-JS library delivers an HTTP request helper that can be used on all supported hosts.
It was quite a challenge as it implied 5 different developments, here are the details.
</div>
<div class="code markdown"><h2>The need for HTTP requests</h2><p>In a world of <a href="https://en.wikipedia.org/wiki/Interoperability" target="_blank">interoperability</a>, <a href="https://en.wikipedia.org/wiki/Internet_of_things" target="_blank">internet of things</a> and <a href="https://en.wikipedia.org/wiki/Microservices" target="_blank">microservices</a>, the - almost 30 years old - <a href="https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol" target="_blank">HTTP protocol</a> defines a communication foundation that is widely known and implemented.</p><p>Originally designed for human-to-machine communication, this protocol also supports machine-to-machine communication through standardized concepts and interfaces:</p><ul><li>Representational state transfer APIs (<a href="https://en.wikipedia.org/wiki/Representational_state_transfer" target="_blank">REST</a>)</li><li><a href="https://en.wikipedia.org/wiki/Web_service" target="_blank">Web services</a></li><li><a href="http://www.odata.org/" target="_blank">OData</a></li><li><a href="https://en.wikipedia.org/wiki/JSONP" target="_blank">JSONP</a> (or <a href="https://en.wikipedia.org/wiki/Cross-origin_resource_sharing" target="_blank">CORS</a>)</li><li>...</li></ul><h3>Evolution of HTTP requests in browsers</h3><p><a href="https://en.wikipedia.org/wiki/Web_browser" target="_blank">Web browsers</a> were the first applications implementing this protocol to access the <a href="https://en.wikipedia.org/wiki/World_Wide_Web" target="_blank">World Wide Web</a>.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/netscape%20communicator.png" target="_blank">Netscape communicator loading screen<br><img src="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/netscape%20communicator.png" alt="Netscape communicator loading screen" title="Netscape communicator loading screen" border="0" style="max-width: 100%"></a></p><p>Before <a href="https://en.wikipedia.org/wiki/Ajax_%28programming%29" target="_blank">AJAX</a> was conceptualized, web pages had to be fully refreshed from the server to reflect any change. JavaScript was used for simple client manipulations. From a user experience point of view, it was OK (mostly because we had no other choices) but this limited the development of user interfaces.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/I%20guess.jpg" target="_blank">I guess<br><img src="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/I%20guess.jpg" alt="I guess" title="I guess" border="0" style="max-width: 100%"></a></p><p>Then AJAX introduced new ways to design web pages: only the new information could be fetched from the server without reloading the page. Therefore, the pages were faster, crisper and fully asynchronous.</p><p>However, each browser had its own implementation of AJAX requests <em>(not mentioning DOM, event handling and other incompatibilities)</em>. And that's why <a href="https://jquery.com/" target="_blank">jQuery</a>, which was initially designed to offer a uniform API that would work identically on any browser, became so popular.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/jQuery%20everywhere.jpg" target="_blank">jQuery everywhere<br><img src="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/jQuery%20everywhere.jpg" alt="jQuery everywhere" title="jQuery everywhere" border="0" style="max-width: 100%"></a></p><p>Today, the situation has changed: almost all browsers are implementing the same APIs and, consequently, modern libraries are considering browsers to be one environment only.</p><h3>GPF-JS</h3><p><a href="https://www.npmjs.com/package/gpf-js" target="_blank">GPF-JS</a> obviously supports browsers and it leverages AJAX requests to implement HTTP requests in this environment. But the library is also compatible with <a href="http://nodejs.org/" target="_blank">NodeJS</a> as well as other - less common - command line hosts:</p><ul><li><a href="http://technet.microsoft.com/en-us/library/bb490887.aspx" target="_blank">cscript/wscript</a></li><li><a href="https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Rhino" target="_blank">Rhino</a></li><li><a href="http://phantomjs.org/" target="_blank">PhantomJS</a></li></ul><p>Designing only one API that is compatible with all these hosts means to deal with each host specificities.</p><h2>How to test HTTP request</h2><p>When you follow the <a href="https://en.wikipedia.org/wiki/Test-driven_development" target="_blank">TDD practice</a>, you write tests before writing any line of production code. But in that case, the first challenge was to figure out how the whole HTTP layer could be tested. <a href="https://en.wikipedia.org/wiki/Mock_object" target="_blank">Mocking</a> was not an option.</p><p>The project development environment heavily relies on the <a href="https://github.com/gruntjs/grunt-contrib-connect" target="_blank">grunt connect</a> task to deliver the dashboard: a place where the developer can access all the tools (source list, tests, documentation...).</p><p><a href="https://arnaudbuchholz.github.io/blog/post/Release%200.1.6/homepage.png" target="_blank">dashboard<br><img src="https://arnaudbuchholz.github.io/blog/post/Release%200.1.6/homepage.png" alt="dashboard" title="dashboard" border="0" style="max-width: 100%"></a></p><div class="note"><p> As a lazy developer, I just need one command line for my development (grunt). Then all the tools are available within the dashboard.</p></div><p>Some middleware is plugged to add extra features such as:</p><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/grunt/connect-middleware/cache.js" target="_blank">cache</a>: introduced with <a href="http://gpf-js.blogspot.ca/2017/03/release-017.html" target="_blank">version 0.1.7</a>, it is leveraged by the command line used to test browsers when <a href="http://www.seleniumhq.org/" target="_blank">Selenium</a> is not available. It implements a data storing service like <a href="https://redis.io/" target="_blank">Redis</a>.</li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/grunt/connect-middleware/fs.js" target="_blank">fs</a>: a file access service used to read, create and delete files within the project storage. For instance, it is used by the sources tile to check if a source has a corresponding test file.</li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/grunt/connect-middleware/grunt.js" target="_blank">grunt</a>: a wrapper used to execute and format the log of grunt tasks.</li></ul><p>Based on this experience, it became obvious that the project needed another extension: the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/grunt/connect-middleware/echo.js" target="_blank">echo</a> service. It basically accepts any HTTP request and the response either reflects the request details or can be modified through URL parameters.</p><p><a href="https://www.getpostman.com/" target="_blank">POSTMAN</a> was used to test the tool that will be used to test the HTTP layer...</p><p><a href="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/echo%20test%20GET.png" target="_blank">GET<br><img src="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/echo%20test%20GET.png" alt="GET" title="GET" border="0" style="max-width: 100%"></a></p><p><a href="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/echo%20test%20GET%20500.png" target="_blank">GET 500<br><img src="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/echo%20test%20GET%20500.png" alt="GET 500" title="GET 500" border="0" style="max-width: 100%"></a></p><p><a href="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/echo%20test%20POST.png" target="_blank">POST<br><img src="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/echo%20test%20POST.png" alt="POST" title="POST" border="0" style="max-width: 100%"></a></p><h2>One API to rule them all</h2><p>Now that the HTTP layer can be tested, the API must be designed to write the tests.</p><h3>Input</h3><p>An HTTP request starts with some parameters:</p><ul><li>The <a href="https://en.wikipedia.org/wiki/URL" target="_blank">Uniform Resource Locator</a> which determines the web address you want to send the request to. There are several ways to specify this location: NodeJS offers an <a href="https://nodejs.org/api/url.html#url_class_url" target="_blank">URL class</a> which exposes the different parts of it (host, port ...). However, the simplest representation remains the one everybody is used to: the string you can read inside the browser location bar.</li></ul><ul><li>The <a href="https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol#Request_methods" target="_blank">request method</a> (also known as verb) which specifies the kind of action you want to execute.</li></ul><ul><li>An optional list of <a href="https://en.wikipedia.org/wiki/List_of_HTTP_header_fields" target="_blank">header fields</a> meant to configure the request processing <em>(such as specifying the expected answer type...)</em>. The simplest way to provide this list is to use a key/value dictionary, meaning an <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object" target="_blank">object</a>.</li></ul><ul><li>The <a href="https://en.wikipedia.org/wiki/HTTP_message_body" target="_blank">request body</a>, mostly used for POST and PUT actions, which contains that data to upload to the server. Even if the library supports the concept of <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.interfaces.IWritableStream.html" target="_blank">streams</a>, most of the expected use cases imply sending an envelope that is synchronously built (text, JSON, XML...). Also, JavaScript (in general) is not good at handling binary data, hence a simple string is expected as a request body.</li></ul><p>This leads to the definition of the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.typedef.html#.httpRequestSettings" target="_blank">httpRequestSettings</a> type.</p><h3>Output</h3><p>On completion, the server sends back a <a href="https://www.w3.org/Protocols/rfc2616/rfc2616-sec6.html" target="_blank">response</a> composed of:</p><ul><li>A <a href="https://en.wikipedia.org/wiki/List_of_HTTP_status_codes" target="_blank">status code</a> that provides feedback about how the server processed the request. Typically, 200 means everything went well. On the contrary, 4xx messages signal an error and 500 is a critical server error.</li></ul><ul><li>A list of response headers. For instance, this is how <a href="https://en.wikipedia.org/wiki/HTTP_cookie" target="_blank">cookies</a> are transmitted by the server to the client (and, actually, they are also sent back by the client to the server through headers).</li></ul><ul><li>The response body: depending on what has been requested, it will contain the server answer. This response could be deserialized using a <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.interfaces.IReadableStream.html" target="_blank">readable stream</a>. But, for the same reasons, a simple string containing the whole response text will be returned.</li></ul><p>This leads to the definition of the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.typedef.html#.httpRequestResponse" target="_blank">httpRequestResponse</a> type.</p><div class="note"><p> If needed, the API may evolve later to introduce the possibility to use streams.</p></div><h3>Waiting for the completion</h3><p>An HTTP request is asynchronous; hence the client must wait for the server to answer. To avoid the <a href="http://callbackhell.com/" target="_blank">callback hell</a>, a <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise" target="_blank">Promise</a> is used to represent the eventual completion of the request.</p><p>This leads to the definition of the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.http.html#.request" target="_blank">gpf.http.request</a> API.</p><p>The promise is <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/resolve" target="_blank">resolved</a> when the server answered, whatever the status code (including <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500" target="_blank">500</a>). The only way the promise would be rejected is when something wrong happened during communication.</p><h3>Shortcuts</h3><p>For simple requests, such as a GET with no specific header, the API must be easy to use. Shortcuts are defined to shorten the call, for instance:</p><p><code class="javascript gpf-blog"><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">http</span><span class="symbol">.</span><span class="identifier">get</span><span class="symbol">(</span><span class="identifier">baseUrl</span><span class="symbol">)</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">response</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">process</span><span class="symbol">(</span><span class="identifier">response</span><span class="symbol">.</span><span class="identifier">responseText</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">handleError</span><span class="symbol">)</span><span class="symbol">;</span></code></p><p>See the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.http.html" target="_blank">documentation</a>.</p><h2>Handling different environments</h2><p>Inside the library, there are almost as many implementations as there are supported hosts. Each one is inside a self-titled file below the <a href="https://github.com/ArnaudBuchholz/gpf-js/tree/master/src/http" target="_blank">http source folder</a>. This will be detailed right after.</p><p>Consequently, there are basically many ways to call the proper implementation depending on the host:</p><ul><li>Inside the request API, create an if / else condition that checks every possibility <code class="javascript gpf-blog"><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">http</span><span class="symbol">.</span><span class="identifier">request</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="comment">/*...*/</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">if</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">_GPF_HOST</span><span class="symbol">.</span><span class="identifier">NODEJS</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="identifier">_gpfHost</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">// call NodeJS implementation</span><span class="space">
</span><span class="symbol">}</span><span class="space"> </span><span class="keyword">else</span><span class="space"> </span><span class="keyword">if</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">_GPF_HOST</span><span class="symbol">.</span><span class="identifier">BROWSER</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="identifier">_gpfHost</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">// call Browser implementation</span><span class="space">
</span><span class="symbol">}</span><span class="space"> </span><span class="keyword">else</span><span class="space"> </span><span class="comment">/* ... */</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">;</span></code></li></ul><ul><li>Have a global variable receiving the proper implementation, using an if condition inside each implementation file <code class="javascript gpf-blog"><span class="comment">// Inside src/host/nodejs.js</span><span class="space">
</span><span class="keyword">if</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">_GPF_HOST</span><span class="symbol">.</span><span class="identifier">NODEJS</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="identifier">_gpfHost</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">_gpfHttpRequestImpl</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="comment">/*...*/</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">/* ... NodeJS implementation ... */</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="comment">// Inside src/http.js</span><span class="space">
</span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">http</span><span class="symbol">.</span><span class="identifier">request</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="comment">/*...*/</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">_gpfHttpRequestImpl</span><span class="symbol">(</span><span class="comment">/*...*/</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">;</span></code></li></ul><ul><li>Create a dictionary indexing all implementations per host and then fetch the proper one on call <code class="javascript gpf-blog"><span class="comment">// Inside src/host/nodejs.js</span><span class="space">
</span><span class="identifier">_gpfHttpRequestImplByHost</span><span class="symbol">[</span><span class="identifier">_GPF_HOST</span><span class="symbol">.</span><span class="identifier">NODEJS</span><span class="symbol">]</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">/* ... NodeJS implementation ... */</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// Inside src/http.js</span><span class="space">
</span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">http</span><span class="symbol">.</span><span class="identifier">request</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="comment">/*...*/</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">_gpfHttpRequestImplByHost</span><span class="symbol">[</span><span class="identifier">_gpfHost</span><span class="symbol">]</span><span class="symbol">(</span><span class="comment">/*...*/</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">;</span></code></li></ul><p>My preference goes to the last choice for the following reasons:</p><ul><li>if / else conditions generate <a href="https://en.wikipedia.org/wiki/Cyclomatic_complexity" target="_blank">cyclomatic complexity</a>. In general, the less if the better. In this case, they are useless because used to compare a variable (here the current host) with a list of predefined values (the list of host names). A dictionary is more efficient.</li></ul><ul><li>It is simpler to manipulate a dictionary to dynamically declare a new host or even update an existing implementation. Indeed, we could imagine a plugin mechanism that would change the way requests are working by replacing the default handler.</li></ul><p>Consequently, the internal library variable <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.1/src/http.js#L48-L67" target="_blank">_gpfHttpRequestImplByHost</a> contains all implementations indexed by host name. The request API calls the proper one by fetching the implementation at runtime.</p><h3>Browsers</h3><p>As explained in the introduction, browsers offer AJAX requests to make HTTP Requests. This is possible through the <a href="https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest" target="_blank">XmlHttpRequest</a> JavaScript class.</p><div class="note"><p> There is one major restriction when dealing with AJAX requests. You are mainly limited to the server you are currently browsing. If you try to access a different server (or even a different port on the same server), then you are entering the realm of <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS" target="_blank">cross-origin requests</a>.</p></div><p>If needed, you will find many examples on the web on <a href="https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/Using_XMLHttpRequest" target="_blank">how to use it</a>. Long story short, you can trigger a simple AJAX request in 5 lines of code.</p><p>In terms of processing, it is interesting to note that, once triggered from the JavaScript code, the network communication is fully handled by the browser: it does not require the JavaScript engine. This means that the page may execute some code while the request is being transmitted to the server as well as while waiting for the response. However, to be able to process the result (i.e. trigger the callback), the JavaScript engine must be idle.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/browser.png" target="_blank">Test preview<br><img src="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/browser.png" alt="Test preview" title="Test preview" border="0" style="max-width: 100%"></a></p><p>Browser implementation is done inside <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/src/http/xhr.js" target="_blank">src/http/xhr.js</a>.</p><p>Two external helpers are defined inside <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/src/http/helpers.js" target="_blank">src/http/helpers.js</a>:</p><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.1/src/http/helpers.js#L31-L47" target="_blank">_gpfHttpGenSetHeaders</a></li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.1/src/http/helpers.js#L49-L65" target="_blank">_gpfHttpGenSend</a></li></ul><p>Setting the request headers and sending request data are almost done the same way for three hosts. To avoid <a href="https://en.wikipedia.org/wiki/Don%27t_repeat_yourself" target="_blank">code duplication</a>, those two functions generates specialized versions capable of calling host specific methods.</p><h3>NodeJS</h3><p>Besides being a JavaScript host, <a href="https://nodejs.org/" target="_blank">NodeJS</a> comes with a complete <a href="https://nodejs.org/en/docs/" target="_blank">set of API</a> for a wide variety of tasks. Specifically, it comes with the <a href="https://nodejs.org/api/http.html#http_http" target="_blank">http feature</a>.</p><p>But unlike AJAX requests, triggering an HTTP requests requires more effort than in a browser.</p><p>The <a href="https://nodejs.org/api/http.html#http_http_request_options_callback" target="_blank">http.request</a> method allocates an <a href="https://nodejs.org/api/http.html#http_class_http_clientrequest" target="_blank">http.ClientRequest</a>. However, it expects a structure that details the web address. That's why the <a href="https://nodejs.org/api/url.html#url_url_parse_urlstring_parsequerystring_slashesdenotehost" target="_blank">URL parsing API</a> is needed.</p><p>The clientRequest object is also a <a href="https://nodejs.org/api/stream.html#stream_class_stream_writable" target="_blank">writable stream</a> and it exposes the method to push data over the connection. Things being done at the lowest level, you are responsible of ensuring the consistency of the request details. Indeed, it is mandatory to set the request headers properly. For instance, forgetting the <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Length" target="_blank">Content-Length specification</a> on a PUSH or a PUT will lead to the HPE_UNEXPECTED_CONTENT_LENGTH error. The library is taking care of that part.</p><p>The same way, the response body is a <a href="https://nodejs.org/api/stream.html#stream_class_stream_readable" target="_blank">readable stream</a>. Fortunately, GPF-JS provides a <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/src/stream/nodejs.js#L103-L158" target="_blank">NodeJS-specific stream reader</a> and it deserializes the content inside a string using <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/src/string/fromStream.js" target="_blank">_gpfStringFromStream</a>.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/nodejs.png" target="_blank">Test preview<br><img src="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/nodejs.png" alt="Test preview" title="Test preview" border="0" style="max-width: 100%"></a></p><p>NodeJS implementation is done inside <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/src/http/nodejs.js" target="_blank">src/http/nodejs.js</a>.</p><h3>WScript</h3><p>WScript is the scripting host available on almost all post Windows XP Microsoft Windows operating systems. It comes in two flavors:</p><ul><li><strong>WScript.exe</strong> which is showing outputs in dialog boxes</li><li><strong>cscript.exe</strong> which is the command line counter part</li></ul><p>This host has a rather old and weird support of the JavaScript features. It does not supports <a href="https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/setTimeout" target="_blank">timers</a> but GPF-JS provides all the necessary <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-COMPATIBILITY.html" target="_blank">polyfills</a> to compensate for the missing APIs.</p><p>Despite all these troubles, it has one unique advantage over the other hosts: it offers the possibility to manipulate <a href="https://en.wikipedia.org/wiki/Component_Object_Model" target="_blank">COM components</a>.</p><p>Indeed, the host-specific class <a href="https://docs.microsoft.com/en-us/scripting/javascript/reference/activexobject-object-javascript" target="_blank">ActiveXObject</a> gives you access to thousands of external features within a script:</p><ul><li>file access, like in <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/src/fs/wscript.js" target="_blank">wscript's IFileStorage implementation</a></li><li>xml parsers (<a href="https://stackoverflow.com/questions/6828703/what-is-the-difference-between-sax-and-dom" target="_blank">DOM and SAX</a>)</li><li><a href="https://en.wikipedia.org/wiki/Windows_Management_Instrumentation" target="_blank">Windows Management Instrumentation</a></li><li>Office applications (outlook, word, excel...)</li><li>...</li></ul><div class="note"><p> For instance, few years ago, I created a script capable of reconfiguring a virtual host to fit user preferences and make it unique on the network.</p></div><p>Among the list of <a href="https://stackoverflow.com/questions/3239610/standard-activexobject-refrence-list" target="_blank">available objects</a>, there is one that is used to generate HTTP requests: the <a href="https://msdn.microsoft.com/en-us/library/windows/desktop/aa384106%28v=vs.85%29.aspx" target="_blank">WinHttp.WinHttpRequest.5.1</a> object.</p><p>Basically, it mimics the interface of the XmlHttpRequest object with one significant difference: its behavior is synchronous. As the GPF API returns a Promise, the developer does not have to care about this difference.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/cscript.png" target="_blank">Test preview<br><img src="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/cscript.png" alt="Test preview" title="Test preview" border="0" style="max-width: 100%"></a></p><p>Wscript implementation is done inside <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/src/http/wscript.js" target="_blank">src/http/wscript.js</a>.</p><h3>Rhino</h3><p><a href="https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Rhino" target="_blank">Rhino</a> is probably one of the most challenging - and fun - environment because it is based on <a href="https://docs.oracle.com/javase/7/docs/api/" target="_blank">java</a>.</p><p>The fascinating aspect of Rhino comes from the tight integration between the two languages. Indeed, the JavaScript engine is implemented in java and you can access any java class from JavaScript. In terms of language support, it is almost the same than WScript: no timers and written on a relatively old specification. Here again, the polyfills are taking care of filling the blanks.</p><p>To implement HTTP requests, one have to figure out which part of the java platform would be used. After doing some investigations (thanks <a href="http://lmgtfy.com/?q=java+http+request" target="_blank">Google</a>), the solution appeared to be the <a href="https://docs.oracle.com/javase/7/docs/api/index.html?java/net/URL.html" target="_blank">java.net.URL</a> class.</p><p>Like NodeJS, java streams are used to send or receive data over the connection. Likewise, the library offers <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/src/stream/rhino.js" target="_blank">rhino-specific streams implementation</a>.</p><div class="note"><p> Stream reading works by consuming bytes. To read text, a <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.1/src/stream/rhino.js#L93-L94" target="_blank">java.util.Scanner</a> instance is used.</p></div><p>Surprisingly, if the status code is in the 5xx range, then <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.1/src/http/rhino.js#L32-L38" target="_blank">getting the response stream</a> will fail and you have to go with the error stream.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/rhino.png" target="_blank">Test preview<br><img src="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/rhino.png" alt="Test preview" title="Test preview" border="0" style="max-width: 100%"></a></p><p>Rhino implementation is done inside <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/src/http/rhino.js" target="_blank">src/http/rhino.js</a>.</p><h3>PhantomJS</h3><p>To put it in a nutshell, <a href="http://phantomjs.org/" target="_blank">PhantomJS</a> is a command line simulating a browser. It is mainly used to script access to web sites and it is the perfect tool for <a href="https://en.wikipedia.org/wiki/Test_automation" target="_blank">test automation</a>.</p><p>But there are basically two styles of PhantomJS scripts:</p><ul><li>On one hand, it browses a website and simulates what would happen in a browser</li><li>On the other hand, it is a command line executing some JavaScript code</li></ul><p>As a matter of fact, GPF-JS uses those two ways:</p><ul><li><a href="https://github.com/kmiyashiro/grunt-mocha" target="_blank">mocha</a> is used to automate browser testing with PhantomJS</li><li>a dedicated <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/test/host/phantomjs.js" target="_blank">command line</a> runs the test suite without any web page</li></ul><p>As a result, in this environment, the XmlHttpRequest JavaScript class is available.</p><p>However, like in a browser, this host is also subject to security concerns. Hence you are not allowed to request a server that is not the one being opened.</p><p>Luckily, you can bypass this constraint using a command line parameter: <a href="http://phantomjs.org/api/command-line.html" target="_blank">--web-security=false</a>.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/phantomjs.png" target="_blank">Test preview<br><img src="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/phantomjs.png" alt="Test preview" title="Test preview" border="0" style="max-width: 100%"></a></p><p>PhantomJS implementation is done inside <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/src/http/xhr.js" target="_blank">src/http/xhr.js</a>.</p><h2>Conclusion</h2><p>If you survived the whole article, congratulations <em>(and sorry for the broken English)</em>.</p><p>Now you might be wondering...</p><p><a href="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/whats%20the%20point.jpg" target="_blank">What's the point?<br><img src="https://arnaudbuchholz.github.io/blog/post/5%20ways%20to%20make%20an%20http%20request/whats%20the%20point.jpg" alt="What's the point?" title="What's the point?" border="0" style="max-width: 100%"></a></p><p>Actually, this kind of challenge satisfies <a href="https://www.linkedin.com/pulse/curious-arnaud-buchholz" target="_blank">my curiosity</a>. I learned a lot by implementing this feature and, actually, it was immediately applied to greatly <a href="http://gpf-js.blogspot.ca/2017/06/release-021-side-project-support.html" target="_blank">improve the coverage
measurement</a>.</p><p>Indeed, each host is tested with instrumented files and the collected coverage data is serialized to be later consolidated and reported on. However, as of today, only two hosts supports file storage: NodeJS and WScript. But thanks to the HTTP support, all hosts are <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/test/host/loader.js#L187-L197" target="_blank">sending the coverage data</a> to the fs middleware so that it generates the file.</p><p><a href="https://en.wikipedia.org/wiki/Q.E.D." target="_blank">Q.E.D.</a></p></div>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com1tag:blogger.com,1999:blog-3388038619207639793.post-9748158363046344352017-06-12T04:45:00.001+02:002018-12-07T19:48:53.655+01:00Release 0.2.1: Side project support<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/release.png" align="left">
This new release supports my side projects by implementing html generation and http request helpers.
It also improves the coverage measurement and this is the first version to be published again as an NPM package.
</div>
<div class="code markdown"><h2>New version</h2><p>Here comes the new version:</p><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/9?closed=1" target="_blank">Stories and bugs</a> implemented</li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/tree/v0.2.1" target="_blank">Sources</a></li><li><a href="https://arnaudbuchholz.github.io/gpf/doc/index.html" target="_blank">Documentation</a></li><li><a href="https://www.npmjs.com/package/gpf-js" target="_blank">NPM package</a></li></ul><h3>NPM Publishing</h3><p>Starting from this version, the library will be published as an <a href="https://www.npmjs.com/" target="_blank">NPM package</a> on every release.</p><p>The package was already existing since it was first published for version <a href="https://github.com/ArnaudBuchholz/gpf-js/tree/cdd69ac0459904c0096853dd5402a33d4b242065" target="_blank">0.1.4</a>. However, the library has since been redesigned in a way that is not backward compatible. That's the reason why the MINOR version number was increased.</p><div class="note"><p> It is violating the normal <a href="http://semver.org/" target="_blank">backward compatibility rule</a> but, actually, nobody was really using it... And I didn't want to increase the MAJOR number to 1 until the library is ready.</p></div><p>An <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/.npmignore" target="_blank">.npmignore</a> file instructs NPM which files should be included or not. The package is almost limited to the build folder.</p><h3>HTML generation helper</h3><p>I was watching the excellent <a href="https://www.youtube.com/channel/UCO1cgjhGzsSYb1rsB4bFe4Q" target="_blank">funfunfunction</a> video about <a href="https://www.youtube.com/watch?v=EmGfdlixQHo" target="_blank">the hidden costs of templating languages</a> and, at some point, he showed some HTML generation helpers which syntax amazed me.</p><p>Hence, to support my side project, I decided to create my own <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.web.html#.createTagFunction__anchor" target="_blank">HTML generation helpers</a> based on this syntax.</p><p>For instance: <code class="javascript gpf-blog"><span class="keyword">var</span><span class="space"> </span><span class="identifier">div</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">web</span><span class="symbol">.</span><span class="identifier">createTagFunction</span><span class="symbol">(</span><span class="string">"div"</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">span</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">web</span><span class="symbol">.</span><span class="identifier">createTagFunction</span><span class="symbol">(</span><span class="string">"span"</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">document</span><span class="symbol">.</span><span class="identifier">getElementById</span><span class="symbol">(</span><span class="string">"placeholder"</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">=</span><span class="space">
</span><span class="identifier">div</span><span class="symbol">(</span><span class="symbol">{</span><span class="identifier">className</span><span class="symbol">:</span><span class="space"> </span><span class="string">"test1"</span><span class="symbol">}</span><span class="symbol">,</span><span class="space"> </span><span class="string">"Hello "</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">span</span><span class="symbol">(</span><span class="symbol">{</span><span class="identifier">className</span><span class="symbol">:</span><span class="space"> </span><span class="string">"test2"</span><span class="symbol">}</span><span class="symbol">,</span><span class="space"> </span><span class="string">"World!"</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">)</span><span class="symbol">.</span><span class="identifier">toString</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span></code></p><p>Sadly, I realize that I completely forgot to <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/175" target="_blank">document the feature properly</a>. Fortunately, the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.1/test/web/tag.js" target="_blank">tests</a> demonstrate the feature.</p><h3>HTTP Requests</h3><p>The prominent feature of this version is the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.http.html#.request__anchor" target="_blank">gpf.http.request</a> helper. With it, you can trigger <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP" target="_blank">HTTP requests</a> from any supported host using only one API. The response is given back to the script asynchronously through <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise" target="_blank">Promises</a>.</p><div class="note"><p> The promise is resolved when the server provides a response, whatever the status code.</p></div><p>Some shortcuts are also defined to improve code readability:</p><p><code class="javascript gpf-blog"><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">http</span><span class="symbol">.</span><span class="identifier">get</span><span class="symbol">(</span><span class="identifier">requestUrl</span><span class="symbol">)</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">response</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">if</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">response</span><span class="symbol">.</span><span class="identifier">status</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="number">500</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">Promise</span><span class="symbol">.</span><span class="identifier">reject</span><span class="symbol">(</span><span class="keyword">new</span><span class="space"> </span><span class="identifier">Error</span><span class="symbol">(</span><span class="string">"Internal Error"</span><span class="symbol">)</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">process</span><span class="symbol">(</span><span class="identifier">response</span><span class="symbol">.</span><span class="identifier">responseText</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="keyword">catch</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">reason</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">onError</span><span class="symbol">(</span><span class="identifier">reason</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span></code></p><p>The supported (and tested) features are:</p><ul><li>Most common <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.http.html#.methods__anchor" target="_blank">HTTP verbs</a></li><li>Possibility to provide request headers</li><li>Possibility to access response headers</li><li>Possibility to submit textual payload (on POST and PUT)</li><li>Access to response text</li></ul><p>Other features will be added depending on the needs but this is enough to make it fully functional.</p><div class="note"><p> I will soon write an article about this very specific part as I had lot of challenges to test and implement it.</p></div><h3>Improved coverage</h3><p>Because each host benefits from its own http request implementation, it was important to assess the code coverage.</p><p>Up to this version, the coverage was measured with NodeJS through <a href="https://github.com/gotwarlost/istanbul" target="_blank">istanbul</a> and <a href="https://github.com/pghalliday/grunt-mocha-test" target="_blank">mochaTest</a>. Each host specific code was flagged to be ignored using the <a href="https://github.com/gotwarlost/istanbul/blob/master/ignoring-code-for-coverage.md" target="_blank">istanbul ignore comment</a>.</p><p>However, as the library grows, more code was actually not verified.</p><p>After analyzing how istanbul generates, stores and consolidates the coverage information, I found a way to run the instrumented code on all hosts. A new <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.1/grunt/tasks/istanbul.js#L46-L58" target="_blank">grunt task</a> consolidate all data into the global one.</p><p>In the end, there are still some branches / instructions that are ignored but they are all <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-COVERAGE.html" target="_blank">documented</a>.</p><ul><li>Statements ignored: 0.74%</li><li>Branches ignored: 1.77%</li><li>Functions ignored: 1.36%</li></ul><h2>Lessons learned</h2><h3>ECHO service</h3><p>To be able to test the http helpers, I needed a server which responses could be controlled. I already had some <a href="https://github.com/ArnaudBuchholz/gpf-js/tree/v0.2.1/grunt/connect-middleware" target="_blank">middleware</a> plugged inside the <a href="https://github.com/gruntjs/grunt-contrib-connect" target="_blank">grunt connect</a> task. I decided to create the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.1/grunt/connect-middleware/echo.js" target="_blank">ECHO service</a>.</p><p>I took me a while to figure out the proper coding: for instance, I had to disable the cache because Internet Explorer was storing the request results and it was failing the tests.</p><p>Also, I had to change the way tests are running to transmit the http port information. This is done through the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.2.1/test/http.js#L5-L7" target="_blank">config</a> global object.</p><h3>Some code was definitely not tested</h3><p>Having the coverage measured on almost every lines revealed sections of untested code. This leaded to some simplifications (boot for instance) but also to new tests.</p><h3>Incomplete design for ending a stream</h3><p>In order to prepare the CSV reader, I created a <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.stream.LineAdapter.html" target="_blank">line adapter stream</a>. It implements both the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.interfaces.IReadableStream.html" target="_blank">IReadableStream</a> and <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.interfaces.IWritableStream.html" target="_blank">IWritableStream</a> interfaces. It caches the data being received with write until some line separator is detected.</p><p>However, because of caching, it requires a method to flush the buffered data.</p><p>As of now, I decided to create a <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.stream.LineAdapter.html#.endOfStream__anchor" target="_blank">method</a> to explicitly flush the stream.</p><p>However, I am still wondering if it may change in the future: if you pipe several streams together, it could be convenient to have a standard way to flush all the different levels. One idea could be to introduce a special token that would be written at the end of a stream but then it would require all streams to implement it.</p><h2>Next release</h2><p>For now the project is on pause because of vacations. I will take some time to plan the next iteration more carefully but, still, I have to support my side project by creating a CSV reader as well as a record container.</p></div>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-65673184670157272682017-04-29T20:36:00.000+02:002018-12-07T19:48:29.027+01:00Release 0.1.9: Records files<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/release.png" align="left">
This new release leverages the interface concept and delivers a file storage implementation for NodeJS and WScript.
It also introduces the necessary tools for a side project I am currently working on.
</div>
<div class="code markdown"><h2>New version</h2><p>Here comes the new version:</p><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/8?closed=1" target="_blank">Stories and bugs</a> implemented</li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/tree/v0.1.9" target="_blank">Sources</a></li><li><a href="https://arnaudbuchholz.github.io/gpf/doc/index.html" target="_blank">Documentation</a></li></ul><h3>Path management</h3><p>Because this version comes with file management, it all started with path management. Some existing <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.9/src/path.js" target="_blank">code</a> was waiting to be re-enabled in the library. This was done easily.</p><p>The <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.path.html" target="_blank">gpf.path</a> helpers are considering that the normal <a href="https://en.wikipedia.org/wiki/Path_%28computing%29" target="_blank">path separator</a> is the Unix one. The translation is done whenever necessary and the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.9/test/path.js#L212-L213" target="_blank">tests</a> are considering each case.</p><h3>IFileStorage and streams</h3><p>Three new interfaces were designed in this release:</p><ul><li><a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.interfaces.IFileStorage.html" target="_blank">IFileStorage</a></li><li><a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.interfaces.IReadableStream.html" target="_blank">IReadableStream</a></li><li><a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.interfaces.IWritableStream.html" target="_blank">IWritableStream</a></li></ul><p>The purpose is to encapsulate the file system in a flexible way using <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise" target="_blank">Promises</a> to handle asynchronicity. Also, streams were introduced to abstract files concept to reading or writing data.</p><p>The method <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.fs.html#.getFileStorage__anchor" target="_blank">gpf.fs.getFileStorage</a> retrieves the current host's IFileStorage (if existing).</p><p>Reading a file becomes host independent:</p><p><code class="javascript gpf-blog"><span class="keyword">function</span><span class="space"> </span><span class="identifier">read</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">path</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">iFileStorage</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">fs</span><span class="symbol">.</span><span class="identifier">getFileStorage</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">iWritableStream</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">new</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">stream</span><span class="symbol">.</span><span class="identifier">WritableString</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">iFileStorage</span><span class="symbol">.</span><span class="identifier">openTextStream</span><span class="symbol">(</span><span class="identifier">path</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">fs</span><span class="symbol">.</span><span class="identifier">openFor</span><span class="symbol">.</span><span class="identifier">reading</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">iReadableStream</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">iReadableStream</span><span class="symbol">.</span><span class="identifier">read</span><span class="symbol">(</span><span class="identifier">iWritableStream</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">iFileStorage</span><span class="symbol">.</span><span class="identifier">close</span><span class="symbol">(</span><span class="identifier">iReadableStream</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">iWritableStream</span><span class="symbol">.</span><span class="identifier">toString</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span></code></p><p>And so is writing to a file:</p><p><code class="javascript gpf-blog"><span class="keyword">function</span><span class="space"> </span><span class="identifier">write</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">path</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">content</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">iFileStorage</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">fs</span><span class="symbol">.</span><span class="identifier">getFileStorage</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">iFileStorage</span><span class="symbol">.</span><span class="identifier">openTextStream</span><span class="symbol">(</span><span class="identifier">path</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">fs</span><span class="symbol">.</span><span class="identifier">openFor</span><span class="symbol">.</span><span class="identifier">appending</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">iWritableStream</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">iWritableStream</span><span class="symbol">.</span><span class="identifier">write</span><span class="symbol">(</span><span class="identifier">content</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">.</span><span class="identifier">then</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">iFileStorage</span><span class="symbol">.</span><span class="identifier">close</span><span class="symbol">(</span><span class="identifier">iWritableStream</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span></code></p><p>Following <a href="https://en.wikipedia.org/wiki/Test-driven_development" target="_blank">TDD</a> practice, all the methods were <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.9/test/fs.js" target="_blank">first tested</a> and then implemented for:</p><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.9/src/fs/NodeJS.js" target="_blank">NodeJS</a></li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.9/src/fs/wscript.js" target="_blank">WScript</a></li></ul><p>Some notes:</p><ul><li>If you need to replace a file content, you must <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.interfaces.IFileStorage.html#deleteFile__anchor" target="_blank">delete</a> it first.</li><li>File storage, streams and hosts implementation were existing and were waiting to be re-enabled. However, they were using a convoluted notification mechanism that has been dropped for Promises.</li><li>Some existing code handles binary streams (even with WScript). However, I doubt this would be useful for the coming projects so I removed it. Indeed, the file storage method is named <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.interfaces.IFileStorage.html#openTextStream__anchor" target="_blank">OpenTextStream</a> .</li><li>I plan to later implement file storage for rhino and even browsers (with a dedicated backend).</li></ul><h3>Filters and Sorters</h3><p>This release was labeled "Record files" because I started a side project in which thousands of records will be manipulated. Handling an array of records requires that you can easily filter or sort them on any property.</p><div class="note"><p> As of now, records are supposed to be flat objects.</p></div><p>To make it efficient, the code generation approach was preferred.</p><p>Two functions are proposed:</p><ul><li><a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.html#.createFilterFunction__anchor" target="_blank">gpf.createFilterFunction</a></li></ul><p>The chosen syntax is <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.typedef.html#.filterItem" target="_blank">documented</a> and lots of examples can be found in the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.9/test/filter.js" target="_blank">test cases</a>. I also integrated a <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.typedef.html#.filterLike" target="_blank">regular expression operator</a> that allows interesting <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.9/test/filter.js#L82-L99" target="_blank">extractions</a>.</p><p>I plan to create some parsers to generate the filter from more readable syntaxes (<a href="https://en.wikipedia.org/wiki/SQL#Queries" target="_blank">SQL</a>, <a href="https://en.wikipedia.org/wiki/Lightweight_Directory_Access_Protocol#Search_and_Compare" target="_blank">LDAP</a>...).</p><ul><li><a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.html#.createSortFunction__anchor" target="_blank">gpf.createSortFunction</a></li></ul><p>Sorting can be done on any property, two types of comparison are offered:</p><ul><li>Number <em>(default)</em> where values are compared using a subtraction</li><li>String through <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/localeCompare" target="_blank">localCompare</a></li></ul><p>You may sort on several properties, see the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.9/test/sort.js" target="_blank">examples</a>.</p><h2>Lessons learned</h2><h3>WScript specific behaviors</h3><p>WScript has a pretty weak JavaScript implementation: I found two issues that broke some of my tests.</p><ul><li>Object properties enumeration has to be tweaked because some are simply ignored, see this <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.9/src/foreach.js#L50-L58" target="_blank">WScript specific forEach implementation</a></li></ul><ul><li>Newly created prototypes automatically have the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/constructor" target="_blank">constructor</a> property assigned. I had to remove it for interfaces, see this <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.9/src/define/interface/build.js#L25" target="_blank">fix</a></li></ul><h3>Tests</h3><p>The library has <a href="https://github.com/ArnaudBuchholz/gpf-js/tree/v0.1.9#metrics" target="_blank">now</a> 541 tests. If you compare with version <a href="https://github.com/ArnaudBuchholz/gpf-js/tree/v0.1.8#metrics" target="_blank">0.1.8</a> that had 396, it is almost 1/3 more!</p><p>In terms of the time required to execute them, it takes only 1.5 second to run one round them with NodeJS. It is still acceptable.</p><p>One notable challenge was to test the NodeJS streams wrappers. Some error situations are almost impossible to simulate. With the help of <a href="https://en.wikipedia.org/wiki/Test_double" target="_blank">mocking</a>, and a good comprehension of the stream events, the <a href="(https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.9/test/stream/NodeJS.js" target="_blank">code was secured</a>.</p><h3>Code coverage</h3><p>Now that the library offers two different implementations for the file storage object, a big change occurred in the way coverage is measured. Indeed, the source version now only loads what's specific to the host so that it prevents adding countless "istanbul ignore" comments. But this also means that some files are not covered anymore.</p><p>I plan to <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/160" target="_blank">fix</a> that in the next version.</p><h2>Next release</h2><p>The <a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/9" target="_blank">next release</a> will be dedicated to support my other project:</p><ul><li>A streamed line reader will be developed (and it may lead to a CSV reader)</li><li>The library will be published to NPM (automatically when releasing the library)</li><li>As stated above, the coverage will be re-designed to include other hosts</li></ul><p>However, because of the side project, the release frequency may slow down.</p></div>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-63468257774338863082017-03-30T14:21:00.001+02:002018-12-07T19:56:08.550+01:00Sneaky JavaScript Technics III<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/ninja.png" align="left">
A ninja is a master of disguise. How do you apply the JavaScript ninjutsu to hide an object member so that nobody
can see (and use) it? Here are several ways.
</div>
<div class="code markdown"><h3>The context</h3><p>These days, I started a new <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/151" target="_blank">task</a> on my <a href="https://github.com/ArnaudBuchholz/gpf-js" target="_blank">gpf-js project</a> and it implies manipulating an <a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree" target="_blank">AST</a> structure within <a href="https://nodejs.org/en/" target="_blank">NodeJS</a>. Because the structure analysis requires to have access to the parent nodes, I augmented the AST items with a link to their parent. The result is later serialized into <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON" target="_blank">JSON</a> for tracing purpose but the generated circular reference broke the conversion.</p><p>Actually, I could have used the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify#Parameters" target="_blank">replacer</a> parameter of the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify" target="_blank">JSON.stringify</a> method but this circular reference also caused trouble in my own code. Hence I had to find another way.</p><h3>Reflection</h3><p>The JavaScript language offers some <a href="https://en.wikipedia.org/wiki/Reflection_%28computer_programming%29" target="_blank">reflection</a> mechanisms. Indeed, the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/for...in" target="_blank">for..in</a> syntax is capable of listing all <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Enumerability_and_ownership_of_properties" target="_blank">enumerable properties</a> of any object.</p><div class="note"><p> I would recommend using <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/keys" target="_blank">Object.keys</a> instead. However, the lack of support of old browsers requires that you polyfilled it with for..in combined with <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/hasOwnProperty" target="_blank">hasOwnProperty</a> .</p></div><p>This fundamental machinery is widely used:</p><ul><li><a href="http://eslint.org/" target="_blank">ESlint</a> offers a rule to detect <a href="http://eslint.org/docs/rules/guard-for-in" target="_blank">unsafe uses</a></li><li>JavaScript evolved to make the enumerable state configurable on a property</li></ul><h3>Object.defineProperty</h3><p>The simplest way to add a property to an object is to simply do it: <code class="javascript gpf-blog"><span class="keyword">var</span><span class="space"> </span><span class="identifier">obj</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="symbol">{</span><span class="symbol">}</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">obj</span><span class="symbol">.</span><span class="identifier">newProperty</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="string">"value"</span><span class="symbol">;</span></code></p><p>This newly created property will be enumerable by default.</p><p><a href="http://www.ecma-international.org/ecma-262/5.1/#sec-15.2.3.6" target="_blank">ECMAScript 5.1</a> introduced <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/defineProperty" target="_blank">Object.defineProperty</a> to create a property with options. This feature is <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/defineProperty#Browser_compatibility" target="_blank">implemented</a> by most recent browsers with some limitations when using it on DOM objects.</p><p>It can be used in different ways and most of them are beyond the scope of this article. I will mostly focus on the possibility to create a non-enumerable property:</p><p><code class="javascript gpf-blog"><span class="keyword">var</span><span class="space"> </span><span class="identifier">obj</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="symbol">{</span><span class="symbol">}</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">Object</span><span class="symbol">.</span><span class="identifier">defineProperty</span><span class="symbol">(</span><span class="identifier">obj</span><span class="symbol">,</span><span class="space"> </span><span class="string">"newProperty"</span><span class="symbol">,</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">value</span><span class="symbol">:</span><span class="space"> </span><span class="string">"value"</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">writable</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">true</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">enumerable</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">false</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">configurable</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">false</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span></code></p><p>By setting <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/defineProperty#Enumerable_attribute" target="_blank">enumerable</a> to false, this property will not be enumerated when using the for..in syntax. By setting <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/defineProperty#Configurable_attribute" target="_blank">configurable</a> to false, this property can't be deleted and can't be reconfigured to make it visible again.</p><p>The advantages are:</p><ul><li>It is easy to implement</li><li>It is available since IE9</li></ul><p>But, regarding the initial purpose, it comes with drawbacks:</p><ul><li>As soon as the name of the property is known, anybody can access it</li><li><a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/getOwnPropertyDescriptors" target="_blank">Object.getOwnPropertyDescriptors</a> is a workaround to enumerate existing properties</li></ul><p>To make the task harder for a hacker to figure out the property name, you may generate a complex random name and store it in a 'private' variable.</p></div><script async="" src="//jsfiddle.net/ArnaudBuchholz/yycjow6e/3/embed/js,result/"></script><iframe src="http://jsfiddle.net/yycjow6e/3/embedded/js,result/?username=ArnaudBuchholz" id="JSFEMB_17872" width="100%" height="677.2000122070312" frameborder="0" sandbox="allow-modals allow-forms allow-scripts allow-same-origin allow-popups"></iframe><div class="code markdown"><h3>Symbol</h3><p>Another simple way to add a property to an object is to use the square bracket syntax: <code class="javascript gpf-blog"><span class="keyword">var</span><span class="space"> </span><span class="identifier">obj</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="symbol">{</span><span class="symbol">}</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">newPropertyName</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="string">"newProperty"</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">obj</span><span class="symbol">[</span><span class="identifier">newPropertyName</span><span class="symbol">]</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="string">"value"</span><span class="symbol">;</span></code></p><p>In that example, the newly created property will be named "newProperty" as per the value of the variable. The only difference with the dot syntax is the use of a variable to name the property.</p><p>But what if the variable is not a string?</p><p>For most standard objects (and primitive types), the variable value is converted to string. Consequently, the following syntax is valid <em>(even if it does not make sense)</em>: <code class="javascript gpf-blog"><span class="keyword">var</span><span class="space"> </span><span class="identifier">obj</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="symbol">{</span><span class="symbol">}</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">function</span><span class="space"> </span><span class="identifier">any</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="symbol">}</span><span class="space">
</span><span class="identifier">obj</span><span class="symbol">[</span><span class="identifier">any</span><span class="symbol">]</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="string">"value"</span><span class="symbol">;</span></code></p><p>The created property name will be "function any() {}"</p><div class="note"><p> This obviously means that you can use names that are not valid <a href="https://developer.mozilla.org/en-US/docs/Glossary/Identifier" target="_blank">identifiers</a>. Hence, it is mandatory to use the bracket syntax to access them.</p></div><p>However, there is one mechanism that behaves differently. It was introduced with <a href="http://www.ecma-international.org/ecma-262/6.0" target="_blank">ECMAScript 2015</a>. Every time you call the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Symbol" target="_blank">Symbol</a> function, it returns a unique value. This value type is primitive <strong>but</strong> it does not convert to string.</p><p>This feature is <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Symbol#Browser_compatibility" target="_blank">implemented</a> only by most recent browsers and not by IE.</p><p>The advantages are:</p><ul><li>It is easy to implement</li><li>There is no way to access the property unless you have the symbol value</li></ul></div><script async="" src="//jsfiddle.net/ArnaudBuchholz/h7psxsym/1/embed/js,result/"></script><iframe src="http://jsfiddle.net/h7psxsym/1/embedded/js,result/?username=ArnaudBuchholz" id="JSFEMB_17872" width="100%" height="548.3999938964844" frameborder="0" sandbox="allow-modals allow-forms allow-scripts allow-same-origin allow-popups"></iframe><div class="code markdown"></div>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-15806090683975944712017-03-28T16:57:00.001+02:002018-12-07T19:47:59.314+01:00Release 0.1.8<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/release.png" align="left">
This new release introduces the interface concept and also prepares future optimizations
of the release version.
</div>
<div class="code markdown"><h2>New version</h2><p>Here comes the new version:</p><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/7?closed=1" target="_blank">Stories and bugs</a> implemented</li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/tree/v0.1.8" target="_blank">Sources</a></li><li><a href="https://arnaudbuchholz.github.io/gpf/doc/index.html" target="_blank">Documentation</a></li></ul><h3>Interfaces</h3><p>A new entity type can now be defined: <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-DEFINE.html" target="_blank">interface</a>.</p><p>Being able to define interfaces and to check if an object conforms with their specification will normalize and improve the way encapsulations are handled within the library but also when using it.</p><h3>Release optimization</h3><p>When the library is built, some source manipulations occur:</p><ul><li>Each module is individually transformed into an <a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree" target="_blank">AST</a> structure using <a href="https://www.npmjs.com/package/esprima" target="_blank">esprima</a></li><li>These structures are concatenated inside an <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/make/UMD.js" target="_blank">Universal Module Definition loader</a></li><li>Eventually, the result is serialized using <a href="https://www.npmjs.com/package/escodegen" target="_blank">escodegen</a></li></ul><p>Before this release, the debug and release versions were almost the same. The main differences were:</p><ul><li>The release version is serialized without the comments (see <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/make/release.json" target="_blank">rewriteOptions</a>)</li><li>Also, the release version is minified using <a href="https://github.com/mishoo/UglifyJS" target="_blank">UglifyJS</a></li></ul><p>Because I am planning to implement some optimization patterns on the release version, I started to redesign the build process. After converting all the relevant sources to ES6, I realized that the former <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.7/make/ast.js" target="_blank">AST manipulation</a> was not working properly. Indeed, it was trying to rename all variables to shorten them but... it didn't work. Not a big deal because UglifyJS was already doing the job.</p><p>Finally, I rewrote the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.8/make/ast.js" target="_blank">AST manipulation</a> to start optimizing the concatenated AST structure.</p><h4>Estimating functions usage</h4><p>I implemented a detection of unused functions which revealed that <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.8/src/constants.js#L140-L142" target="_blank">_gpfIsUnsignedByte</a> was not used! Hence, it is removed from the output.</p><h4>NOP</h4><p>The build process includes a <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.8/make/preprocess.js" target="_blank">pre-processor</a> capable of conditioning some JavaScript lines using C-like defines. However, as a lazy developer, I don't want to wrap all DEBUG specific functions inside a #ifdef / #endif pair.</p><p>That's why some variables are tagged with a comment containing only gpf:nop. This is a way to signal that the variable represents a no operation.</p><p>For instance: <code class="javascript gpf-blog"></code><code class="javascript gpf-blog"><span class="identifier">javascript</span><span class="space">
</span><span class="comment">/*#ifdef(DEBUG)*/</span><span class="space">
</span><span class="comment">// DEBUG specifics</span><span class="space">
</span><span class="identifier">_gpfAssertAttributeClassOnly</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">value</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">_gpfAsserts</span><span class="symbol">(</span><span class="symbol">{</span><span class="space">
</span><span class="string">"Expected a class parameter"</span><span class="symbol">:</span><span class="space"> </span><span class="string">"function"</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="keyword">typeof</span><span class="space"> </span><span class="identifier">value</span><span class="symbol">,</span><span class="space">
</span><span class="string">"Expected an Attribute-like class parameter"</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">value</span><span class="symbol">.</span><span class="identifier">prototype</span><span class="space"> </span><span class="keyword">instanceof</span><span class="space"> </span><span class="identifier">_gpfAttribute</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">_gpfAssertAttributeOnly</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">value</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">_gpfAssert</span><span class="symbol">(</span><span class="identifier">value</span><span class="space"> </span><span class="keyword">instanceof</span><span class="space"> </span><span class="identifier">_gpfAttribute</span><span class="symbol">,</span><span class="space"> </span><span class="string">"Expected an Attribute-like parameter"</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">;</span><span class="space">
</span><span class="comment">/* istanbul ignore if */</span><span class="space"> </span><span class="comment">// Because tested in DEBUG</span><span class="space">
</span><span class="keyword">if</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">!</span><span class="identifier">_gpfAssertAttributeClassOnly</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">/*#else*/</span><span class="space">
</span><span class="comment">/*gpf:nop*/</span><span class="space"> </span><span class="identifier">_gpfAssertAttributeClassOnly</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">_gpfEmptyFunc</span><span class="symbol">;</span><span class="space">
</span><span class="comment">/*gpf:nop*/</span><span class="space"> </span><span class="identifier">_gpfAssertAttributeOnly</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">_gpfEmptyFunc</span><span class="symbol">;</span><span class="space">
</span><span class="comment">/*#endif*/</span><span class="space">
</span><span class="comment">/*#ifdef(DEBUG)*/</span><span class="space">
</span><span class="symbol">}</span></code><code class="javascript gpf-blog"></code></p><p>The optimizer is now capable of locating all variables flagged with gpf:nop and safely remove them from the output.</p><h3>Automated release (final)</h3><p>In the last release, I forgot one last step: closing the milestone. This is now <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/148" target="_blank">completed</a>.</p><h2>Lessons learned</h2><h3>Performance measurement</h3><p>When dealing with performances, one of the biggest challenge is to establish the point of reference. Indeed, if you want to quantify how much you gain, you need to make sure that the measurement environment is stable enough so that you can compare your results.</p><p>In my case, this is tough: the JavaScript hosts are evolving very fast and, since I started this project, I already changed my workstation twice. Furthermore, I don't have any performance specific tests.</p><p>So, I decided to take a pragmatic approach.</p><p>I reuse the test cases (that cover almost all the library) and compare the debug version with the release one. By quantifying the difference of execution between the two versions, this would give me a good indication on how much the release is optimized.</p><p>Today, both version demonstrates similar performances, even after implementing some of the optimizations.</p><h3>jsdoc plugin</h3><p>In the article <a href="http://gpf-js.blogspot.ca/2016/12/my-own-jsdoc-plugin.html" target="_blank">My own jsdoc plugin</a>, I explained how I tweaked jsdoc to facilitate generation of the documentation.</p><p>However, I noticed that the <a href="http://usejsdoc.org/about-plugins.html" target="_blank">defineTags</a> could be a better alternative to define custom tags. After trying to experiment it (knowing that jsdoc plugins are badly documented), it appeared that it is extremely limited:</p><ul><li>No lookup function to access other existing doclets</li><li>Very limited information on the current doclet (for instance: I tried to implement @gpf:chainable but the memberOf property was not set). Indeed, we can't know when the onTagged is triggered (no control on the order).</li></ul><h2>Next release</h2><p>The <a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/8" target="_blank">next release</a> will introduce some features that are required for a side project I created.</p></div>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-72551777230381126702017-03-06T17:43:00.002+01:002018-12-07T19:47:42.352+01:00Release 0.1.7<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/release.png" align="left">
This new release secures the class mechanism and improves project tools.
</div>
<div class="code markdown"><h2>New version</h2><p>Here comes the new version:</p><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/6?closed=1" target="_blank">Stories and bugs</a> implemented</li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/tree/v0.1.7" target="_blank">Sources</a></li><li><a href="https://arnaudbuchholz.github.io/gpf/doc/index.html" target="_blank">Documentation</a></li></ul><h3>Improved $super keyword</h3><p>When writing the article <a href="http://gpf-js.blogspot.in/2017/02/my-own-super-implementation.html" target="_blank">My own super implementation</a> I found several issues that were immediately fixed.</p><h3>Better tooling</h3><p>The following tools were modified:</p><ul><li>It is now possible to remove unused files from the sources page</li><li>Each file modification triggers one pass of tests (it takes less than 1 second to execute). This way, I know as soon as possible if something goes wrong</li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/grunt/connect-middleware/fs.js" target="_blank">fs middleware</a> is secured to limit access to the project files only. It now supports the DELETE verb</li><li>The <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/grunt/watch.js" target="_blank">watch</a> and <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/grunt/tasks/serve.js" target="_blank">serve</a> tasks are monitoring the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/src/sources.json" target="_blank">sources.json</a> file modifications to update the list of linted files. This way, it is no more required to restart the grunt task.</li></ul><h3>More flavors for browser testing</h3><p>Selenium was upgraded to version 3 and the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/test/host/selenium/detect.js" target="_blank">detection</a> has been fixed to make it more reliable.</p><p>On top of Selenium, a <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/test/host/browser.js" target="_blank">command line execution wrapper</a> combined with a <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/grunt/connect-middleware/cache.js" target="_blank">caching service</a> (to store & grab the results) allows testing of non-automated browsers (such as Safari on Windows). Once the tests are done, the command line is killed.</p><h3>Automated release</h3><p>Releasing a version has never been so easy, a script using <a href="https://www.npmjs.com/package/github-api" target="_blank">github-api</a> module to call the <a href="https://developer.github.com/v3/" target="_blank">GitHub API</a> implements the following steps:</p><ul><li>Check version number</li><li>Update package.json (if needed)</li><li>Check GitHub milestones to identify the milestone details and check that all issues are closed</li><li>Update README.md</li><li>Grunt make</li><li>Copy tmp/plato/report.history.<em> to build/ (grunt copy:releasePlatoHistory)</em></li><li>commit & push</li><li>Create a new release on GitHub</li><li>Copy build/tests.js into test/host/legacy/{version}.js</li><li>commit & push</li></ul><p>However, once last step was forgotten: closing the milestone. An <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/148" target="_blank">incident</a> is created.</p><h2>Lessons learned</h2><h3>Documenting features makes them better</h3><p>I will task the risk of repeating myself here but the article about <a href="http://gpf-js.blogspot.in/2017/02/my-own-super-implementation.html" target="_blank">super</a> made me realize several mistakes in the implementation of the $super equivalent. Furthermore, taking the time to explore the ECMAScript super keyword gave me a better understanding of the feature.</p><p>In general, it is valuable to step back from the code and document the intent behind a feature.</p><h3>Better Selenium detection</h3><p>One of the reasons why I wanted to remove Selenium was the buggy detection. Indeed, there are some failures which are not encapsulated properly in a Promise. As a result, the whole process fails when it happens.</p><p>After digging on the web, I found this excellent thread on <a href="http://stackoverflow.com/questions/7310521/node-js-best-practice-exception-handling" target="_blank">NodeJS Exception handling</a>. It allowed me to handle those unmanaged exceptions the proper way and it secured the detection.</p><h2>Next release</h2><p>The <a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/7" target="_blank">next release</a> will introduce the interface concept.</p></div>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-8264441911304386612017-02-18T21:43:00.001+01:002018-12-07T19:45:08.261+01:00My own super implementation<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/student.png" align="left">
Release 0.1.6 of GPF-JS delivers a basic class definition mechanism.
Working on the release 0.1.7, the focus is to improve this implementation by providing mechanism that mimic
the ES6 class definition. In particular, the super keyword is replaced with a $super member that provides
the same level of functionalities. Here is how.
</div>
<div class="code markdown"><h2>Introduction</h2><p>The <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/super" target="_blank">super</a> keyword was introduced with ECMAScript 2015. Its goal is to simplify the access to parent methods of an object. It can be used within a class definition or directly in object literals. We will focus on class definition.</p><h3>Class examples</h3><p>To demonstrate the usage, let define a simple class A. <code class="javascript gpf-blog"><span class="identifier">class</span><span class="space"> </span><span class="identifier">A</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">constructor</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">value</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="string">"a"</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">this</span><span class="symbol">.</span><span class="identifier">_a</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">true</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">this</span><span class="symbol">.</span><span class="identifier">_value</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">value</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="identifier">getValue</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="keyword">this</span><span class="symbol">.</span><span class="identifier">_value</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="symbol">}</span></code></p><p>In that example, the class A offers a constructor with an optional parameter (defaulted to "a"). Upon execution, it sets the member _a to true (this will be used later to validate the constructor call). Also, the member _value receives the value of the parameter. Finally, the method getValue exposes _value.</p><p>Then, let subclass it with class B. <code class="javascript gpf-blog"><span class="identifier">class</span><span class="space"> </span><span class="identifier">B</span><span class="space"> </span><span class="identifier">extends</span><span class="space"> </span><span class="identifier">A</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">constructor</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">super</span><span class="symbol">(</span><span class="string">"b"</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">this</span><span class="symbol">.</span><span class="identifier">_b</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">true</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="identifier">getValue</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">super</span><span class="symbol">.</span><span class="identifier">getValue</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">.</span><span class="identifier">toUpperCase</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="symbol">}</span></code></p><p>When instances of B are built, the constructor of A is explicitly called with the parameter "b". Also, the behavior of the method getValue is modified to uppercase the result of parent implementation.</p><p>None of these features are new to JavaScript. Indeed, the exact same definition can be achieved without any of the ECMAScript 2015 keywords.</p><p>For instance:</p><p><code class="javascript gpf-blog"><span class="keyword">function</span><span class="space"> </span><span class="identifier">A</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">value</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">this</span><span class="symbol">.</span><span class="identifier">_a</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">true</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">this</span><span class="symbol">.</span><span class="identifier">_value</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">value</span><span class="space"> </span><span class="symbol">||</span><span class="space"> </span><span class="string">"a"</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="identifier">Object</span><span class="symbol">.</span><span class="identifier">assign</span><span class="symbol">(</span><span class="identifier">A</span><span class="symbol">.</span><span class="identifier">prototype</span><span class="symbol">,</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">getValue</span><span class="symbol">:</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="keyword">this</span><span class="symbol">.</span><span class="identifier">_value</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">function</span><span class="space"> </span><span class="identifier">B</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">A</span><span class="symbol">.</span><span class="identifier">call</span><span class="symbol">(</span><span class="keyword">this</span><span class="symbol">,</span><span class="space"> </span><span class="string">"b"</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">this</span><span class="symbol">.</span><span class="identifier">_b</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">true</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="identifier">B</span><span class="symbol">.</span><span class="identifier">prototype</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">Object</span><span class="symbol">.</span><span class="identifier">create</span><span class="symbol">(</span><span class="identifier">A</span><span class="symbol">.</span><span class="identifier">prototype</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">Object</span><span class="symbol">.</span><span class="identifier">assign</span><span class="symbol">(</span><span class="identifier">B</span><span class="symbol">.</span><span class="identifier">prototype</span><span class="symbol">,</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">getValue</span><span class="symbol">:</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">A</span><span class="symbol">.</span><span class="identifier">prototype</span><span class="symbol">.</span><span class="identifier">getValue</span><span class="symbol">.</span><span class="identifier">call</span><span class="symbol">(</span><span class="keyword">this</span><span class="symbol">)</span><span class="symbol">.</span><span class="identifier">toUpperCase</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span></code></p><div class="note"><p> There are several ways to implement inheritance in JavaScript. In this example, the pattern used in GPF-JS is demonstrated.</p></div><h3>Differences</h3><p>Whether you use one syntax or the other, both versions of A and B will look (and behave) the same:</p><ul><li>A and B are functions</li><li>A.prototype has a method named getValue</li><li>b instances only have own properties _a, _b and _value</li><li>b instanceof A works</li></ul><p><a href="https://jsfiddle.net/ArnaudBuchholz/um9511we/1/" target="_blank">Class version (Chrome & Firefox only)</a></p><p><a href="https://jsfiddle.net/ArnaudBuchholz/kj457uvu/2/" target="_blank">Function version</a></p><h3>So, why would you use the super keyword?</h3><p>As you may see in the examples, accessing the parent methods without super is possible but requires the knowledge of the parent class being extended. Furthermore, the syntax is not easy to remember... Well, after using it a thousand times, you end knowing it by heart.</p><ul><li>In the constructor, <code class="javascript gpf-blog"><span class="identifier">super</span><span class="symbol">(</span><span class="string">"b"</span><span class="symbol">)</span></code> is replaced with <code class="javascript gpf-blog"><span class="identifier">A</span><span class="symbol">.</span><span class="identifier">call</span><span class="symbol">(</span><span class="keyword">this</span><span class="symbol">,</span><span class="space"> </span><span class="string">"b"</span><span class="symbol">)</span></code></li></ul><ul><li>In a method, <code class="javascript gpf-blog"><span class="identifier">super</span><span class="symbol">.</span><span class="identifier">getValue</span><span class="symbol">(</span><span class="symbol">)</span></code> is replaced with <code class="javascript gpf-blog"><span class="identifier">A</span><span class="symbol">.</span><span class="identifier">prototype</span><span class="symbol">.</span><span class="identifier">getValue</span><span class="symbol">.</span><span class="identifier">call</span><span class="symbol">(</span><span class="keyword">this</span><span class="symbol">)</span></code></li></ul><p>As a consequence, any update in the class hierarchy would lead to a mass search & replace in the code.</p><p>Beside this, one could say that this keyword is a typical example of <a href="https://en.wikipedia.org/wiki/Syntactic_sugar" target="_blank">syntaxic sugar</a> as it does not bring new feature.</p><div class="note"><p> if you forget about object literals...</p></div><h3>Exploring the feature</h3><p>Even if the documentation on super is extensive, some questions remains about the way it reacts to <a href="https://en.wikipedia.org/wiki/Edge_case" target="_blank">edge cases</a>.</p><h4>Redefining parent method</h4><p>What happens if the parent prototype is modified? Does it call the modified method or does it call the method that was existing when the child method is defined.</p><p>The link is dynamic.</p><p><a href="https://jsfiddle.net/ArnaudBuchholz/Layhbjye/" target="_blank">Example (Chrome & Firefox only)</a></p><div class="note"><p> This is consistent with the function implementation: A.prototype.getValue re-evaluates the member every time it is called.</p></div><h4>Getting function object</h4><p>Is it possible to access the parent method without invoking it? Does it return a function object?</p><p>It returns the parent function object.</p><p><a href="https://jsfiddle.net/ArnaudBuchholz/34n4ae99/4/" target="_blank">Example (Chrome & Firefox only)</a></p><div class="note"><p> It is important to notice that if not invoked immediately (look at getSuperGetValue in the example), the value of <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/this" target="_blank">this</a> is undefined.</p></div><h4>Checking parent method existence</h4><p>Finally, how does the super keyword validate the method that is accessed: what happens if you try to reference a non-existing member: does it fail when generating the class or upon method execution?</p><p>Accessing a non-existing member returns <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/undefined" target="_blank">undefined</a>.</p><p><a href="https://jsfiddle.net/54L2eovo/1/" target="_blank">Example (Chrome & Firefox only)</a></p><div class="note"><p> This is also consistent with the function implementation: it makes sense that the error is thrown at evaluation time.</p></div><h2>A super idea</h2><p>One of the goals of <a href="https://github.com/ArnaudBuchholz/gpf-js" target="_blank">GPF-JS</a> is to provide the same feature set whatever the host running the script. Because some of them are old (Rhino and WScript), it is not only impossible to use recent features but also it prevents the use of <a href="https://scotch.io/tutorials/javascript-transpilers-what-they-are-why-we-need-them" target="_blank">transpilers</a>.</p><div class="note"><p> Transpilers like <a href="https://babeljs.io/" target="_blank">babel</a> are capable of generating compatible JavaScript code out of next-gen JavaScript source.</p></div><p><a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/doc/tutorials/DEFINE.md#class" target="_blank">gpf.define</a> is a class definition helper exposed by the library since <a href="http://gpf-js.blogspot.ca/2017/02/release-016.html" target="_blank">version 0.1.6</a>. But it would not be complete without a mechanism that mimics the super keyword in order to reduce the complexity of calling parent methods.</p><p>super being a reserved keyword, it could not be used. But as the library reserves $ properties for specific usage, the idea of defining $super naturally came up.</p><p>In order to make the $super keyword a global one (like super), the library had to tweak the global context object which generated lots of issues (leaks detected in mocha, validation errors in linters, the variable could already be defined by the developer...). So, $super had to be attached to the context of the class instance.</p><p>this.$super was defined and had to support two different syntaxes:</p><ul><li>Calling <code class="javascript gpf-blog"><span class="keyword">this</span><span class="symbol">.</span><span class="identifier">$super</span></code> must be equivalent to <code class="javascript gpf-blog"><span class="identifier">super</span></code></li></ul><ul><li>Calling <code class="javascript gpf-blog"><span class="keyword">this</span><span class="symbol">.</span><span class="identifier">$super</span><span class="symbol">.</span><span class="identifier">methodName</span></code> must be equivalent to <code class="javascript gpf-blog"><span class="identifier">super</span><span class="symbol">.</span><span class="identifier">methodName</span></code></li></ul><h3>Class definition</h3><p>The library internally uses an object to retain the initial definition dictionary, parse it and build the class handler. This <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/src/define/class/class.js" target="_blank">class definition</a> object is not yet exposed but will be in the future through a read-only interface.</p><p>This object is a key component of this implementation as it <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/src/define/class/check.js" target="_blank">keeps track</a> of the class properties such as the extended base class. This will be leveraged to access parent methods.</p><div class="note"><p> <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/getPrototypeOf" target="_blank">Object.getPrototypeOf</a> could be used to escalate the prototype chain and retrieve the base methods. However, it is <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/src/compatibility/object.js#L52-L60" target="_blank">poorly polyfilled</a> on old hosts and it does not work as expected with standard objects.</p></div><h3>Wrapping methods</h3><p>In order to be able to cope with this.$super calls inside a method, the library has to make sure that the $super member exists before executing the method.</p><p>A long time ago, when studying JavaScript inheritance, I found this very interesting <a href="http://ejohn.org/blog/simple-javascript-inheritance/" target="_blank">article</a> from <a href="https://www.linkedin.com/in/jeresig" target="_blank">John Resig</a> (the creator of <a href="https://jquery.com/" target="_blank">jQuery</a>).</p><p>It took me ages to fully understand its Class.extend helper but it demonstrates a brilliant JavaScript ninja technique: by testing the method with a regular expression, it is capable of finding out if a class method uses the _super keyword. If so, the method is wrapped inside a container function that defines the _super member for the lifetime of the call.</p><p><code class="javascript gpf-blog"><span class="comment">// Check if we’re overwriting an existing function</span><span class="space">
</span><span class="identifier">prototype</span><span class="symbol">[</span><span class="identifier">name</span><span class="symbol">]</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">typeof</span><span class="space"> </span><span class="identifier">prop</span><span class="symbol">[</span><span class="identifier">name</span><span class="symbol">]</span><span class="space"> </span><span class="symbol">==</span><span class="space"> </span><span class="string">"function"</span><span class="space"> </span><span class="symbol">&&</span><span class="space">
</span><span class="keyword">typeof</span><span class="space"> </span><span class="identifier">_super</span><span class="symbol">[</span><span class="identifier">name</span><span class="symbol">]</span><span class="space"> </span><span class="symbol">==</span><span class="space"> </span><span class="string">"function"</span><span class="space"> </span><span class="symbol">&&</span><span class="space"> </span><span class="identifier">fnTest</span><span class="symbol">.</span><span class="identifier">test</span><span class="symbol">(</span><span class="identifier">prop</span><span class="symbol">[</span><span class="identifier">name</span><span class="symbol">]</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">?</span><span class="space">
</span><span class="symbol">(</span><span class="keyword">function</span><span class="symbol">(</span><span class="identifier">name</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">fn</span><span class="symbol">)</span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="keyword">function</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">tmp</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">this</span><span class="symbol">.</span><span class="identifier">_super</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// Add a new ._super() method that is the same method</span><span class="space">
</span><span class="comment">// but on the super-class</span><span class="space">
</span><span class="keyword">this</span><span class="symbol">.</span><span class="identifier">_super</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">_super</span><span class="symbol">[</span><span class="identifier">name</span><span class="symbol">]</span><span class="symbol">;</span><span class="space">
</span><span class="comment">// The method only need to be bound temporarily, so we</span><span class="space">
</span><span class="comment">// remove it when we’re done executing</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">ret</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">fn</span><span class="symbol">.</span><span class="identifier">apply</span><span class="symbol">(</span><span class="keyword">this</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">arguments</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">this</span><span class="symbol">.</span><span class="identifier">_super</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">tmp</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">ret</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">(</span><span class="identifier">name</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">prop</span><span class="symbol">[</span><span class="identifier">name</span><span class="symbol">]</span><span class="symbol">)</span><span class="space">
</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">prop</span><span class="symbol">[</span><span class="identifier">name</span><span class="symbol">]</span><span class="symbol">;</span></code></p><p>Typically, GPF-JS uses the same strategy to <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/90b7c70c900a226fb652e7efd10a8573be5e270c/src/define/class/super.js#L207-L210" target="_blank">detect</a> the use of $super and <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/90b7c70c900a226fb652e7efd10a8573be5e270c/src/define/class/super.js#L191-L196" target="_blank">wrap</a> the method in a new one that defines the value of this.$super upon execution.</p><div class="note"><p> The use of _gpfFunctionDescribe and _gpfFunctionBuild ensures that the signature of the final method will be the same as the initial one. Indeed, GPF-JS will soon enable interfaces validation and signatures of methods have to match.</p></div><h3>Dynamic mapping of super method</h3><p>So, when the class is being defined, a dictionary mapping method names to their implementation is passed to <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/master/doc/tutorials/DEFINE.md#class" target="_blank">gpf.define</a>. This definition dictionary is enumerated so that when the $super use is detected in a method, the name of the parent method is deduced.</p><p>This name (as well as members of $super, it will be explained right after) is remembered in a <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/90b7c70c900a226fb652e7efd10a8573be5e270c/src/define/class/super.js#L195" target="_blank">closure</a> and passed to the function <a href="https://github.com/ArnaudBuchholzgpf-js/blob/90b7c70c900a226fb652e7efd10a8573be5e270c/src/define/class/super.js#L147" target="_blank">_get$Super</a> before calling the method.</p><h3>Building a new $super method object</h3><p>The class definition method _get$Super <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/90b7c70c900a226fb652e7efd10a8573be5e270c/src/define/class/super.js#L195" target="_blank">creates a new function</a> instead of reusing the parent one. The reason is quite simple: JavaScript functions being objects, it is allowed to add properties to them... and this will be required to define expected additional super method names.</p><p>But then you may wonder why the parent function object is not used? those additional member names could be backed up, overwritten and restored once the call is completed. In the end, it would allow the child method to use parent one members.</p><p>However, there are several considerations here:</p><ul><li>This object could be frozen with <a href="https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Object/freeze" target="_blank">Object.freeze</a> meaning it would be read-only.</li></ul><ul><li>This function object could be used elsewhere meaning that the modification could be visible outside of the method.</li></ul><div class="note"><p> One may argue that this is also true for this.$super. However, detection of this member makes it to be overwritten.</p></div><div class="note"><p> While writing this above comment, I realized that the current implementation has an <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/145" target="_blank">issue</a>. If you ever wonder why I wrote this article, this is a good reason.</p></div><ul><li>Even if super returns the parent method object, it would be extremely confusing to have members that are being used. Consider the following example: super.getValue, how do you know if it is a parent method named getValue or the member getValue of the parent method?</li></ul><div class="note"><p> I suspect this is the reason why super() is supported only in constructor functions. Try using super() in a class method, you will get an error "SyntaxError: 'super' keyword unexpected here". this.$super overcomes this limitation.</p></div><ul><li>If the developer expects to get members on the parent method, he would have a hard time defining them and reusing them (not mentioning the code complexity). This encapsulation prevents this bad practice and avoids headaches.</li></ul><h3>Detecting $super members</h3><p>Once $super is detected in a method content, the list of $super members is <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/90b7c70c900a226fb652e7efd10a8573be5e270c/src/define/class/super.js#L208" target="_blank">extracted</a> using a <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/90b7c70c900a226fb652e7efd10a8573be5e270c/src/define/class/super.js#L118-L134" target="_blank">regular expression</a>.</p><div class="note"><p> This detection part is critical as it greatly improves performances by generating only what is required.</p></div><p>Then, for each extracted name, the member is <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/90b7c70c900a226fb652e7efd10a8573be5e270c/src/define/class/super.js#L150-L152" target="_blank">created</a> inside _get$Super right after allocating $super.</p><h3>Invoking super methods</h3><p>When calling this.$super(), the method $super would obviously receive the proper <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/this" target="_blank">context</a>.</p><p>However, things are more complicated when calling this.$super.methodName().</p><p>If you understand how <a href="https://www.safaribooksonline.com/library/view/javascript-the-good/9780596517748/ch04s03.html" target="_blank">JavaScript function invocation</a> works, you know that inside methodName, <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/this" target="_blank">this</a> would be equal to this.$super.</p><p>And that is a function object.</p><p>So how can the library make sure that the proper context is transmitted to methodName?</p><p>Function binding could be used to force the value of this but then we would lose the possibility to invoke it with any context.</p><h3>Function binding</h3><p>Before <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Function/bind" target="_blank">Function.prototype.bind</a> was introduced, people used to create a closure to force the value of this inside a function.</p><p><code class="javascript gpf-blog"><span class="identifier">Function</span><span class="symbol">.</span><span class="identifier">prototype</span><span class="symbol">.</span><span class="identifier">bind</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">function</span><span class="symbol">(</span><span class="identifier">oThis</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">fToBind</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="keyword">this</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">fToBind</span><span class="symbol">.</span><span class="identifier">apply</span><span class="symbol">(</span><span class="identifier">oThis</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">arguments</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">;</span></code></p><p>This concept was also made popular with <a href="https://api.jquery.com/jQuery.proxy/" target="_blank">jQuery.proxy</a>.</p><p>The drawback is that, once a function is bound, it is no more possible to change the context it is executed with.</p><p><a href="https://jsfiddle.net/ArnaudBuchholz/duxnd03m/" target="_blank">Demonstration</a> <code class="javascript gpf-blog"><span class="keyword">function</span><span class="space"> </span><span class="identifier">getValue</span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="keyword">this</span><span class="symbol">.</span><span class="identifier">value</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="comment">// Passing the context</span><span class="space">
</span><span class="identifier">log</span><span class="symbol">(</span><span class="identifier">getValue</span><span class="symbol">.</span><span class="identifier">call</span><span class="symbol">(</span><span class="symbol">{</span><span class="space">
</span><span class="identifier">value</span><span class="symbol">:</span><span class="space"> </span><span class="string">"Hello World"</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">)</span><span class="symbol">;</span><span class="space"> </span><span class="comment">// output "Hello World"</span><span class="space">
</span><span class="comment">// Binding</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">boundGetValue</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">getValue</span><span class="symbol">.</span><span class="identifier">bind</span><span class="symbol">(</span><span class="symbol">{</span><span class="space">
</span><span class="identifier">value</span><span class="symbol">:</span><span class="space"> </span><span class="string">"Bound"</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">log</span><span class="symbol">(</span><span class="identifier">boundGetValue</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">)</span><span class="symbol">;</span><span class="space"> </span><span class="comment">// output "Bound"</span><span class="space">
</span><span class="comment">// Trying to pass a different context</span><span class="space">
</span><span class="identifier">log</span><span class="symbol">(</span><span class="identifier">boundGetValue</span><span class="symbol">.</span><span class="identifier">call</span><span class="symbol">(</span><span class="symbol">{</span><span class="space">
</span><span class="identifier">value</span><span class="symbol">:</span><span class="space"> </span><span class="string">"Hello World"</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">)</span><span class="symbol">;</span><span class="space"> </span><span class="comment">// output "Bound"</span><span class="space">
</span><span class="comment">// Trying to bind again</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">reboundGetValue</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">boundGetValue</span><span class="symbol">.</span><span class="identifier">bind</span><span class="symbol">(</span><span class="symbol">{</span><span class="space">
</span><span class="identifier">value</span><span class="symbol">:</span><span class="space"> </span><span class="string">"Hello World"</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">log</span><span class="symbol">(</span><span class="identifier">reboundGetValue</span><span class="symbol">(</span><span class="symbol">)</span><span class="symbol">)</span><span class="symbol">;</span><span class="space"> </span><span class="comment">// output "Bound"</span></code></p><p>Back to the $super.methodName example, it requires a sort of weak bind: a method binding that could be overwritten with a different context using <a href="https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_objects/Function/bind" target="_blank">bind</a>, <a href="https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_objects/Function/call" target="_blank">call</a> or <a href="https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_objects/Function/apply" target="_blank">apply</a>.</p><h3>Weak binding</h3><p>$super being known when the methods are created, it can be compared with the value of this and <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/90b7c70c900a226fb652e7efd10a8573be5e270c/src/define/class/super.js#L92" target="_blank">substituted</a> when matched.</p><p>This realizes the weak binding and allows the developer to bind, call or apply the method without any problem.</p><h2>Conclusion</h2><p>There is no revolution in this article and many will consider this realization useless as they focus on modern environment and they use latest JavaScript features. However, <a href="https://www.linkedin.com/pulse/curious-arnaud-buchholz" target="_blank">my curiosity</a> is satisfied as I learned a lot about the super feature. Moreover, the library will soon deliver new features on top of this one that should make the difference.</p></div>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com1tag:blogger.com,1999:blog-3388038619207639793.post-86703047266118750292017-02-06T15:53:00.001+01:002018-12-07T19:47:18.207+01:00Release 0.1.6<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/release.png" align="left">
This new release delivers the initial class mechanism as well as minor improvements.
</div>
<div class="code markdown"><h2>New version</h2><p>Here comes the new version:</p><ul><li><a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/5?closed=1" target="_blank">Stories and bugs</a> implemented</li><li><a href="https://github.com/ArnaudBuchholz/gpf-js/tree/v0.1.6" target="_blank">Sources</a></li><li><a href="https://arnaudbuchholz.github.io/gpf/doc/index.html" target="_blank">Documentation</a></li></ul><h3>Easier setup</h3><p>The very first time you clone the project and run grunt, a configuration menu will be prompted:</p><ul><li>It allows you to select the http port the server will run on</li><li>It will detect cscript (or you may force the detection status)</li><li>It allows you to change the quality metrics</li><li>It detects the selenium-compatible browsers</li></ul><p><a href="https://arnaudbuchholz.github.io/blog/post/Release%200.1.6/config.png" target="_blank">grunt<br><img src="https://arnaudbuchholz.github.io/blog/post/Release%200.1.6/config.png" alt="grunt" title="grunt" border="0" style="max-width: 100%"></a></p><p>Once finished, it builds the library so that the metrics will appear in the project homepage.</p><p><a href="https://arnaudbuchholz.github.io/blog/post/Release%200.1.6/homepage.png" target="_blank">homepage<br><img src="https://arnaudbuchholz.github.io/blog/post/Release%200.1.6/homepage.png" alt="homepage" title="homepage" border="0" style="max-width: 100%"></a></p><h3>Better compatibility across hosts</h3><p>One major feature of the library is to provide the same level of features whatever the host it runs on. Consequently, I am always looking for methods that exist in recent JavaScript versions and that are missing on some hosts (specifically Rhino and Wscript).</p><p>For instance, this version introduces <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/some" target="_blank">Array.prototype.some</a>.</p><p>I am also planning to implement <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/assign" target="_blank">Object.assign</a> and deprecate <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.html#.extend__anchor" target="_blank">gpf.extend</a> as it does the same.</p><h3>Backward compatibility</h3><p>Each version's test file is now <a href="https://github.com/ArnaudBuchholz/gpf-js/tree/master/test/legacy" target="_blank">kept</a> and tested automatically during the build process. This ensures backward compatibility.</p><h3>Simple class definition</h3><p>This version offers the new <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.html#.define__anchor" target="_blank">gpf.define</a> API. It simplifies class creation through a definition dictionary, check the <a href="https://arnaudbuchholz.github.io/gpf/doc/tutorial-DEFINE.html" target="_blank">documentation</a>.</p><div class="note"><p> Your feedback is welcome: this is the early stage of entity definition and there is plenty of time to improve it.</p></div><h2>Lessons learned</h2><h3>Improving maintainability using regular expression</h3><p>If you have read the other articles on my blog you know that I recently changed my mind about regular expressions. Actually, I found myself using those more and more to reduce the code complexity.</p><h3>Removing Selenium</h3><p>I have lots of troubles with Selenium, this is due to several reasons:</p><ul><li>I use version 2 and version 3 has been released 5 month ago. FireFox does not work anymore with version 2.</li><li>Selenium relies on drivers. Browsers are updated automatically which implies that drivers must also be updated regularly.</li></ul><p>Looking at the way I use Selenium, it could be replaced with a simpler mechanism. I just need it to start the browser, run the proper page and wait for the final result. There is not much automation involved.</p><p>Hence I am planning to remove Selenium and implement a custom mechanism in the next version.</p><h2>Next release</h2><p>The <a href="https://github.com/ArnaudBuchholz/gpf-js/milestone/6" target="_blank">next release</a> will mostly consist in securing the gpf.define API and handle all the bugs detected during the development of version 0.1.6.</p></div>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0tag:blogger.com,1999:blog-3388038619207639793.post-20003213625565575482016-12-17T15:05:00.002+01:002018-12-07T19:35:49.096+01:00My own jsdoc plugin<div class="summary">
<img src="https://ArnaudBuchholz.github.io/blog/student.png" align="left">
Jsdoc provides a convenient way to document APIs by adding comments directly in the code.
However, the task can be tedious when it comes to documenting every details of each symbol.
Luckily, the tool provides ways to interact with the parsing process through the creation of plugins.
</div>
<div class="code markdown"><h2>Introduction</h2><p>Just before the release of <a href="http://gpf-js.blogspot.ca/2016/12/release-015.html" target="_blank">version 0.1.5</a>, I focused on <a href="https://arnaudbuchholz.github.io/gpf/doc/index.html" target="_blank">documenting the API</a>. I knew I had to use <a href="http://usejsdoc.org/" target="_blank">jsdoc</a> so I started adding comments early in the development process.</p><h3>Documentation</h3><p>Before going any further with jsdoc, I would like to quickly present my point of view on documentation.</p><p>I tend to agree with <a href="http://softwareengineering.stackexchange.com/questions/285787/clean-code-comments-vs-class-documentation" target="_blank">Uncle Bob's view on documentation</a> meaning that I first focus on making the code clean and, on rare occasions, I put some comments to clarify some non-obvious facts.</p><p><a href="http://thequotes.in/code-never-lies-comments-sometimes-do-ron-jeffries/" target="_blank">Code never lies, comments sometimes do.</a></p><p>This being said, you don't expect developers to read the code to understand which methods they have access to and how to use them. That's why, you need to document the API.</p><h3>Automation and validation</h3><p>To make it a part of the build process, I installed <a href="https://github.com/krampstudio/grunt-jsdoc" target="_blank">grunt-jsdoc</a> and configured <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/grunt/jsdoc.js" target="_blank">two tasks</a>:</p><ul><li>One <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/doc/private.json" target="_blank">'private'</a> to see all symbols (including the private and the internal ones)</li><li>One <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/doc/public.json" target="_blank">'public'</a> for the official documentation (only the public symbols)</li></ul><div class="note"><p> The default rendering of jsdoc is quite boring, I decided to go with <a href="https://www.npmjs.com/package/ink-docstrap" target="_blank">ink-docstrap</a> for the public documentation.</p></div><p>To make sure my jsdoc comments are consistent and correctly used, I also <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/.eslintrc#L29-L44" target="_blank">configured eslint</a> to validate them.</p><div class="note"><p> jsdoc offers many aliases (for instance @return and @returns). That's why eslint allows you to decide which tokens should be preferred.</p></div><p>Finally, I decided to control which files would be used to generate documentation through the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/src/sources.json" target="_blank">sources.json</a> doc properties.</p><h3>The reality</h3><p>After fixing all the linter errors, I quickly realized that I had to do lot of copy & paste to generate the proper documentation.</p><p>For example: when an internal method is exposed as a public API, the comment must be copied.</p><ul><li>On one hand, the internal method must be flagged with <a href="http://usejsdoc.org/tags-private.html" target="_blank">@private</a></li><li>On the other hand, the public method has the same comment but flagged with <a href="http://usejsdoc.org/tags-public.html" target="_blank">@public</a></li></ul><p><code class="javascript gpf-blog"><span class="comment">/**
* Extends the destination object by copying own enumerable properties from the source object.
* If the member already exists, it is overwritten.
*
* @param {Object} destination Destination object
* @param {...Object} source Source objects
* @return {Object} Destination object
* @private
*/</span><span class="space">
</span><span class="keyword">function</span><span class="space"> </span><span class="identifier">_gpfExtend</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">destination</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">source</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">_gpfIgnore</span><span class="symbol">(</span><span class="identifier">source</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">[</span><span class="symbol">]</span><span class="symbol">.</span><span class="identifier">slice</span><span class="symbol">.</span><span class="identifier">call</span><span class="symbol">(</span><span class="identifier">arguments</span><span class="symbol">,</span><span class="space"> </span><span class="number">1</span><span class="symbol">)</span><span class="symbol">.</span><span class="identifier">forEach</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">nthSource</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">_gpfObjectForEach</span><span class="symbol">(</span><span class="identifier">nthSource</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">_gpfAssign</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">destination</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">destination</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="comment">/**
* Extends the destination object by copying own enumerable properties from the source object.
* If the member already exists, it is overwritten.
*
* @param {Object} destination Destination object
* @param {...Object} source Source objects
* @return {Object} Destination object
* @public
*/</span><span class="space">
</span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">extend</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">_gpfExtend</span><span class="symbol">;</span></code></p><p>This implies double maintenance with the risk of forgetting to replace @private with @public.</p><p>The lazy developer in me started to get annoyed and I started to look at ways I could do things more efficiently.</p><p>In that case, we could instruct jsdoc to copy the comment from the internal method and use the name to detect if the API is public or private (depending if it starts with '_').</p><h2>jsdoc plugins</h2><p>That's quite paradoxical for a documentation tool to have such a short explanation on <a href="http://usejsdoc.org/about-plugins.html" target="_blank">plugins</a>.</p><h3>Comments and doclets</h3><p>So let's start with the <a href="http://usejsdoc.org/about-getting-started.html" target="_blank">basis</a>: jsdoc relies on specific comment blocks (starting with exactly two stars) to detect documentation placeholders. It is not required for these blocks to be located near a symbol but, when they do, the symbol context is used to determine what is documented.</p><p><code class="javascript gpf-blog"><span class="comment">/** this is a valid jsdoc description for variable a */</span><span class="space">
</span><span class="keyword">var</span><span class="space"> </span><span class="identifier">a</span><span class="symbol">;</span><span class="space">
</span><span class="comment">/**
@file This is also a valid jsdoc description for the whole file
*/</span><span class="space">
</span><span class="comment">/*** this comment is not a valid jsdoc one */</span><span class="space">
</span><span class="comment">/*
* This is not a valid jsdoc comment, even if it contains jsdoc tags
* @return {Object} Empty object
* @public
*/</span><span class="space">
</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="symbol">{</span><span class="symbol">}</span><span class="space">
</span><span class="symbol">}</span></code></p><p>Each valid jsdoc comment block is converted into a JavaScript object, named doclet, containing extracted information.</p><p>For instance, the following comment and function <code class="javascript gpf-blog"><span class="comment">/**
* Extends the destination object by copying own enumerable properties from the source object.
* If the member already exists, it is overwritten.
*
* @param {Object} destination Destination object
* @param {...Object} source Source objects
* @return {Object} Destination object
* @private
*/</span><span class="space">
</span><span class="keyword">function</span><span class="space"> </span><span class="identifier">_gpfExtend</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">destination</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">source</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">_gpfIgnore</span><span class="symbol">(</span><span class="identifier">source</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">[</span><span class="symbol">]</span><span class="symbol">.</span><span class="identifier">slice</span><span class="symbol">.</span><span class="identifier">call</span><span class="symbol">(</span><span class="identifier">arguments</span><span class="symbol">,</span><span class="space"> </span><span class="number">1</span><span class="symbol">)</span><span class="symbol">.</span><span class="identifier">forEach</span><span class="symbol">(</span><span class="keyword">function</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">nthSource</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">_gpfObjectForEach</span><span class="symbol">(</span><span class="identifier">nthSource</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">_gpfAssign</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">destination</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">return</span><span class="space"> </span><span class="identifier">destination</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span></code></p><p>generates the following doclet <code class="javascript gpf-blog"><span class="symbol">{</span><span class="space"> </span><span class="identifier">comment</span><span class="symbol">:</span><span class="space"> </span><span class="string">'/**\n * Extends the destination object by copying own enumerable properties from the source object.\n * If the member already exists, it is overwritten.\n *\n * @param {Object} destination Destination object\n * @param {...Object} source Source objects\n * @return {Object} Destination object\n * @since 0.1.5\n */'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">meta</span><span class="symbol">:</span><span class="space">
</span><span class="symbol">{</span><span class="space"> </span><span class="identifier">range</span><span class="symbol">:</span><span class="space"> </span><span class="symbol">[</span><span class="space"> </span><span class="number">834</span><span class="symbol">,</span><span class="space"> </span><span class="number">1061</span><span class="space"> </span><span class="symbol">]</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">filename</span><span class="symbol">:</span><span class="space"> </span><span class="string">'extend.js'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">lineno</span><span class="symbol">:</span><span class="space"> </span><span class="number">34</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">path</span><span class="symbol">:</span><span class="space"> </span><span class="string">'J:\\Nano et Nono\\Arnaud\\dev\\GitHub\\gpf-js\\src'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">code</span><span class="symbol">:</span><span class="space">
</span><span class="symbol">{</span><span class="space"> </span><span class="identifier">id</span><span class="symbol">:</span><span class="space"> </span><span class="string">'astnode100000433'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">name</span><span class="symbol">:</span><span class="space"> </span><span class="string">'_gpfExtend'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">type</span><span class="symbol">:</span><span class="space"> </span><span class="string">'FunctionDeclaration'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">paramnames</span><span class="symbol">:</span><span class="space"> </span><span class="symbol">[</span><span class="identifier">Object</span><span class="symbol">]</span><span class="space"> </span><span class="symbol">}</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">vars</span><span class="symbol">:</span><span class="space"> </span><span class="symbol">{</span><span class="space"> </span><span class="string">''</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">null</span><span class="space"> </span><span class="symbol">}</span><span class="space"> </span><span class="symbol">}</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">description</span><span class="symbol">:</span><span class="space"> </span><span class="string">'Extends the destination object by copying own enumerable properties from the source object.\nIf the member already exists, it is overwritten.'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">params</span><span class="symbol">:</span><span class="space">
</span><span class="symbol">[</span><span class="space"> </span><span class="symbol">{</span><span class="space"> </span><span class="identifier">type</span><span class="symbol">:</span><span class="space"> </span><span class="symbol">[</span><span class="identifier">Object</span><span class="symbol">]</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">description</span><span class="symbol">:</span><span class="space"> </span><span class="string">'Destination object'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">name</span><span class="symbol">:</span><span class="space"> </span><span class="string">'destination'</span><span class="space"> </span><span class="symbol">}</span><span class="symbol">,</span><span class="space">
</span><span class="symbol">{</span><span class="space"> </span><span class="identifier">type</span><span class="symbol">:</span><span class="space"> </span><span class="symbol">[</span><span class="identifier">Object</span><span class="symbol">]</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">variable</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">true</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">description</span><span class="symbol">:</span><span class="space"> </span><span class="string">'Source objects'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">name</span><span class="symbol">:</span><span class="space"> </span><span class="string">'source'</span><span class="space"> </span><span class="symbol">}</span><span class="space"> </span><span class="symbol">]</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">returns</span><span class="symbol">:</span><span class="space"> </span><span class="symbol">[</span><span class="space"> </span><span class="symbol">{</span><span class="space"> </span><span class="identifier">type</span><span class="symbol">:</span><span class="space"> </span><span class="symbol">[</span><span class="identifier">Object</span><span class="symbol">]</span><span class="symbol">,</span><span class="space"> </span><span class="identifier">description</span><span class="symbol">:</span><span class="space"> </span><span class="string">'Destination object'</span><span class="space"> </span><span class="symbol">}</span><span class="space"> </span><span class="symbol">]</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">name</span><span class="symbol">:</span><span class="space"> </span><span class="string">'_gpfExtend'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">longname</span><span class="symbol">:</span><span class="space"> </span><span class="string">'_gpfExtend'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">kind</span><span class="symbol">:</span><span class="space"> </span><span class="string">'function'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">scope</span><span class="symbol">:</span><span class="space"> </span><span class="string">'global'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">access</span><span class="symbol">:</span><span class="space"> </span><span class="string">'private'</span><span class="space"> </span><span class="symbol">}</span></code></p><p>The structure itself is not fully documented as it depends on the tags used and the symbol context. However, some properties are most likely to be found, see the <a href="http://usejsdoc.org/about-plugins.html" target="_blank">newDoclet event</a> documentation.</p><p>I strongly recommend running jsdoc using the command line and output <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/doc/gpf.js#L15-L23" target="_blank">some traces</a> to have a better understanding on how doclets are generated.</p><div class="note"><p> In the GPF-JS welcome page, I created a link named "JSDoc plugin test" for that purpose. It uses an <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/grunt/exec.js#L49-L56" target="_blank">exec:jsdoc task</a>.</p></div><h3>Plugins interaction</h3><p>The plugins can be used to interact with jsdoc at three different levels:</p><ul><li>Interact with the parsing process through event handlers (beforeParse, jsdocCommentFound, newDoclet, processingComplete...)</li><li>Define tags and be notified when they are encountered inside a jsdoc comment: it gives you the chance to alter the doclet that is generated</li><li>Interact with the parsing process through an <a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree" target="_blank">AST</a> node visitor</li></ul><p>The most important thing to remember is that you can interfere with doclet generation by altering them or even preventing them. But, I struggled to find ways to generate them on the fly (i.e. without any jsdoc block comment).</p><div class="note"><p> It looks like there is a way to generate new doclets with a node visitor. However, the documentation is not very clear on that part. See <a href="http://www.zuojj.com/tdocs/jsdoc3/about-plugins.html" target="_blank">this example</a>.</p></div><h2>gpf.js plugin</h2><p>Most of the following mechanisms are triggered during the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/doc/gpf.js#L275-L277" target="_blank">processingComplete event</a> so that all doclets are already generated and available.</p><h3>Member types</h3><p>When creating a class, I usually declare members and initialize them with a default value that is representative of the expected member type. This works well with primitive types or arrays but it gets more complicated when dealing with object references (which are most of the time initialized with null).</p><p>For instance, in <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/src/error.js#L25-L35" target="_blank">error.js</a>:</p><p><code class="javascript gpf-blog"><span class="identifier">_gpfExtend</span><span class="symbol">(</span><span class="identifier">_GpfError</span><span class="symbol">.</span><span class="identifier">prototype</span><span class="symbol">,</span><span class="space"> </span><span class="comment">/** @lends gpf.Error.prototype */</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">constructor</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">_GpfError</span><span class="symbol">,</span><span class="space">
</span><span class="comment">/**
* Error code
*
* @readonly
* @since 0.1.5
*/</span><span class="space">
</span><span class="identifier">code</span><span class="symbol">:</span><span class="space"> </span><span class="number">0</span><span class="symbol">,</span></code></p><p>In that case, the member type can easily be deduced from the AST node:</p><p><code class="javascript gpf-blog"><span class="symbol">{</span><span class="space"> </span><span class="identifier">comment</span><span class="symbol">:</span><span class="space"> </span><span class="string">'/**\n * Error code\n *\n * @readonly\n * @since 0.1.5\n */'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">meta</span><span class="symbol">:</span><span class="space">
</span><span class="symbol">{</span><span class="space"> </span><span class="identifier">range</span><span class="symbol">:</span><span class="space"> </span><span class="symbol">[</span><span class="space"> </span><span class="number">801</span><span class="symbol">,</span><span class="space"> </span><span class="number">808</span><span class="space"> </span><span class="symbol">]</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">filename</span><span class="symbol">:</span><span class="space"> </span><span class="string">'error.js'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">lineno</span><span class="symbol">:</span><span class="space"> </span><span class="number">35</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">path</span><span class="symbol">:</span><span class="space"> </span><span class="string">'J:\\Nano et Nono\\Arnaud\\dev\\GitHub\\gpf-js\\src'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">code</span><span class="symbol">:</span><span class="space">
</span><span class="symbol">{</span><span class="space"> </span><span class="identifier">id</span><span class="symbol">:</span><span class="space"> </span><span class="string">'astnode100000209'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">name</span><span class="symbol">:</span><span class="space"> </span><span class="string">'code'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">type</span><span class="symbol">:</span><span class="space"> </span><span class="string">'Literal'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">value</span><span class="symbol">:</span><span class="space"> </span><span class="number">0</span><span class="space"> </span><span class="symbol">}</span><span class="space"> </span><span class="symbol">}</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">description</span><span class="symbol">:</span><span class="space"> </span><span class="string">'Error code'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">readonly</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">true</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">since</span><span class="symbol">:</span><span class="space"> </span><span class="string">'0.1.5'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">name</span><span class="symbol">:</span><span class="space"> </span><span class="string">'code'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">longname</span><span class="symbol">:</span><span class="space"> </span><span class="string">'gpf.Error#code'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">kind</span><span class="symbol">:</span><span class="space"> </span><span class="string">'member'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">memberof</span><span class="symbol">:</span><span class="space"> </span><span class="string">'gpf.Error'</span><span class="symbol">,</span><span class="space">
</span><span class="identifier">scope</span><span class="symbol">:</span><span class="space"> </span><span class="string">'instance'</span><span class="space"> </span><span class="symbol">}</span></code></p><p>Indeed the AST structure provides the literal value the member is initialized with (see meta.code.value).</p><p>This is done in the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/doc/gpf.js#L115-L134" target="_blank">_addMemberType</a> function.</p><h3>Access type based on naming convention</h3><p>There are no real private members in JavaScript. There are <a href="https://developer.mozilla.org/en-US/Add-ons/SDK/Guides/Contributor_s_Guide/Private_Properties" target="_blank">ways</a> to achieve similar behavior (such as function scoped variables used in closure methods) but this is not the discussion here.</p><p>The main idea is to detail, through the documentation, which members the developer can rely on (public or protected when inherited) and which ones should not be used directly (because they are private).</p><p>Because of the way JavaScript is designed, everything is public by default. But I follow the naming convention where the underscore at the beginning of the member name means that the member is private.</p><p>As a consequence, the symbol name gives information about its access type.</p><p>This is leveraged in the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/doc/gpf.js#L136-L144" target="_blank">_checkAccess</a> function.</p><h4>Access type based on class definitions</h4><p>In the next version, I will implement a class definition method and the structure will provide information about members visibility. This will include a way to define static members.</p><p>The idea will be to leverage the node visitor to keep track of which visibility is defined on top of members.</p><h3>Custom tags</h3><p>Through custom tags, I am able to instruct the plugin to modify the generated doclets in specific ways. I decided to prefix all custom tags with "gpf:" to easily identify them, a dictionary defines all the existing names and their associated handlers. It is leveraged in the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/doc/gpf.js#L96-L113" target="_blank">_handleCustomTags </a> function.</p><h4>@gpf:chainable</h4><p>When a method is designed to return the current instance so that you can easily chain calls, the tag @gpf:chainable is used. It instructs jsdoc that the return type is the current class and the description is normalized to "Self reference to allow chaining".</p><p>It is implemented <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/doc/gpf.js#L52-L59" target="_blank">here</a>.</p><h4>@gpf:read / @gpf:write</h4><p>Followed by a member name, it provides pre-defined signatures for getters and setters. Note that the member doclet must be defined when the tag is executed.</p><p>They are implemented <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/doc/gpf.js#L61-L74" target="_blank">here</a>.</p><h4>@gpf:sameas</h4><p>This basically solves the problem I mentioned at the beginning of the article by copying another symbol documentation, provided the doclet exists.</p><p>It is implemented <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/doc/gpf.js#L76-L92" target="_blank">here</a>.</p><h3>The enumeration case</h3><p>The library uses an enumeration to describe the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.html#.hosts__anchor" target="_blank">host type</a>. The advantage of an enumeration is the encapsulation of the value that is used internally. Sadly, jsdoc reveals this value as the 'default value' in the generated documentation.</p><p>Hence, I decided to remove it.</p><p>This is done <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/doc/gpf.js#L146-L151" target="_blank">here</a>, based on <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/doc/gpf.js#L162-L164" target="_blank">this condition</a>.</p><div class="note"><p> Actually, the type is also listed but the enumeration itself is a type... It will be <a href="https://github.com/ArnaudBuchholz/gpf-js/issues/119" target="_blank">removed</a>.</p></div><h3>Error generation in GPF-JS</h3><p>That's probably the best example to demonstrate that laziness can become a virtue.</p><p>In the library, error management is handled through specific exceptions. Each error is associated to a specific class which comes with an error code and a message. The message can be built with substitution placeholders. The <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.Error.html" target="_blank">gpf.Error</a> class offers shortcut methods to create and throw errors in one call.</p><p>For instance, the <a href="https://arnaudbuchholz.github.io/gpf/doc/gpf.Error.AssertionFailed.html" target="_blank">AssertionFailed</a> error is thrown with: <code class="javascript gpf-blog"><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">Error</span><span class="symbol">.</span><span class="identifier">assertionFailed</span><span class="symbol">(</span><span class="symbol">{</span><span class="space">
</span><span class="identifier">message</span><span class="symbol">:</span><span class="space"> </span><span class="identifier">message</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span></code></p><p>The <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/test/error.js#L33-L47" target="_blank">test case</a> shows the exception details: <code class="javascript gpf-blog"><span class="keyword">var</span><span class="space"> </span><span class="identifier">exceptionCaught</span><span class="symbol">;</span><span class="space">
</span><span class="keyword">try</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">Error</span><span class="symbol">.</span><span class="identifier">assertionFailed</span><span class="symbol">(</span><span class="symbol">{</span><span class="space">
</span><span class="identifier">message</span><span class="symbol">:</span><span class="space"> </span><span class="string">"Test"</span><span class="space">
</span><span class="symbol">}</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space"> </span><span class="keyword">catch</span><span class="space"> </span><span class="symbol">(</span><span class="identifier">e</span><span class="symbol">)</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="identifier">exceptionCaught</span><span class="space"> </span><span class="symbol">=</span><span class="space"> </span><span class="identifier">e</span><span class="symbol">;</span><span class="space">
</span><span class="symbol">}</span><span class="space">
</span><span class="identifier">assert</span><span class="symbol">(</span><span class="identifier">exceptionCaught</span><span class="space"> </span><span class="keyword">instanceof</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">Error</span><span class="symbol">.</span><span class="identifier">AssertionFailed</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">assert</span><span class="symbol">(</span><span class="identifier">exceptionCaught</span><span class="symbol">.</span><span class="identifier">code</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">Error</span><span class="symbol">.</span><span class="identifier">CODE_ASSERTIONFAILED</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">assert</span><span class="symbol">(</span><span class="identifier">exceptionCaught</span><span class="symbol">.</span><span class="identifier">code</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="identifier">gpf</span><span class="symbol">.</span><span class="identifier">Error</span><span class="symbol">.</span><span class="identifier">assertionFailed</span><span class="symbol">.</span><span class="identifier">CODE</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">assert</span><span class="symbol">(</span><span class="identifier">exceptionCaught</span><span class="symbol">.</span><span class="identifier">name</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="string">"assertionFailed"</span><span class="symbol">)</span><span class="symbol">;</span><span class="space">
</span><span class="identifier">assert</span><span class="symbol">(</span><span class="identifier">exceptionCaught</span><span class="symbol">.</span><span class="identifier">message</span><span class="space"> </span><span class="symbol">===</span><span class="space"> </span><span class="string">"Assertion failed: Test"</span><span class="symbol">)</span><span class="symbol">;</span></code></p><h4>Errors generation</h4><p>You might wonder how the AssertionFailed class is declared?</p><p>Actually, this is almost done in <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/src/error.js#L155-L169" target="_blank">two lines of code</a>: <code class="javascript gpf-blog"><span class="identifier">_gpfErrorDeclare</span><span class="symbol">(</span><span class="string">"error"</span><span class="symbol">,</span><span class="space"> </span><span class="symbol">{</span><span class="space">
</span><span class="comment">/* ... */</span><span class="space">
</span><span class="comment">/**
* ### Summary
*
* An assertion failed
*
* ### Description
*
* This error is triggered when an assertion fails
*
* @see {@link gpf.assert}
* @see {@link gpf.asserts}
* @since 0.1.5
*/</span><span class="space">
</span><span class="identifier">assertionFailed</span><span class="symbol">:</span><span class="space">
</span><span class="string">"Assertion failed: {message}"</span><span class="symbol">,</span></code></p><p>The _gpfErrorDeclare internal method is capable of creating the exception class (its properties and throwing helper) using only an exception name and a description. It extensively uses <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/src/error.js#L72-L106" target="_blank">code generation technics</a>.</p><h4>Documentation generation</h4><p>As you might notice, the jsdoc block comment preceding the assertionFailed declaration does not contain any class or function documentation. Indeed, this comment is reused by the plugin to generate new comments.</p><p>Actually, this is done in two steps:</p><strong>Creating new documentation blocks during the beforeParse</strong><p>Hooking the beforeParse event, the plugin will search for any use of the _gpfErrorDeclare method.</p><p>A <a href="https://regex101.com/r/DyffIn/1" target="_blank">regular expression</a> captures the function and extract the two parameters. Then, a <a href="https://regex101.com/r/5878Ac/1" target="_blank">second one</a> extracts each name, message and description to <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/doc/gpf.js#L172-L203" target="_blank">generate new jsdoc comments</a>.</p><strong>Blocking the default handling through the node visitor</strong><p>By default, the initial jsdoc block comment will document a temporary object member. Now that the proper comments have been injected through the beforeParse event, a node visitor prevents any doclet to be generated <strong>inside</strong> the _gpfErrorDeclare method.</p><p>This is implemented <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/doc/gpf.js#L251-L255" target="_blank">here</a>.</p><div class="note"><p> Actually, I could have removed the comment block during the beforeParse event but the lines numbering would have been altered.</p></div><h2>ESLint customization</h2><p>Adding new function signature tags through the jsdoc plugin helps me to reduce the amount of comments required to document the code. As mentioned at the beginning of the article, I <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/.eslintrc#L29-L44" target="_blank">configured eslint</a> to validate any jsdoc comment.</p><p>However, because the linter is not aware of the plugin, it started to tell me that my jsdoc comments were invalid.</p><p>So I duplicated and customized the <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/.eslintrules/valid-jsdoc.js#L324-L329" target="_blank">valid-jsdoc.js</a> rule to make it aware of those new tags.</p><h2>@since</h2><p>Knowing in which version an API is introduced may be helpful. The is the purpose of the tag <a href="http://usejsdoc.org/tags-since.html" target="_blank">@since</a>. However, manually setting it can be boring (and you might forget some comments).</p><p>Here again, this was <a href="https://github.com/ArnaudBuchholz/gpf-js/blob/v0.1.5/make/version.js" target="_blank">automated</a>.</p><h2>Conclusion</h2><p><a href="https://arnaudbuchholz.github.io/blog/post/My%20own%20jsdoc%20plugin/bill-gates-lazy.jpg" target="_blank">Bill Gates quote<br><img src="https://arnaudbuchholz.github.io/blog/post/My%20own%20jsdoc%20plugin/bill-gates-lazy.jpg" alt="Bill Gates quote" title="Bill Gates quote" border="0" style="max-width: 100%"></a></p><p>Obviously, there is an upfront investment to automate everything but, now, the lazy developer in me is satisfied.</p></div>Arnaudhttp://www.blogger.com/profile/14797479358159256275noreply@blogger.com0