This new release leverages the interface concept and delivers a file storage implementation for NodeJS and WScript.
It also introduces the necessary tools for a side project I am currently working on.
New version
Here comes the new version:
- Stories and bugs implemented
- Sources
- Documentation
Path management
Because this version comes with file management, it all started with path management. Some existing code was waiting to be re-enabled in the library. This was done easily.
The gpf.path helpers are considering that the normal path separator is the Unix one. The translation is done whenever necessary and the tests are considering each case.
IFileStorage and streams
Three new interfaces were designed in this release:
The purpose is to encapsulate the file system in a flexible way using Promises to handle asynchronicity. Also, streams were introduced to abstract files concept to reading or writing data.
The method gpf.fs.getFileStorage retrieves the current host's IFileStorage (if existing).
Reading a file becomes host independent:
function read (path) {
var iFileStorage = gpf.fs.getFileStorage(),
iWritableStream = new gpf.stream.WritableString();
return iFileStorage.openTextStream(path, gpf.fs.openFor.reading)
.then(function (iReadableStream) {
return iReadableStream.read(iWritableStream);
.then(function () {
return iFileStorage.close(iReadableStream);
})
.then(function () {
return iWritableStream.toString();
});
});
}
And so is writing to a file:
function write (path, content) {
var iFileStorage = gpf.fs.getFileStorage();
return iFileStorage.openTextStream(path, gpf.fs.openFor.appending)
.then(function (iWritableStream) {
return iWritableStream.write(content)
.then(function () {
return iFileStorage.close(iWritableStream);
});
});
}
Following TDD practice, all the methods were first tested and then implemented for:
Some notes:
- If you need to replace a file content, you must delete it first.
- File storage, streams and hosts implementation were existing and were waiting to be re-enabled. However, they were using a convoluted notification mechanism that has been dropped for Promises.
- Some existing code handles binary streams (even with WScript). However, I doubt this would be useful for the coming projects so I removed it. Indeed, the file storage method is named OpenTextStream .
- I plan to later implement file storage for rhino and even browsers (with a dedicated backend).
Filters and Sorters
This release was labeled "Record files" because I started a side project in which thousands of records will be manipulated. Handling an array of records requires that you can easily filter or sort them on any property.
As of now, records are supposed to be flat objects.
To make it efficient, the code generation approach was preferred.
Two functions are proposed:
The chosen syntax is documented and lots of examples can be found in the test cases. I also integrated a regular expression operator that allows interesting extractions.
I plan to create some parsers to generate the filter from more readable syntaxes (SQL, LDAP...).
Sorting can be done on any property, two types of comparison are offered:
- Number (default) where values are compared using a subtraction
- String through localCompare
You may sort on several properties, see the examples.
Lessons learned
WScript specific behaviors
WScript has a pretty weak JavaScript implementation: I found two issues that broke some of my tests.
- Object properties enumeration has to be tweaked because some are simply ignored, see this WScript specific forEach implementation
- Newly created prototypes automatically have the constructor property assigned. I had to remove it for interfaces, see this fix
Tests
The library has now 541 tests. If you compare with version 0.1.8 that had 396, it is almost 1/3 more!
In terms of the time required to execute them, it takes only 1.5 second to run one round them with NodeJS. It is still acceptable.
One notable challenge was to test the NodeJS streams wrappers. Some error situations are almost impossible to simulate. With the help of mocking, and a good comprehension of the stream events, the code was secured.
Code coverage
Now that the library offers two different implementations for the file storage object, a big change occurred in the way coverage is measured. Indeed, the source version now only loads what's specific to the host so that it prevents adding countless "istanbul ignore" comments. But this also means that some files are not covered anymore.
I plan to fix that in the next version.
Next release
The next release will be dedicated to support my other project:
- A streamed line reader will be developed (and it may lead to a CSV reader)
- The library will be published to NPM (automatically when releasing the library)
- As stated above, the coverage will be re-designed to include other hosts
However, because of the side project, the release frequency may slow down.
