Recipe for yarn:

# To get cache dir of specific yarn installation:
# RUN yarn cache dir
RUN mkdir -p /usr/local/share/.cache/yarn
RUN --mount=type=cache,target=/usr/local/share/.cache/yarn yarn

So here comes Tiva, an "expensive" plain object validator leverages TypeScript language service, and it supports extended validation functions.

Usage

It's simple to use:

import {Tiva} from 'tiva';

let tiva = new Tiva();

tiva
  .validate(
    {module: 'module-specifier', type: 'AwesomeType'},
    {foo: 'abc', bar: 123},
  )
  .then(console.info, console.error);

And the ability to extend makes it powerful:

interface Schema {
  /** @uuid */
  id: string;
}

There are several built-in extensions like uuid, unique, pattern. And writing custom extensions is easy:

let tiva = new Tiva({
  extensions: {
    custom(value) {
      if (value === 'custom') {
        return undefined;
      }
      
      return `Value "${value}" must be "custom"`;
    },
  },
});

How it works

Tiva provides a Validator class that manipulates TypeScript language service to do the heavy lifting; and a Tiva class that creates a worker to run Validator in another thread.

The type check part is simple: it just gets the diagnostic messages from TypeScript. The tricky part is the extended validation.

Here's how Tiva does it:

1. It recursively visits the types used by the type to be validate against.
2. It find tags like @uuid in those types, and asks TypeScript to find the implementations within the plain object to be validated.
3. It then validates the value against the extension.

Again the heavy lifting is done by the TypeScript language service. And doing this way also makes it possible to have Tiva work with complex types including condition types, mapping types etc.

Conclusion

Tiva might be a nice tool for certain scenarios, but the way it works makes it "expensive": slow initializing, slow validating, and with a large footprint by any means.

If you find JSON schema related tools insufficient or trivial to setup for your scenarios and performance is not a huge concern, you may want to try out Tiva. But otherwise, I don't think it would be the best choice.

I've been working on a remote console as part of our IoT JavaScript platform Ruff. The console needs to evaluate expressions with following constraints:

1. It needs to evaluate inside an immediately called function. We are compiling modules within an immediately called function just like what Node.js does. Developer might define variables and functions inside a module, and if its the entrance module of an application, we would like to access variables and functions inside.
2. It needs access to variables or functions previously defined in other evaluated expressions. This is relatively easy to understand, as we usually want to keep the context.

The solution applied is to transform expressions with Babel by writing a simple Babel plugin that visits VariableDeclaration and FunctionDeclaration and transform them into global assignments. For example, var abc = 123; is transformed to global.abc = 123; undefined;.

However, a colleague of mine became interested into this thing and thought it might be possible to do so without transforming the expressions. My attitude to that thought was like: come on, I know JavaScript much better than you do, and I've told you it cannot be done! But soon on the road to lunch I realized that I might be wrong. We exchanged the idea and came up with two similar implementations.

Here's my version (refined):

var __eval = function () {
    __eval = eval(`(${arguments.callee.toString()})`);
    return eval(arguments[0]);
};

The idea is to evaluate the function expression that contains eval that evaluates the function expression "recursively". It might be a little tricky to understand but let's unfold the eval expression for function:

var __eval = function () {
    __eval = function () {
        __eval = eval(`(${arguments.callee.toString()})`);
        return eval(arguments[0]);
    };
    return eval(arguments[0]);
};

And again:

var __eval = function () {
    __eval = function () {
        __eval = function () {
            __eval = eval(`(${arguments.callee.toString()})`);
            return eval(arguments[0]);
        };
        return eval(arguments[0]);
    };
    return eval(arguments[0]);
};

So everytime __eval is called, it creates another __eval function inside previous __eval, thus the context may persist!

However, this approach is not perfect. On the one hand, it makes the chain unable to be freed; and on the other hand, newly defined variables could "block" the access to previously defined ones. Though the idea of recursively evaluating is still inspiring, and hope you enjoy it as well.

For me Ctrl/Cmd+Shift+C is a convenient way to open a command prompt or terminal to execute some commands. However currently Visual Studio Code does not support specifying custom console yet.

I mentioned how to modify VS Code program files to achieve this, but as now we can write extensions for Code, why not just do it in this more elegant way?

So I wrote a tiny extension for opening a console at the directory of workspace.

You may find this extension by Ctrl/Cmd+P and typing ext install console.

After installation, the extension will add a keybinding overrides default Ctrl/Cmd+Shift+C with its own console.open command. You'll need to configure the console executable before using that shortcut again.

For ConEmu on Windows, the basic configuration is like:

{
    "console.executable": "cmd.exe",
    "console.args": "/s /c \"\"C:\\Program Files\\ConEmu\\ConEmu64.exe\" /single\""
}

Now try to open a folder in Visual Studio Code and press Ctrl/Cmd+Shift+C, you should be seeing the new console. ;)

GitHub https://github.com/vilic/vscode-console

As a great tool for JavaScript, TypeScript is trending. However, it's still new and you may be confused how would one actually use TypeScript in real world?

I've been using TypeScript in my projects since 1.0. In this post, I will create a website in TypeScript, with real-world file structure and technologies.

Though to make this single post more focused on back-end TypeScript, I will not touch how front-end files should be post-processed.

Meet Visual Studio Code

Visual Studio Code is a free, open source, and cross-platform editor that ships with built-in TypeScript support. In this post, I will use Code as the editor. But every thing except the task configuration part can be applied to editors like Sublime Text with latest TypeScript plugin.

Prerequisites

• Node.js (prefer newer version).
• tsc (TypeScript compiler) command line tool.
• tsd (TypeScript declaration) command line tool.

To install tsc and tsd command line tool, open a terminal and execute:

npm install typescript tsd -g

Create and Initialize the Project

To create a project, let's create a folder named code-ts, or whatever you like.

Open this folder in terminal and initialize the project with npm.

npm init

Default values will do.

Now we'll need to install the modules we need for this project:

npm install express vio consolidate handlebars --save

And declaration files for framework/library written in JavaScript:

tsd install express consolidate

tsd comes with a --save option, but I don't recommend using that. The reason is, not all details in declaration files are accurate, and sometimes you may need to make some changes yourself and commit it to your project repo.

The Structure

Here's the file structure we're going to have:

- code-ts
  - static
    - style.css
  - views
    - default.hbs
  - src
    - routes
      - default.ts
    - server.ts
  - bld
    - routes
      - default.js
    - server.js
  - tsconfig.json
  - package.json

tsconfig.json

tsconfig.json tells TypeScript compiler how to compile your code, it also tells the editor how to provide intellisense and error hinting.

A basic tsconfig.json for Node.js could be like this:

{
    "compilerOptions": {
        "target": "ES5",
        "module": "commonjs"
    }
}

But the version I'll recommend is like this:

{
    "compilerOptions": {
        "target": "ES5",
        "module": "commonjs",
        "noEmitOnError": true,
        "noImplicitAny": true,
        "experimentalDecorators": true,
        "sourceMap": true,
        "sourceRoot": "src",
        "outDir": "bld"
    },
    "exclude": [
        "bld",
        "node_modules"
    ]
}

You may want to know more about these options but for now let's just copy and paste.

Build the Web

Open or create following files:

src/server.ts

import * as Path from 'path';

import * as express from 'express';
import { handlebars } from 'consolidate';
import { Router } from 'vio';

let app = express();

app.engine('hbs', handlebars);

app.use('/static', express.static(Path.join(__dirname, '../static')));

let router = new Router(app, {
    routesRoot: Path.join(__dirname, 'routes'),
    viewsRoot: Path.join(__dirname, '../views'),
    viewsExtension: 'hbs'
});

app.listen(1337);

src/routes/default.ts

import { Controller, get } from 'vio';

export default class Default extends Controller {
    @get()
    default() {
        return {
            title: 'Hello, World!',
            content: 'Keep calm and read the source code.'
        };
    }
}

views/default.hbs

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>{{title}}</title>
    <link rel="stylesheet" href="/static/style.css">
</head>
<body>
    <h1>{{title}}</h1>
    <p>{{content}}</p>
</body>
</html>

static/style.css

body {
    font-family: sans-serif;
    color: #666;
}

h1 {
    color: #333;
}

Compile The Magic

Now we have all the source code ready, time to compile!

Typically we can run tsc in the directory containing tsconfig.json without specifying an argument. And if we want the compiler to watch the changes of source code and do incremental compilation, we may specify -w or --watch argument:

tsc -w

But let's Ctrl+C and moving back to Visual Studio Code.

Now press Ctrl/Cmd+Shift+B, and you should be seeing a prompt asking you to configure task runner.

Click "Configure Task Runner" and it would create .vscode/tasks.json file with a configuration template.

Let's replace the content with:

{
    "version": "0.1.0",
    "command": "tsc",
    "isShellCommand": true,
    "isWatching": true,
    "showOutput": "silent",
    "args": ["-w"],
    "problemMatcher": "$tsc-watch"
}

And now press Ctrl/Cmd+Shift+B again. If you get lucky, there would be a spinning line at left bottom of Visual Studio Code. Now press Ctrl/Cmd+Shift+U to checkout output of the task.

If everything goes well, you will now have the bld folder with compiled JavaScript files.

And it's time to launch the site:

node bld/server.js

Now you can visit http://localhost:1337/ to see the rendered page.

Try modify src/routes/default.ts and views/default.hbs to see updated outputs.

The router we used in this tutorial https://github.com/vilic/vio.

Nexe is a command-line utility that compiles your Node.js application into a single executable file. And Commander.js is the most popular command-line interfaces solution for Node.js. But unfortunately, they don't play well with each other when it comes to sub commands.

To accomplish what's not possible, I started tricky works between Nexe and Commander.js.

But again unfortunately, it required both library to be modified (for Nexe actually it was bug fix).

The fix to Nexe is now merged and published to npm, but the pull request to Commander.js has not yet been merged (and I doubt whether they will ever merge it, as it's not directly relevant to its own functionality).

Luckily you get my fork, and we are going to play with Nexe and Commander.js sub commands now.

npm install nexe -g
npm install vilic/commander.js --save

Nexe will analyze the files to packed based on string literals inside require call. But Commander.js will try to find sub commands dynamically. So the first thing we'll need to do, as what's suggested in README of Nexe, is to add require of each sub command to someplace that will never be executed (pay attention to your uglify task and make sure it's not removing it).

For example:

function commands() {
    require('./command-foo');
    require('./command-bar');
}

Great, expecting more? But that's it.

Enjoy.

ES6 brings a new way to organize modules in JavaScript. Actually by using TypeScript, we have already been using import ... from '...'; for a long time since TypeScript 1.5 alpha.

But, this is what we get:

And it doesn't look good, especially when you need to type the module paths in varieties of files. So what's the partial solution?

The Pattern

The key is export * from '...';, and what we need to do is just using a file as the proxy of your modules.

index.ts

export { Promise } from 'thenfail';

export * from './module-a';
export * from './module-b';
export * from './module-c';

module-a.ts

import {
    Promise,
    somethingFromModuleA,
    somethingFromModuleB
} from './index';

In this pattern, you will also have better awareness of some potential issues might be caused by circular references.

Tools

At this time point, you may use either TypeScript, Babel or Traceur as ES6 module polyfill. Maybe also System.js.

Cheers and that's all. You still need to fill the name of what you've imported, and that's why I highlighted the "partial" word.

In the previous part, we've made the task queue usable. However, for better performance, we usually want multiple tasks in the queue run simultaneously.

What comes to your mind first?

Start Multiple Tasks Simultaneously

We are dealing with an asynchronous issue, but we can still take something that runs synchronously: The for loop.

Think about this:

var tasks = [];
var concurrency = 10;

for (var i = 0; i < concurrency; i++) {
    next();
}

function next() {
    var task = tasks.shift();
    
    task(function () {
        next();
    });
}

The idea behind it couldn't be simpler: Start some tasks, when one of them completes, start another one. Thus the number of running tasks can always be a fixed number if there are sufficient tasks in the queue.

And if there's no more tasks to start:

1. Do nothing if it's not the last running task completing.
2. Callback if it is the last running task.

Callback at Proper Time

We've added the variable settled to help determine the callback status. Now we also need to determine whether we really should start another queued task after completing the current one.

The rule we'll set here is: If any error happens during running these tasks, callback with the error and no other tasks will be started.

To achieve this, we can add another variable queueSettled.

Let's again put the lines together:

function runTasks(tasks, concurrency, callback, onprogress) {
    tasks = tasks.concat();
    
    var completed = 0;
    var total = tasks.length;
    
    // To determine whether we need to start the next task when one completes.
    var queueSettled = false;
    
    var sync = true;
    
    // Concurrency can't be greater than total number of tasks.
    concurrency = Math.min(concurrency, total);
    
    // Start multiple tasks.
    for (var i = 0; i < concurrency; i++) {
        next();
    }
    
    sync = false;

    function invokeCallback(error) {
        if (sync) {
            setTimeout(function () {
                callback(error);
            }, 0);
        } else {
            callback(error);
        }
    }

    function next() {
        onprogress(completed, total);

        if (tasks.length === 0) {
            // Now `tasks.length === 0` doesn't mean all tasks have completed any more.
            // So we need to compare `completed` with `total`.
            if (completed === total) {
                invokeCallback();
            }
            
            return;
        }

        var task = tasks.shift();
        var settled = false;

        try {
            task(function (error) {
                if (settled) {
                    return;
                }

                settled = true;

                if (error) {
                    queueSettled = true;
                    invokeCallback(error);
                } else {
                    completed++;
                    if (!queueSettled) {
                        next();
                    }
                }
            });
        } catch (error) {
            if (settled) {
                return;
            }

            settled = true;
            queueSettled = true;
            invokeCallback(error);
        }
    }
}

Seems to be good, right? But there are some small problems.

If the tasks complete synchronously, the first next call in for loop might actually run out all tasks, and later calls in the same loop might invoke next unnecessarily as there might not be task in the queue any more.

As a result, onprogress might be called multiple times with exactly the same completed argument. It could be okay but we want it more "predictable".

The solution is simple too.

We added the flag sync to help determine whether we need an extra setTimeout to make sure callback of function runTasks is called asynchronously.

But we could have a easier way: Use setTimeout to start the first task. And now we'll use setTimeout to start the first several tasks.

Another problem is we forgot to deal with empty tasks queues.

So here's what we get:

function runTasks(tasks, concurrency, callback, onprogress) {
    tasks = tasks.concat();
    
    var completed = 0;
    var total = tasks.length;
    var queueSettled = false;
    
    var sync = true;
    
    // At least trigger `next` one time.
    concurrency = Math.max(Math.min(concurrency, total), 1);
    
    for (var i = 0; i < concurrency; i++) {
        // Use `setTimeout`.
        setTimeout(next, 0);
    }
    
    sync = false;

    function invokeCallback(error) {
        if (sync) {
            setTimeout(function () {
                callback(error);
            }, 0);
        } else {
            callback(error);
        }
    }

    function next() {
        onprogress(completed, total);

        if (tasks.length === 0) {
            if (completed === total) {
                invokeCallback();
            }
            
            return;
        }

        var task = tasks.shift();
        var settled = false;

        try {
            task(function (error) {
                if (settled) {
                    return;
                }

                settled = true;

                if (error) {
                    queueSettled = true;
                    invokeCallback(error);
                } else {
                    completed++;
                    if (!queueSettled) {
                        next();
                    }
                }
            });
        } catch (error) {
            if (settled) {
                return;
            }

            settled = true;
            queueSettled = true;
            invokeCallback(error);
        }
    }
}

I bet it's much easier that what you had thought about!

Pause and Resume

The lines have been growing, but from this section on, I'll stop updating these lines that are never tested (though I will test them some day after I complete this part, I think). :P

Pausing running tasks could be non-trivial. Before start the implementation, you could consider what you want to achieve:

• Will pausing the task queue wait for current running tasks to complete?
• Or will it abort current running tasks?

The first one is relatively easy and does not require given tasks to provide handlers to abort. But anyway the second one can fallback to the first one.

And you need to think how you want to achieve it, of course. E.g. how to expose the API?

Retry on Failures

Now we are shifting the task queue for getting the next task to run. So there's two options come to my mind:

1. Queue the failed task again.
2. Keep the retrying process in the same scope after fetching the task at the first place.

I tend to use the second approach as I see less indeterminacy on it. To implement, we can either create a sub function inside next to split the logic into get the task from task queue (next) and try (sub function). Or just use one next function to get the task from a failed execution or task queue and try.

Also we usually don't want a task being retried unlimited times, a limitation could be applied as an option.

Summary of Part 3

In this part, we added concurrency control to the task queue and discussed about some extra features. The closer to our goals, the worse we found the API we provided is. The simple runTasks function have carried too much and it certainly could be something better, e.g. a method on a task queue instance.

But we've achieved our goals anyway. Finding out problems sometimes is better than just feeling good, and do not be too lazy to face them. ;)

Continue with the previous part, in which we created a task queue that's able to execute asynchronous tasks one by one if no error occurs. Now we'll do more with it and make it really usable.

Handle Errors

In a task, we should be able to either synchronously throw an error or callback with an error as the first argument to tell something is not right.

Handle Synchronous Errors

A synchronous error means an error thrown:

1. After the function is called.
2. Before the function returns.

And obviously we'll need try...catch statement here to handle:

try {
    task(function () {
        next();
    });
} catch (error) {
    // ...
}

Handle Asynchronous Errors

An asynchronous error means an error occurs after the function returns.

To know whether there is an asynchronous error, we'll rely on the first argument of the callback passed into task. Though this is assuming that any asynchronous error will be called back as the first argument.

task(function (error) {
    if (error) {
        // ...
    } else {
        next();
    }
});

Callback with Error We Caught

We have defined the interface of runTasks function, but it could not tell whether an error happened during the execution of tasks.

Our tasks will call back with the error (if any) as the first argument. If you are familiar with Node.js, you might find it sort of a convention. So we'll apply this form here, too.

runTasks(tasks, function (error) {
    if (error) {
        console.log('An error occurs.');
        console.log(error);
    } else {
        console.log('Done without error.');
    }
});

Then we may implement some error handling logic:

function runTasks(tasks, callback) {
    next();
    
    function next() {
        if (tasks.length === 0) {
            callback();
            return;
        }
    
        var task = tasks.shift();
        
        try {
            task(function (error) {
                if (error) {
                    // Callback with asynchronous error.
                    callback(error);
                } else {
                    // Current task completes with no error,
                    // go on with the next one.
                    next();
                }
            });
        } catch (error) {
            // Callback with synchronous error.
            callback(error);
        }
    }
}

Some Details before We Moving On

One goal of this part is to make the task queue practically usable. Which means we have to avoid some issues in real life, even if they are so rare.

Callback Passed to the Task Being Called More than Once

Usually for this case, we choose to ignore any call other than the first one. To achieve this, another variable that indicates the callback status of current task could be added.

var settled = false;

task(function (error) {
    // If this callback has already been called, skip this call.
    if (settled) {
        return;
    }
    
    // Mark the current task as settled.
    settled = true;
    
    if (error) {
        callback(error);
    } else {
        next();
    }
});

Callback Passed to the Task Being Called after the Task Synchronously Throws an Error

Think it twice and we found a variant of the previous issue, so we'll also need to mark the task as settled in catch block.

Callback Passed to the Task Being Called before the Task Synchronously Throws an Error

Again, we also need to determine whether the task is already settled before we invoke the callback in catch block.

Do Not Release Zalgo

Consider code like this:

runTasks(tasks, function () {
    console.log('second');
});

console.log('first');

We are expecting "first" always been logged before "second". However without the control of how tasks behave, the code above can actually result in "second" before "first". For example, an empty tasks array or tasks that call callbacks synchronously.

To make things more predictable, let's make sure callback of function runTasks will be called asynchronously.

After some efforts, we get the code below:

function runTasks(tasks, callback) {
    // If this task queue ends synchronously, sync will still be `true`
    // when `invokeCallback` helper is called.
    var sync = true;
    next();
    sync = false;
    
    function invokeCallback(error) {
        if (sync) {
            // We can also use `setImmediate` if available.
            setTimeout(function () {
                callback(error);
            }, 0);
        } else {
            callback(error);
        }
    }
    
    function next() {
        if (tasks.length === 0) {
            invokeCallback();
            return;
        }
    
        var task = tasks.shift();
        var settled = false;
        
        try {
            task(function (error) {
                if (settled) {
                    return;
                }
                
                settled = true;
                
                if (error) {
                    invokeCallback(error);
                } else {
                    next();
                }
            });
        } catch (error) {
            if (settled) {
                return;
            }
            
            settled = true;
            invokeCallback(error);
        }
    }
}

And congratulations! Now we have a practically usable task queue!

Report Progress

Having walked through so many "troubles", progress reporting should now be a very simple feature to implement.

But before that, let's confirm the design of interface. What I want is:

1. The number of completed tasks.
2. The number of all tasks.

We may either use the same callback we used before, or another callback especially for progress. Here we'll go with the later choice, an onprogress handler.

runTasks(tasks, function (error) {
    console.log(error || 'Done.');
}, function (completed, total) {
    console.log('Progress ' + completed + '/' + total + '...');
});

And let's fill the implementation:

function runTasks(tasks, callback, onprogress) {
    // Avoid changes to this list.
    tasks = tasks.concat();
    
    // Counters.
    var completed = 0;
    var total = tasks.length;
    
    var sync = true;
    next();
    sync = false;
    
    function invokeCallback(error) {
        if (sync) {
            // We can also use `setImmediate` if available.
            setTimeout(function () {
                callback(error);
            }, 0);
        } else {
            callback(error);
        }
    }
    
    function next() {
        // Call `onprogress` handler with the information we want.
        onprogress(completed, total);

        if (tasks.length === 0) {
            invokeCallback();
            return;
        }
        
        var task = tasks.shift();
        var settled = false;
        
        try {
            task(function (error) {
                if (settled) {
                    return;
                }
                
                settled = true;
                
                if (error) {
                    invokeCallback(error);
                } else {
                    completed++;
                    next();
                }
            });
        } catch (error) {
            if (settled) {
                return;
            }
            
            settled = true;
            invokeCallback(error);
        }
    }
}

Summary of Part 2

In this part, we improved the usability of our task queue by properly handling errors and adding progress reporting. We didn't make it perfect at the very beginning, but as we think (and practice) again and again, we are making it better for dealing with real-life issues.

In the next part, we'll finally come to concurrency control. :)

It should be quite simple for a qualified JavaScript programmer to write an asynchronous task queue. If you are not sure about it, you may take this tutorial a test of your skill as well.

Goals

Sometimes in practice we need to queue specific tasks due to varieties of reasons:

• Writing a spider
• Managing Email delivering
• Resource sensitive tasks
• And so on

To deal with these real-life problems, we may want features like:

• Queuing asynchronous tasks
• Handling errors
• Reporting progress
• Concurrency number control

And some extras:

• Ability to pause and resume
• Ability to retry when a task fails

Asynchronous Tasks

An asynchronous task could be in the form of function with a callback:

function getSomethingDone(callback) {
    setTimeout(function () {
        callback();
    }, 1000);
}

Function getSomethingDone simulates an asynchronous task that takes 1000ms to complete. But how would we know if this task fails to complete at the beginning or after several hundreds of milliseconds?

1. Throw an error in the handler synchronously.
2. Callback with a value that satisfies !!value === true as the error object.

E.g., a task that fails synchronously:

function taskThatFailsSynchronously(callback) {
    throw new Error('Oops');
}

And a task that fails asynchronously:

function taskThatFailsAsynchronously(callback) {
    setTimeout(function () {
        callback(new Error('Async oops'));
        // NEVER throw an error here or any other asynchronous handlers
        // if you are not sure what's going on.
    }, 1000);
}

Before Action

Now we have the idea of how the tasks would look like, and that's "almost everything" we need to know for the implementation of the task queue.

But before that, let's think about HOW.

To make things easier, we'll start from a task queue that will run tasks one by one.

One by One

Let's not worry about the concurrency issue and start from a basic one-by-one task queue.

Fixed Tasks

When we run a fixed list of asynchronous tasks, it could be really easy and intuitive:

taskOne();

function taskOne() {
    console.log('Running task one...');
    
    // Simulates asynchronous operation by `setTimeout`.
    setTimeout(function () {
        taskTwo();
    }, 500);
}

function taskTwo() {
    console.log('Running task two...');
    setTimeout(function () {
        taskThree();
    }, 500);
}

function taskThree() {
    console.log('Running task three...');
    setTimeout(function () {
        console.log('Done.');
    }, 500);
}

Array of Tasks

Having written some fixed tasks, we can summary the pattern:

1. Start the first/next task.
2. When the current task is done.
   2.1. If it's not the last task, continue with step 1.
   2.2. If it is the last task, go to step 3.
3. Complete.

I really wish this pattern could now be longer, though we will have a longer one later.

Before actually writing the task queue, let's fabricate some tasks with no error:

var tasks = [
    function taskOne(callback) {
        console.log('Running task one...');
        setTimeout(function () {
            callback();
        }, 200);
    },
    function taskTwo(callback) {
        console.log('Running task two...');
        setTimeout(function () {
            callback();
        }, 200);
    },
    function taskThree(callback) {
        console.log('Running task three...');
        setTimeout(function () {
            callback();
        }, 200);
    }
];

To run these tasks one by one, the first thing that comes to your mind might be for loop. However, for loop is not an option.

We need a function that can start a task as described in step 1, and can be called after a task is done.

Personally I use the name next most of the time:

function next() {
    // Get and remove the first task in the queue.
    var task = tasks.shift();
    
    // Run the task.
    task(function () {
        // Will be called when this task completes.
    });
}

Now when we call function next, we'll get the next task to run. The task accepts a callback to tell when it's done. So to run the next task after the current one completes, we can simply call next again if there's still tasks in the queue:

function next() {
    var task = tasks.shift();
    
    task(function () {
        if (tasks.length > 0) {
            next();
        } else {
            console.log('Done.');
        }
    });
}

Now we got a working task queue for a queue with more than one tasks if all of them would finish without error.

To deal with an empty task queue, we could either add another if at the beginning of function next, or slightly modify our logic by moving if to the beginning:

function next() {
    if (tasks.length === 0) {
        console.log('Done.');
        // Return here to prevent subsequent lines from running.
        return;
    }

    var task = tasks.shift();
    
    task(function () {
        next();
    });
}

How to start this task queue? Well if you have to ask, call next.

Assemble The Helper

Now we can handle the simplest situation with code above, but certainly we want to get things in something that can be reused. The code is now directly logging a string "Done." when all tasks get done, which should be something else.

Imagine, we want to run some tasks and get to be notified when everything's done. An interface like this might be applicable:

runTasks(tasks, function () {
    console.log('Done.');
});

If you really understand function in JavaScript, it should be extremely easy to wrap the next stuffs into a function like what we thought of just now.

function runTasks(tasks, callback) {
    // Start everything.
    next();
    
    function next() {
        if (tasks.length === 0) {
            // Replace `console.log` with a call to `callback`.
            callback();
            return;
        }
    
        var task = tasks.shift();
        
        task(function () {
            next();
        });
    }
}

Summary of Part 1

In this part, we addressed the practical issue of queuing asynchronous tasks and listed the desired features. To get things done, we started from simplifying the needs and began the implementation with basis.

In the next part, we'll continue with error handling and progress reporting. :)

This post is out of date, please read another post for an extension which does some of the job.

Let's put simple things simple and straightforward.

1. Open file %LOCALAPPDATA%\Code\app-[version]\resources\app\client\vs\workbench\workbench.main.js.
2. Search openTerminal and locate the function that's related to "cmd.exe".
3. Replace "start","/wait" of ["/c","start","/wait"] with what you want. E.g., "C:\\Program Files\\Git\\git-bash.exe".

Done and enjoy.

Promise

Promise has become an important tool for managing asynchronous operations in JavaScript. However, sometimes we would still find promises kind of annoying.

Promise
    // Here `then` is an equivalence to `Promise.resolve().then`.
    .then(() => {
        // Start.
    })
    .then(() => {
        if (wantToBreakHere) {
            // How?
        }
    })
    .then(() => {
        // Something to skip.
    });

We may nest the following promise, but that will be painful if it's a long long chain.

Luckily we have a partial workaround for many promise implementations (as most of them have the helper catch):

class BreakSignal { }

Promise
    .then(() => {
        // Start.
    })
    .then(() => {
        if (wantToBreakHere) {
            throw new BreakSignal();
        }
    })
    .then(() => {
        // Something to skip.
    })
    .catch(BreakSignal, () => {
        // Use catch method to filter BreakSignal.
    });

This approach is somewhat gentle, but if you have other catches (rejection handlers) in this chain, you will have to relay BreakSignal manually.

Consider another situation than "break":

page.on('load', () => {
    Promise
        .then(() => asyncMethodA())
        .then(result => asyncMethodB(result))
        .then(result => {
            // Update something...
        });
});

If the load event fires twice in a short time, we'll need to cancel the first promises chain, or at least to prevent it from updating related data or view (If this is the desired behavior).

Again, the workaround:

class BreakSignal { }

let context;

function createWrapper() {
    let currentContext = context;
    
    return function (handler) {
        return function () {
            if (context !== currentContext) {
                throw new BreakSignal();
            }
            
            return handler.apply(undefined, arguments);
        };
    };
}

page.on('unload', () => {
    context = undefined;
});

page.on('load', () => {
    context = {};
    let wrap = createWrapper();

    Promise
        .then(wrap(asyncMethodA))
        .then(wrap(asyncMethodB))
        .then(result => {
            // Update something...
        })
        .catch(BreakSignal, () => { });
});

ThenFail

After these mess, I come up adding the ability to break or cancel a promises chain in my own promise implementation ThenFail.

Promise
    .then(() => {
        // Start.
    })
    .then(() => {
        if (wantToBreakHere) {
            Promise.break;
        }
        
        return Promise
            .then(() => {
                Promise.break;
            })
            .then(() => {
                // Never reaches here.
            });
            // No need to enclose nested context.
    })
    .then(() => {
        // Something to skip.
    })
    // Enclose current context to prevent from breaking too many.
    // This is important if the promise chain might be used by code out of your control.
    .enclose();

Actually you may also "break" each of ThenFail:

Promise
    .each([1, 2, 3], value => {
        if (value > 1) {
            Promise.break;
            
            // Or asynchronously:
            return Promise
                .then(() => {
                    // Do some work.
                })
                .break;
        }
    })
    .then(completed => {
        if (completed) {
            // Some code...
        }
    });

I've mentioned the concept of context in ThenFail. And to cancel an entire promises chain (of the same context), you just need to dispose the context.

let context;

page.on('unload', () => {
    context.dispose();
});

page.on('load', () => {
    let promise = Promise
        .then(() => asyncMethodA())
        .then(result => asyncMethodB(result))
        .then(result => {
            // Update something...
        });
    
    context = promise.context;
});

BTW, disposing a context will also dispose nested contexts. :D

Errors related to this post:

• COMException (0x80010001): Call was rejected by callee.
• COMException (0x8001010A): The message filter indicated that the application is busy.
• COMException (0x800A1735): The requested member of the collection does not exist.
• COMException (0x800A03EC): Cannot access the file xxx.
• Maybe more.

My recent project requires the ability to convert PowerPoint and Word files into images. Our first solution is to use LibreOffice (PowerPoint/Word to PDF) and ImageMagick (PDF to images), but soon we found LibreOffice sometimes could be unreasonably slow, and soffice would hang if there were multiple instances of it at the same time.

Then I realize we could try Office Interop API, after all it's official.

It went kind of smoothly when I started to write a command line tool on my own computer (using C#). The code was actually like what you would get searching related keywords:

// Here's the code of exporting a Word file into images.
// As it's longer (comparing to PowerPoint version).

var app = new Word.Application();

var document = app.Documents.Open(inputFile);
var window = document.ActiveWindow;
var pane = window.ActivePane;
var pages = pane.Pages;

for (var i = 1; i <= pages.Count; i++) {
    try {
        var page = pages[i];

        using (var stream = new MemoryStream((byte[])(page.EnhMetaFileBits))) {
            var image = System.Drawing.Image.FromStream(stream);
            image.Save(String.Format("{0}.png", i));
        }
    } catch (COMException e) {
        Console.WriteLine(e);
    }
}

app.Quit();

Then I found it exported only the first page of the Word file. After some efforts, I found adding app.Visible = true; before open documents solved this issue (which is NOT part of the solution).

Later I integrated this tool with our Node.js server happily and was hoping to enjoy a break.

Wait, Node.js? You are not even saying ASP.NET? Actually the main issue I met (and probably you met as you are reading this post) is not about a specific language, but about permission and identity.

But from this line on, let's go to the complete solution directly.

1. I am using LocalSystem as application pool identity, and I haven't tested other identities.
2. Create folder Desktop under C:\Windows\System32\config\systemprofile\, and if you are running on a 64-bit Windows, you might also need to create one under C:\Windows\SysWOW64\config\systemprofile\.
3. Open related Office apps and dismiss all prompts. Use Office apps as default apps, or you might need step 4.
4. (Optional if everything works fine) Download PSTools on your server and use it to start Word/PowerPoint with LocalSystem identity and ensure you dismissed all prompts (e.g. default program prompt).
5. Besides the identity issue we addressed, we still have the app.Visible = true; to refine. Change how you get a page as below (and we don't need app.Visible = true; any more):

Word.Page page;

while (true) {
    try {
        // Load this page manually.
        window.Selection.GoTo(Word.WdGoToItem.wdGoToPage, Type.Missing, i, Type.Missing);
        page = pages[i];

        break;
    } catch (Exception e) {
        // If not error "The requested member of the collection does not exist."
        if ((uint)e.HResult != 0x800A1735) {
            throw;
        }

        Thread.Sleep(500);
    }
}

Now you should have a go. ;)

Months ago I updated my old blog to WordPress 4.2.1, and it got into some problems and no longer function. Finally I fixed it yesterday and found it might be some incompatibility with my 5-year-old theme.

Yeah, time flies.

Weeks ago I started looking for a new job, which I hope I will love. And of course some HR asked if I write blogs. I said yes but my blog was down.

It was down for months. And I was actually okay with that.

I wrote less and less posts these years, not because I didn't want to share my thoughts any more, but because I was not able to find things that I believe valuable to share.

Years ago I wrote posts almost everyday, recording my life and the progresses learning new things.

Then I stopped. I started to think and reflect much more on myself, and try to become a wise person:

I gave myself much more restrictions and made myself smaller and smaller.

Still, I don't know whether this is the best path. Though I believe:

Every path is the right path. Everything could've been anything else. And it would have just as much meaning.

Mr. Nobody

Years went by, I am no long who I used to be. It's good, it's bad. I would sometimes doubt the posts written years ago: Was I really that naive back to the age? Or on the other hand: What a genius I was.

So I decided to start at new place with nothing from my old blog. I still haven't get the hint of what I could share, but soon or later, more or less, time will tell.