black@t

Angular 8 Bootstrap in deep

2019-12-22T00:00:00+00:00

The Angular bootstrap sequence starts after the load of the index.html and the JavaScript bundles produced by Webpack. Angular Runtime creates the platform where the application can be started and finally the root component is rendered.

What follows is related to Angular View Engine and experimented on Angular 8.2.9.

View Engine is going to be replaced by Angular Ivy in version 9, detail introduction can be found in this blog post.

Disclaimer The post contains the thoughts of a preliminary investigation on how Angular works reading the some parts of the source code, debugging a simple application and reading how the compiler works. Some terms or definition could be wrong.

Slides

This post comes along with a presentation written in markdown, rendered via reveal.js and available on GitHub.

Lingo

Angular View Engine: Angular rendering architecture (compiler and runtime) introduced in version 4 and substituted with Ivy in Angular version 9.
Angular compiler: compiles templates and decorators into a code the runtime can execute.
Angular runtime: execute the JavaScript code produced by the compiler to run the application.
Object Model (OM): a way to model via object-oriented techniques (objects, classes, interfaces, properties, inheritance, encapsulation, etc) a system for development purpose. For instance Apache POI implements a OM of Microsoft Excel that manipulates via a Java API.
Data Model (DM): it represents entities at database level, it deals with table schema, relationships between tables (FKs, PKs) but not advanced object-oriented concepts like inheritance or polymorphism. DM represents how OM classes are stored in a database.
DOM: an object oriented representation of a HTML document in a tree fashion that can be manipulated via the DOM API, for instance HTMLButtonElement is one of the DOM interfaces.
Shadow DOM: it allows to separate DOM into smaller and encapsulated object-oriented representations of a HTML element.
Tree and nodes: the DOM is organized in a logical tree where its nodes are the components or HTML elements.
Rendering/painting: the browser process that transform the DOM into the UI.
Virtual DOM: the virtual representation of the real DOM.
Diffing: operation that compare two Virtual DOM.
Incremental DOM: a technique to render and update an Angular component when change detection is fired.

The plan

Hello Reader, this is a long post so feel free to skip certain sections I have used to introduce and give a more complete context to the Angular bootstrap sequence that is the goal :bowtie:

The post starts with an introduction about the DOM and two rendering strategies used to speed up the page repainting. The incremental DOM strategy is the base of the Angular rendering architecture.

The Welcome to Angular simple application will help to introduce and talk about the Angular compiler, why and how the Angular declarative syntax is transformed into JavaScript code that can be executed by the Angular runtime in the browser. A deep look into the generated code and the Angular source code will show how the framework creates the DOM element and answer to change detection.

Some of the content and mechanisms has been changed with the introduction of the new rendering architecture called Angular Ivy.

Browser DOM

The Document Object Model (DOM) connects web pages to scripts or programming languages by representing the structure of a document such as the HTML representing a web page in memory.

MDN - Mozilla Developer Network

Tip The HTML document is represented in object-oriented fashion, as objects in a logical tree, by the DOM that also provides the API to manipulate those objects.

The DOM rendered gives the HTML page visible to the end user.

DOM rendering is slow

Being the DOM represented as a tree, it makes easier to change and update it. What the user sees is the result of the DOM rendering operation that is the slow part. More a page or a component is complex more could take time to render it.

A page is usually made of many components, complex and non-complex. Every time one of them changes the all page (or big part of it) needs to be re-rendered, a really expensive operation.

Tip Frequent DOM manipulations make the user interface slow since the re-painting of the user interface is the most expensive part. In general, it is something that is not considered when the page is going to be implemented. For instance changing visibility of an element forces the browser to verify/check visibility of all other dom nodes.

Actions like changing visibility or the background of an element trigger a repaint. A simple click of the user could correspond to many different actions behind the scene and so many repainting actions slowing down the web page.

Two different techniques have been developed to overcome the rendering issue for complex web applications: Virtual DOM and Incremental DOM.

Virtual DOM

The key idea is to render the DOM the least possible. When a change detection occurs, instead of updating the real DOM, frameworks like React updates a Virtual DOM.

The Virtual DOM is a tree as well, made of nodes that are the page elements. When a new element is added/removed a new Virtual DOM is created, the difference between the two trees is calculated.

A transformations series is calculated to update the browser DOM so that it matches the latest new Virtual DOM. These transformations are both the minimal operations to be applied to the real DOM and the ones that reduce the performance cost of the DOM update.

Internals The rendering process happens only on the difference. The bulk changes to be applied are optimized to improve the performance cost.

What Virtual DOM looks like

The Virtual DOM is something not official, no specification is provided differently from DOM and shadow DOM.

It is a copy of the original DOM as a plain JavaScript object (JSON) so that it can be modified how many times we want without affecting the real DOM. The Virtual DOM can be divided in chunks so that it is easier to diffing the changes.

Example

When a new item is added to an unordered list of elements a copy of the Virtual DOM containing the new element is created.

The diffing process collects the differences between the two Virtual DOM objects so changes can be transformed in a bulk update against the real DOM.

Tip No distinction about reflow (element layout that is position recalculation and geometry) and repaint (element visibility) has been done so far since most of the considered actions involve the repaint operation.

How React uses the Virtual DOM

In React a user interface is made of a set of components, each component has a state. For instance the state of a drop-down menu is the array of the available elements and the current selected one.

Via the observer pattern React listens to state change to update the Virtual DOM. The diffing process makes React aware of which Virtual DOM objects have changed, only those objects will be updated in the real DOM.

Tip As developer you don’t need to be aware about how DOM manipulation happens at each state change. React does the job optimizing the performance cost behind the scenes.

React reduces the re-painting cost applying updates in bulk not at every single state change.

The great advantage of using the Virtual DOM is that we don’t need any compiler. JSX for instance, is really close to JavaScript, the key point is the render function that can be implemented using any programming language.

Virtual DOM drawbacks

The Virtual DOM required an interpreter to interpret the component. At compile time no way exists to know which parts of the interpreter will be required at runtime, so the whole stuff has to be loaded by the browser.
Every time there is a change, a new Virtual DOM has to be created, may be a chunk and not the whole tree, but the memory footprint is high.

Incremental DOM

The key idea of the incremental DOM is:

Every component gets compiled into a series of instructions. These instructions create DOM trees and update them in-place when the data changes.

Victor Savkin - Understanding incremental DOM

Each component is then compiled into two main instruction sequences:

view creation: invoked the first time the page is rendered, add the component to the DOM;
change detection: invoked at every state change to update the component into the DOM.

The advantages of the Incremental DOM are a low memory footprint and a skinny interpreter/runtime tailored on the compiled application.

Angular Ivy The Incremental DOM strategy is already present in the Angular View Engine. As it will be shown, each component is compiled into a creation function and an update function. Angular Ivy goes further, it allows the tree-shaking of the Angular runtime that is not possible with the current rendering architecture.

Angular compiler

An Angular application is mainly made by Angular components organized in a tree fashion. Each component is implemented to accomplish a certain mission, for instance the navigation bar, the drop-down menu, etc.

Angular component

An Angular component is characterized by a class, TypeScript code that expresses the logic, and a decorator that allow to define some metadata such as the selector, the template, etc. The HTML template represents the presentation layer of the component and it is implemented using a specific Angular declarative syntax.

Tip When the developer write a component uses TypeScript and the Angular declarative syntax for the template to bind a variable from the logic to the presentation layer and vice versa. Pay attention that no change detection is required to be added. Change detection works at runtime thanks to the compiler that adds it during the compilation phase.

Example

Consider a very simple component, template can be inline or separated:

import { Component } from '@angular/core';

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css']
})
export class AppComponent {
  title = 'a simple component';
}

A template is a bunch of HTML code with binding variables to present, with a certain look and feel, some content.

<div style="text-align:center">
  <h1>
    Welcome to {{ title }}!
  </h1>
</div>

Browser cannot render an Angular component

The browser is the execution environment, it loads the application and executes it. Unfortunately it cannot execute an Angular component as it is.

Tip A browser can interpret JavaScript and render HTML but not if written using the Angular declarative syntax. Angular provides a compiler that, together with the TypeScript one, transforms “everything in something else” that a browser can understand.

During the build of an Angular project, two compilers come into play with different purposes:

tsc is the TypeScript compiler and generates the JavaScript w.r.t. the target specified in the tsconfig.json, for instance target: es2015.
ngc is the Angular compiler that translates the templates and decorators into JavaScript. The Angular compiler can work in two different modes:
- Ahead-of-Time (AoT): work at build time so that the templates are bundled along with the application, suitable for production.
- Just-in-Time (JIT): templates are not pre-compiled, the compiler comes along with the application, it is loaded by the browser and does the work at runtime, suitable for development.

Internals During the development phase ng serve provides live reload functionality. The process goes through @ngtools/webpack, compiled code is not saved to disk, everything is consumed in memory via streams and emitters.

Angular vs. browser

What are then the roles of the browser and Angular?

Once the Angular application has been fully transformed into JavaScript (HTML templates included), WebPack bundles it along with library dependencies in order to improve performance and load times.

Browser

The browser role is to load the index.html and to provide the execution environment, the render and the event loop.

<!DOCTYPE html>
<html lang="en">
  <head>
    ...
  </head>
  <body>
    <app-root></app-root>

    <script src="runtime-es2015.js" type="module"></script>
    <script src="polyfills-es2015.js" type="module"></script>
    <script src="styles-es2015.js" type="module"></script>
    <script src="vendor-es2015.js" type="module"></script>
    <script src="main-es2015.js" type="module"></script>

    <!-- nomodule defer -->
    <script src="runtime-es5.js" nomodule defer></script>
    ...
  </body>
</html>

The scripts can be loaded both by modern browsers that supports ESM modules and by old ones that do not support modules via nomodule defer attributes.

Angular

Consider an Angular application made of only the component previously introduced. The main-es2015.js contains the fully bundled application while runtime-es2015.js is the Angular runtime. Finally third party libraries and styles.

Tip The transformed HTML template into JavaScript becomes a series of instructions that, once called, render the page building the components. Skipping some details, roughly an element is a factory function that uses the injected Angular renderer to render the element w.r.t. the browser platform.

The Angular runtime bootstraps the AppModule that, in turn, creates and renders the root element of the application <app-root>. The file main-es2015.js contains the view definition factories produced by the compiler and enriched by Webpack.

Internals If the browser platform is chosen, @angular/platform-browser, the element will be rendered creating the HTML code into the DOM via the Document interface: document.createElement(). When something changes, the element will update itself calling the update function.

Angular Ivy The compilation process of View Engine produces .metadata.json and .ngfactory.js files. With Angular Ivy no more special files are produced, too complex to manage and to merge them. Ivy instructions are directly put into the component, a component knows how to create and update itself.

Analyze the compiled code

Let’s see how to compile the application invoking only the ngc compiler and nothing else to inspect the compiled code easily and see where the generated JavaScript code invoke the DOM API to create the element.

Tip The HTML template has been compiled into a sequence of JavaScript instructions that will be executed by the Angular runtime. The goal of the coming sections is to find where the document.createElement() is invoked after the different Angular entities (platform, application and component) have been instantiated.

Setup the compilation task

Open the package.json file and add:

"scripts": {
  ...
  "compile": "ngc"
},

then in the tsconfig.json enable the d.ts files generation to have the TypeScript definitions:

"compilerOptions": {
  ...
  "declaration": true,
  ...
}

One and simple component

Create a new Welcome to Angular application via the Angular CLI.

The module and the component

The module is as follows:

@NgModule({
  declarations: [
    AppComponent
  ],
  imports: [
    BrowserModule
  ],
  providers: [],
  bootstrap: [AppComponent]
})
export class AppModule { }

then the component of the Welcome to Angular application:

@Component({
  selector: 'app-root',
  template: `
    <div style="text-align:center">
      <h1>
        Welcome to {{ title }}!
      </h1>
    </div>
  `,
  styleUrls: []
})
export class AppComponent {
  @Input() title = 'Angular';
}

Compile

Run the command npm run compile and look into the folder dist/out-tsc/src/app where everything has been transformed into JavaScript and saved to disk.

The Angular compiler has produced some files, skip the .metadata and .d.ts ones:

app.module.js               // module class
app.module.ngfactory.js     // module factory, transformed metadata decorator
app.component.js            // component class
app.component.ngfactory.js  // component factory, transformed metadata decorator

Module factory function

The app.module.ngfactory.js contains the factory function creator:

import * as i0 from "@angular/core";
import * as i1 from "./app.module";
import * as i2 from "./app.component";
import * as i3 from "./app.component.ngfactory";
...
var AppModuleNgFactory = i0.ɵcmf(i1.AppModule, [i2.AppComponent], function(_l) {...}
...

Warning The functions produced by the Angular template compiler start with ɵ to clearly warn to not use them because for sure the code will change soon in the future.

The function ɵcmf stands for create module factory, the map between the name and the real function is defined in the following static map object Map<ExternalReference, any>:

function createBuiltinExternalReferencesMap() {
  const map = new Map<ExternalReference, any>();
  ...
  map.set(Identifiers.createModuleFactory, ɵcmf);
  ...
  return map;

Angular Ivy The aforementioned map object is one of the reason why the View Engine is not tree-shakable. Angular Ivy should get rid off or change how this static map is defined to allow the runtime to be tree-shaked by the any open source tool.

What is going to happen

The compiler has transformed the decorators, @NgModule and @Component, into JavaScript instructions. Now “imagine” that the TypeScript class has been transpiled into JavaScript and the @Component decorator that decorates the class became the factory that tell Angular runtime how to create the component into the DOM (create view) and how to update it (change detection). The @NgModule decorators will tell the Angular runtime how to instantiate the application module and get service providers injected.

The module factory function will create an application object that, in turn, will bootstrap the application module and finally the root component.

Module factory implementation

The module factory function ɵcmf creates the module factory object AppModuleNgFactory previously shown. here the implementation:

export function createNgModuleFactory(
    ngModuleType: Type<any>, bootstrapComponents: Type<any>[],
    defFactory: NgModuleDefinitionFactory): NgModuleFactory<any> {
      return new NgModuleFactory_(ngModuleType, bootstrapComponents, defFactory);
    }

it implements the following interface:

export abstract class NgModuleFactory<T> {
    abstract get moduleType(): Type<T>;
    abstract create(parentInjector: Injector|null): NgModuleRef<T>;
}

Module creation

The module factory object can create a module of type AppModule defined in the class app.module.js, that will bootstrap a component of type AppComponent defined in the file app.component.js.

The defFactory is a module defintion function, ɵmod, used by the create method to produce the real module object. It contains an array of ɵmpd module provider definitions that, for instance, tell which sanitizer or producer has to be created and injected:

...
var AppModuleNgFactory = i0.ɵcmf(i1.AppModule, [i2.AppComponent], function(_l) {
  return i0.ɵmod([
    ...
    i0.ɵmpd(4608, i5.DomSanitizer, i5.ɵDomSanitizerImpl, [i4.DOCUMENT]),
    i0.ɵmpd(6144, i0.Sanitizer, null, [i5.DomSanitizer]),
    ...
    i0.ɵmpd(6144, i0.RendererFactory2, null, [i5.ɵDomRendererFactory2]),
    ...
  ]
}

Component factory function

Open app.component.ngfactory.js and look at ɵccf or create component factory function:

import * as i1 from "@angular/core";
import * as i2 from "./app.component";
...
var AppComponentNgFactory = i1.ɵccf(
  "app-root",
  i2.AppComponent, /* class or type */
  View_AppComponent_Host_0, /* factory that produces the app-root, component host, the node defintion */
  {},
  {},
  []
);

it is defined as follows:

export function createComponentFactory(
    selector: string, componentType: Type<any>, viewDefFactory: ViewDefinitionFactory,
    inputs: {[propName: string]: string} | null, outputs: {[propName: string]: string},
    ngContentSelectors: string[]): ComponentFactory<any> {
  
  return new ComponentFactory_(
      selector, componentType, viewDefFactory, inputs, outputs, ngContentSelectors
    );
}

The factory function is similar to the module one except for some more parameters. A component can have @Input() and @Output properties and hence the arrays inputs and outputs.

Tip It starts to be more and more clear how the component declaration is transformed into a set of arguments used by a factory to programmatically create the component at runtime.

Compiled template

What happened to template? This is why you have read so far… I hope :sweat_smile:

The component template has been transformed into a JavaScript object with the following interface:

export interface ViewDefinition extends Definition<ViewDefinitionFactory> {
  flags: ViewFlags;
  updateDirectives: ViewUpdateFn;
  updateRenderer: ViewUpdateFn;
  handleEvent: ViewHandleEventFn;
  nodes: NodeDef[];
  nodeFlags: NodeFlags;
  rootNodeFlags: NodeFlags;
  lastRenderRootNode: NodeDef|null;
  bindingCount: number;
  outputCount: number;
  nodeMatchedQueries: number;
}

The view definition ɵvid with the app-root host selector:

export function View_AppComponent_Host_0(_l) {
  return i1.ɵvid(
    0,
    [
      (_l()(),
        i1.ɵeld(
          0,0,null,null,1,"app-root",[],null,null,null,
          View_AppComponent_0,RenderType_AppComponent
        )),
      i1.ɵdid(1, 49152, null, 0, i2.AppComponent, [], null, null)
    ],
    null,
    null
  );
}

Host selector since the component is attached/hosted by the selector, the Angular component is a directive, hence the view definition is characterized by (links point to the Angular source code on GitHub):

element definition, ɵeld, the app-root, the function produces an ElementDef;
directive definition, ɵdid, the directive that represents the component, the function directiveDef produces an object of type NodeDef.

Both produced objects are of type NodeDef.

The element definition ɵeld references then View_AppComponent_0, the other JavaScript code that represents the component template:

export function View_AppComponent_0(_l) {
  return i1.ɵvid(0,
    [
      (_l()(),
      i1.ɵeld(0, 0, null, null, 1, "h1", [], null, null, null, null, null)),
      (_l()(), i1.ɵted(1, null, ["Welcome to ", "!"]))
    ],
    null,
    function(_ck, _v) {
      var _co = _v.component;
      var currVal_0 = _co.title;
      _ck(_v, 1, 0, currVal_0);
    }
  );
}

The ɵvid, viewDef function, takes two view update functions: updateDirectives and updateRenderer for the change detection along with the function to create the element the first time the application is loaded.

Internals

In a view defintion function ɵvid there are two interesting things:

NodeDef[] array of nodes that is responsible of the view creation;
updateDirectives and updateRenderer functions responsible of the change detection update.

Angular Ivy In Angular Ivy there are no more .ngfactory.js files, all the required code for view creation and change detection is inside the component. Imho the incremental DOM if fully implemented in Ivy, what is missing in View Engine is the possibility to tree-shake the runtime to squeeze it as much as possible.

How Angular application bootstraps

Once the compiled code has been analyzed, it is interesting to see the call sequence to the Angular runtime to find which function renders the component. At the end of the sequence there must be the sought after document.createElement() function call to the DOM API.

Build the application and start a live server to debug it into the browser:

ng build --aot
npx http-server dist/test-ivy

The Angular AOT compiler extracts metadata to interpret the parts of the application that Angular runtime is supposed to manage.

Basically the compiler manages metadata interpretation and template compilation that can be controlled specifying some template compiler options in the tsconfig.json.

Angular Ivy Activate the Ahead-of-Time compilation to have everything in JavaScript and saved to disk make it easier to inspect the generated code. With Angular Ivy --aot is not necessary anymore since it is activated by default. Ivy compilation is so fast that AoT compilation can be always used.

0. IIEF

The application starts in the file main-es2015.js. The option --aot contributes to some optimizations, bootstrapModule is replaced by bootstrapModuleFactory as you can observe from the file main-aot.ts:

import { platformBrowser } from '@angular/platform-browser';

import { AppModuleNgFactory } from './app.module.ngfactory';

// *** Follow bootstrapModuleFactory() ***
platformBrowser().bootstrapModuleFactory(AppModuleNgFactory);

Pay attention: in each piece of code there is a comment that allows to follow the bootstrap call sequence // *** Follow.

Tip When invoking the ng build and not simply the compiler as done before, Webpack bundles what has been produce by the compiler, so opening the files results in a slightly different code.

Basically the IIEF function bootstraps the platform PlatformRef, that, in turn, instantiates the application ApplicationRef and then the module along with all the required injectable providers. Finally the component is created and rendered into the DOM.

Internals The application code is composed by app.module.ts and app.component.ts. First Angular runtime has to be started, then it creates the platform linked to the page, starts the application that is the module. Once the module has been started the component can be instantiated and rendered.

1. Platform

The Angular platform PlatfromRef is the entry point for Angular on a web page. Each page has exactly one platform and services bound to its scope. A page’s platform is initialized implicitly when a platform is created via a platform factory (e.g. platformBrowser).

class PlatformRef {
    ...
    /**
     * Creates an instance of an `\@NgModule` for the given platform
     * for offline compilation.
     */
    bootstrapModuleFactory(moduleFactory, options) {
      // Note: We need to create the NgZone _before_ we instantiate the module,
      ...
      return ngZone.run((
        const ngZoneInjector = Injector.create(
          {providers: providers, parent: this.injector, name: moduleFactory.moduleType.name});

        // from here the ApplicationRef is created and available to be injected
        const moduleRef = InternalNgModuleRef<M>moduleFactory.create(ngZoneInjector);
        ...
        // *** Follow _moduleDoBootstrap() ***
        // moduleType: *class AppModule*
        this._moduleDoBootstrap(moduleRef);
        return moduleRef;
        ...
      ));
    }
    ...
    /**
     * Bootstrap all the components of the module
     */
    _moduleDoBootstrap(moduleRef) {
      ...
      const appRef = moduleRef.injector.get(ApplicationRef) as ApplicationRef;
      ...
      // loop over the array defined in the @NgModule, bootstrap: [AppComponent]
      moduleRef._bootstrapComponents.forEach((
        // *** Follow bootstrap() ***
        // bootstrap the root component *AppComponent* with selector *app-root*
        f => appRef.bootstrap(f)));
      ));
    }
}

Basically change detection is managed by Zone.js that run the module bootstrap. ApplicationRef reference is created and then it bootstraps the AppComponent component.

2. Application

The ApplicationRef reference represents an Angular application running on a page.

  class ApplicationRef {
      ...
      /**
       * Bootstrap a new component at the root level of the application.
       * When bootstrapping a new root component into an application, Angular mounts the
       * specified application component onto DOM elements identified by the componentType's
       * selector and kicks off automatic change detection to finish initializing the component.
       */
      bootstrap(componentOrFactory, rootSelectorOrNode) {
        ...
        /**
         * Use the componentFactory to create the root element app-root having this information:
         * componentType: class AppComponent
         * viewDefFactory: View_AppComponent_Host_0()
         * selector: app-root
         */
        // *** Follow create() ***
        const compRef = componentFactory.create(Injector.NULL, [], selectorOrNode, ngModule);
        ...
      }
  }

3. Root component

Build the root component:

  class ComponentFactory_ extends ComponentFactory {
    ...
    create(injector, projectableNodes, rootSelectorOrNode, ngModule) {
      const view = Services.createRootView(injector, projectableNodes || [], rootSelectorOrNode, viewDef, ngModule, EMPTY_CONTEXT);
    }
  }

Basically the Angular component_factory.ts holds the base class method to create a component of a certain type:

class ComponentFactory_ extends ComponentFactory<any> {
  
  viewDefFactory: ViewDefinitionFactory;

  /**
   * Creates a new component.
   */
  create(
      injector: Injector, projectableNodes?: any[][], rootSelectorOrNode?: string|any,
      ngModule?: NgModuleRef<any>): ComponentRef<any> {
    if (!ngModule) {
      throw new Error('ngModule should be provided');
    }
    const viewDef = resolveDefinition(this.viewDefFactory);
    const componentNodeIndex = viewDef.nodes[0].element !.componentProvider !.nodeIndex;
    // *** Follow createRootView() ***
    const view = Services.createRootView(
        injector, projectableNodes || [], rootSelectorOrNode, viewDef, ngModule, EMPTY_CONTEXT);
    const component = asProviderData(view, componentNodeIndex).instance;
    if (rootSelectorOrNode) {
      view.renderer.setAttribute(asElementData(view, 0).renderElement, 'ng-version', VERSION.full);
    }

    return new ComponentRef_(view, new ViewRef_(view), component);
  }
}

Basically the implementation uses the function resolveDefinition() to load the view definition. This function will be used many times around the code. The createRootView() function creates a ViewData object that contains the information that will be used later on to render the node into the DOM.

4. Create nodes

The code is going to arrive to the point where the DOM API is called to create and attach the element to the DOM.

function createRootView(root, def, context) {
  const view = createView(root, root.renderer, null, null, def);
  initView(view, context, context);
  // *** Follow createViewNodes() ***
  createViewNodes(view);
  return view;
}

the function function createViewNodes(view: ViewData){...} creates a DOM element w.r.t. its type:

function createViewNodes(view) {
  const nodes = view.nodes;
  for (let i = 0; i < def.nodes.length; i++) {
    switch (nodeDef.flags & 201347067 /* Types */) {
      case 1 /* TypeElement */:
        // H1 DOM element of type any, the function calls the DOM renderer to render the element
        // *** Follow createElement() ***
        const el = (createElement(view, renderHost, nodeDef)));
        ...
        // View_AppComponent_0()
        const compViewDef = resolveDefinition(((nodeDef.element)).componentView)));
        ...
        break;
      case 2 /* TypeText */:
        ...
        break;
      ...
    }
  }
}

5. The renderer

The createElement function will use the injected renderer to create the element.w.r.t. the platform where the application runs.

In case of PlatformBrowser, the DefaultDomRenderer2 class invokes the document interface method to create the real DOM element. DefaultDomRenderer2 extends and implements abstract class Renderer2.

createElement(name: string, namespace?: string): any {
    if (namespace) {
      // In cases where Ivy (not ViewEngine) is giving us the actual namespace, the look up by key
      // will result in undefined, so we just return the namespace here.
      return document.createElementNS(NAMESPACE_URIS[namespace] || namespace, name);
    }

    // *** FOUND ***
    return document.createElement(name);
  }

Tip An HTML template is transformed in an intermediate format or Object Model by the Angular compiler. Factory functions are automatically generated by the compiler and they are able to produce an object that can create a component or a node or a module. Then a renderer, specified by the chosen platform, will produce DOM elements in case of a DOM renderer.

Conclusions

It has been shown how the Angular compiler transforms the Angular declarative syntax and the decorators into something the Angular runtime can execute. The Angular compiler and the runtime constitute the rendering architecture.

A developer can use a simple syntax without worrying about change detection and performance optimization w.r.t. the DOM updates since the Angular framework, behind the scenes, does all the job. When new optimizations are available can be got transparently and effortlessly.

One of the big issue with the current rendering architecture, View Engine, is to not be tree-shakable and difficult to expand. Angular Ivy will solve all these issues being composed by an instruction set that can be easily expanded and tree-shaken to avoid the delivery of the full Angular runtime to the browser as today.

References

DOM

Virtual DOM

Angular compiler

Incremental DOM and Ivy

Zone

I reverse-engineered Zones (zone.js) and here is what I’ve found

Angular 9 compiler

2019-12-22T00:00:00+00:00

Why does Angular have a compiler at all? Main job of the compiler is to turn the template you write into the code that runs the Angular runtime.

Alex Rickabaugh - Angular Connect 2019

In Angular, the developer write templates declaratively that is what to render, the binding, etc. but not how it happens at runtime, there is not description about how the change detection mechanism works. The Angular runtime does not understand the declarative template syntax so it has to be translated into something the runtime can run into a browser.

The compiler takes the declarative Angular syntax and turns it into imperative code.

Why, again, a compiler is required? not possible to do by hand?

Lot of boilerplate code, better to focus on the business logic than how things work under the hood.
Angular team can optimize and improve by release w.r.t. browser evolution the imperative code.

Lingo

Angular Compiler: part of the Angular rendering architecture that transforms templates and decorators into something that the Angular runtime can understand.
Angular Runtime: part of the Angular rendering architecture that runs an Angular application.
Angular template declarative syntax: describes what the view has to look like, what the view has to display, but not how.
Angular application imperative code: describes how the view has to be rendered via a sequence of JavaScript instructions/commands.
Metadata: set of data enclosed into a decorator that describes the entity represented by the decorator itself. For instance a component has the selector, the template, etc. Metadata are then reused by the compiler and put into the definition files .d.ts, the component API. Basically the metadata help to preserve the information removed from .js files.

Quick overview

The work done on the new Angular compiler can be divided in three categories as stated in the implementation status:

@angular/compiler-cli: TypeScript transformer pipeline which includes two CLI tools
- ngtsc: the Angular TypeScript compiler which looks for the Angular decorators like @Component and substitute them with their specific Angular Runtime instructions/counterparts like ɵɵdefineComponent.
- ngcc: the Angular Compatibility Compiler which converts pre-ivy modules into ivy-module, can be even run as part of a code loader like Webpack to have packages from node_modules transpiled on-the-fly.
@angular/compiler: Ivy Compiler which converts decorators into Ivy.
@angular/core: decorators which can be converted by the @angular/compiler.

Angular Ivy compiler model

The Ivy model foresees to compile the Angular decorators like @Injectable, @Component, etc into static properties.

All the decorators do not need global knowledge of the application, except @Component which requires information coming from @NgModule. In the module other selectors used by the component template are declared and so the transitive closure of the exports of that module imports.

Without those information the component def (ɵcmp) cannot be correctly generated.

Consider the Welcome to Angular! application:

@Component({
  selector: 'app-root',
  template: `
    <div style="text-align:center">
      <h1>
        Welcome to {{ title }}!
      </h1>
    </div>
  `,
  styleUrls: []
})
export class AppComponent {
  @Input() title = 'Angular';
}

The previous code is characterized by decorators like @Component and @Input, part of the Angular template declarative syntax that provides developers with an easy template grammar to both

write control flow statements, like the for loop and if conditional statement,
and make data binding between the variables declared in the controller and used in the templates.

Tip For most of the developers the binding is just the use of the same variable name between the template and the controller, but there is a mechanism that manages the change detection at runtime. This is a great thing and the Angular compiler automatically adds this mechanism. Less boilerplate code, templates are mode readable and less error prone.

The Angular runtime is a collection of JavaScript instructions/functions able to render a component template into the DOM and to answer to change detection when something in the model has changed (MVC). Everything must be in JavaScript, ready to be run, so the Angular declarative syntax is translated into these instructions.

To make it more clear try to develop a standard Web Component: when the developer has to deal with the template he must deal with the DOM API to create an element and attach it to the DOM, write some code to detect changes in the model and update the view. In the Angular code there is not trace of these operations since a template is translated into JavaScript imperative code, a series of JavaScript instructions, part of the Angular runtime, that, when invoked, creates the component in the browser.

Most of the tedious and repetitive job is done by the Angular compiler in conjunction with the Angular runtime.

Compiler enables decoupling

The developer just writes the Angular modules and components, that is the what, via the Angular declarative syntax but it does not know/care about the how, that is how things are executed at runtime: compiler enables to decouple the what from the how.

This approach has multiple advantages:

minimize or eliminate side effects making the developer life simpler;
if the Angular rendering architecture evolves, small or null changes are required to use the latest version;
browsers change more and more often as the EcmaScript hence the how the templates are rendered can change accordingly optimizing web performance for instance;
templates and decorators can be compiled differently w.r.t. the platform where the code is run such ES5 or ES6 with module support.

Simple project setup

Angular 9 has been recently released, generate a simple project with Angular 9 running:

npm install -g @angular/cli     // to install the Angular CLI
ng new angular-nine-ivy         // or the name you want

In case you already have a workspace with some changes you have to commit or stash them, otherwise add the flag --allow-dirty.

If you already have the old Angular CLI, it seems the better way is to uninstall the old one and the install the new globally.

Let’s compile

There are two compiler entry points in the @angular/compiler-cli: ngtsc and ngcc.

`ngtsc`

It is the TypeScript to JavaScript transpiler that reifies the Angular decorators into static properties. ngc works like the ngtsc when Ivy is enabled.

`ngcc`

It is the Angular Compatibility Compiler that process the NPM library code from the node_modules folder producing equivalent library code Ivy compatible. ngcc can also be run by a code loader like Webpack to get packages transpiled on demand instead of written in the node_module package folder.

`ngc`

Open the package.json and add the ngc script:

"scripts": {
    "ng": "ng",
    ...
    "ngc": "ngc"
}

In the tsconfig.json set "declaration": true, in order to have the .d.ts files as well, then run:

$ npm run ngc

The result of the component compilation is located at dist\out-tsc\src. The Welcome to Angular! component is translated into:

import { Component, Input } from '@angular/core';
import * as i0 from "@angular/core";
export class AppComponent {
    constructor() {
        this.title = 'Angular';
    }
}
AppComponent.ɵfac = function AppComponent_Factory(t) { return new (t || AppComponent)(); };
AppComponent.ɵcmp = i0.ɵɵdefineComponent({ type: AppComponent, selectors: [["app-root"]], inputs: { title: "title" }, decls: 3, vars: 1, consts: [[2, "text-align", "center"]], template: function AppComponent_Template(rf, ctx) { if (rf & 1) {
        i0.ɵɵelementStart(0, "div", 0);
        i0.ɵɵelementStart(1, "h1");
        i0.ɵɵtext(2);
        i0.ɵɵelementEnd();
        i0.ɵɵelementEnd();
    } if (rf & 2) {
        i0.ɵɵadvance(2);
        i0.ɵɵtextInterpolate1(" Welcome to ", ctx.title, " ");
    } }, styles: [""] });
/*@__PURE__*/ (function () { i0.ɵsetClassMetadata(AppComponent, [{
        type: Component,
        args: [{
                selector: 'app-root',
                template: `
    <div style="text-align:center">
      <h1>
        Welcome to 
      </h1>
    </div>
  `,
                styleUrls: ['./app.component.css']
            }]
    }], null, { title: [{
            type: Input
        }] }); })();

and then the app.component.d.ts definition file:

import * as i0 from "@angular/core";
export declare class AppComponent {
    title: string;
    static ɵfac: i0.ɵɵFactoryDef<AppComponent>;
    static ɵcmp: i0.ɵɵComponentDefWithMeta<AppComponent, "app-root", never, { "title": "title"; }, {}, never>;
}

Attention By default Ivy is enabled, if you disable it you will get a different compilation output.

JIT vs. AoT

Compiler can work in JiT (Just in Time) mode, it is delivered along with the application and compiles at runtime. AoT (Ahead of Time) compilation instead compiles everything at build time making the application faster and it does not required to ship the compiler with the application.

With Angular 9, thanks to Ivy, the compilation is faster and by default "aot": true.

Compilation models

Both TypeScript and Angular Compiler preserve important metadata that cannot be part of the emitted .js files. Angular enhances the .d.ts files with framework specific metadata to be reused for better component type checking.

TypeScript compilation model

JavaScript code has no type information, it is just ready to be executed in the browser. The TypeScript compiler has transformed all the .ts source files but typing information are not totally lost, the definition files help to preserve the interface for future use.

For instance the file library.ts is compiled into library.js and library.d.ts is produced, it describes the interface or public API of the library.

The definition file brings type information to help TypeScript to static type check the usage of the library. Fo example, when an application declare a npm dependency library can then import functionality:

import {AwesomeLib} from 'awesome-lib';
// use the lib

TypeScript will static type check the use of the library in the app.js exploiting the library.d.ts definition file:

declare class AwesomeLib {
    awesomeMethod(): string;
}

Angular Ivy compilation model

An Angular component has some useful information declared in the decorator like the selector. These information are really important to use the component elsewhere and static type check the code.

The Angular Ivy compiler has been improved to not waste this information and to enrich the definition file public interface. For instance, the following Angular component:

@Component({
    selector: 'awesome-comp'
})
export class AwesomeComponent {
    @Input() value: string;
}

will be compiled into JavaScript and the metadata will enrich the definition file as following:

export declare class AwesomeComponent() {
    value: string;
    static ngComponentDef: ng.ComponentDef<
        AwesomeComponent,
        `awesome-comp`,
        {value: 'value'}
    >;
}

The definition file became the component public API. The compiler can exploit it in order to type check the code that will use it:

<awesome-comp [value]="a value">
    just a component
</awesome-comp>

TypeScript transpiler architecture

The following diagram, thanks to Angular team, shows the normal tsc flow and the steps to transpile a .ts file into the .js one.

                                                                |------------|
                           |----------------------------------> | TypeScript |
                           |                                    |   .d.ts    |
                           |                                    |------------|
                           |
|------------|          |-----|               |-----|           |------------|
| TypeScript | -parse-> | AST | ->transform-> | AST | ->print-> | JavaScript |
|   source   |    |     |-----|       |       |-----|           |   source   |
|------------|    |        |          |                         |------------|
                  |    type-check     |
                  |        |          |
                  |        v          |
                  |    |--------|     |
                  |--> | errors | <---|
                       |--------|

parse: recursive descent parser that produces the abstract syntax tree (AST);
type check: perform type analysis on every file, report found error. Not modified by the ngtsc;
AST to AST: remove type declarations, convert class into ES5, async methods, etc.

Extension points

TypeScript tsc provides some extension points to alter its output:

CompilerHost.getSourceFile to modify the source;
CustomTransformers to alter the list of transforms;
WriteFileCallback to intercept the output before it is written.

Compilation steps

The ngtsc is a wrapper around the tsc, the TypeScript compiler, that extends and modify the normal compilation flow.

Tip The TypeScript transpiler cannot compile the Angular templates and decorators, so the Angular compiler kicks in to reify Angular decorators into static properties. Once finished the TypeScript compiler can go on producing JavaScript code. In other words the Angular compiler allows the code written in Angular declarative syntax to participate to the TypeScript compilation process.

1. Program creation

Starting from the tsconfig.json file, the TypeScript process discover the application source files via the import statements.

2. Analysis

The Angular compiler takes all the .ts files collected and, class by class, looks for Angular declarative syntax code, basically Angular things. The compiler gathers isolation information about components for instance, but not about modules. Remember, for the @Component the compiler requires a global knowledge about the module due transitive closure resolution of the exports of the component module template imports used in the component template.

3. Resolve

The Angular compiler looks again at the whole application but this time in a larger picture including modules as well. All the code now is understandable and parsable by the next step of the TypeScript compiler. Optimizations will take place at this step.

4. Type checking

TypeScript checks error in the application, templates included, that now are a series of imperative instructions.

5. Emit

The most expensive operation in the compilation process, the TypeScript code is transformed into JavaScript ready to be run by the browser. Angular component classes have now only imperative code to describe what a template looks like.

Compiler features

Angular compiler has many interesting features, some of them have been enhanced and improved with the new Angular Ivy architecture. Let’s see some of them.

Module compilation scope

The module scope allows the compiler to resolve uniquely the Angular components used by the application. Consider the following application module:

@NgModule({
    declarations: [
        AppComponent,
        HelloComponent
    ]
})
export class AppModule {}

The module decorator property declarations is the module compilation scope, it holds an array of Angular components used in the application templates. The HelloComponent is a component coming from a library with its own definition file enriched with metadata as seen before.

The developer declares the willingness to use the component in the array, then can use the component adding the corresponding selector in one of the application templates.

The compiler can than uniquely match the component, verify the use of the selectors along with its attributes.

Export compilation scope

A module can be used to export Angular components as a method to make visible the external world some implemented components.

Tip The module pattern is quite frequently used in computer languages as a way to separate a complex application in small chunks that can be reused elsewhere. Just think about ESM or CommonJS etc. Angular provides a way to create a modular application via the module concept as well. A module is used also to hide implementation details and to select what should be public. The exports property is the way chosen to make Angular component public so re-usable by the rest of the world.

In the following example the HelloModule both declares and exports the HelloWorld component. The module library just implements one component and exports it to all the applications that want to use it:

@NgModule({
    declaration: [HelloComponent],
    exports: [HelloComponent]
})
export class HelloModule {}

When an application wants to use the HelloWorld component has just to declare it:

@NgModule({
    declarations: [
        AppComponent,
        HelloComponent
    ]
})
export class AppModule {}

The compiler then can:

uniquely find the component definition;
figure out from the template which components are used;
generate the code and reference it accordingly;
help the tree-shaker to remove things that are not referenced.

Partial evaluation

Compiler actually attempt to almost run TypeScript code in decorators

Alex Rickabaugh - Angular Connect 2019

Angular compiler contains almost a complete TypeScript interpreter. In the following example:

import {SOME_MODULES} from './some_module';

@NgModule({
    declarations: [HelloComponent, ByeComponent],
    exports: [HelloComponent, ByeComponent],
    imports: [...SOME_MODULES]
})
export class AnotherModuel {...}

the compiler can follow and evaluate the import references. Some evaluation are just partial since there are not enough information at compiler time.

For instance:

import {SOME_MODULES} from './some_module';

@NgModule({
    imports: [SOME_MODULES.modules]
})

export const SOME_MODULES = {
    modules: [HelloModule, ByeModule],

    // not available at compile time
    viewportSize: {
        x: document.body.scrollWidth,
        y: document.body.scrollHeight
    }
}

only at runtime some values will available. This value are dynamic expressions. The compiler sees the previous object more or less like this:

SOME_MODULES: {
    "modules": [
        Reference(HelloModule),
        Reference(ByeModule)
    ],
    "viewportSize": {
        x: DynamicValue(document.body.scrollWidth),
        y: DynamicValue(document.body.scrollHeight)
    }
}

where:

SOME_MODULES is an object with two properties;
DynamicValue is an indicator to say “cannot get past”.

The compilation does not stop, it goes on since a situation like this could be quite common.

import {SOME_MODULES} from './some_module'
@Component({
    styles: [`
        :host {
            width: $SOME_MODULES.viewportSize.x
        }
    `]
})

While the compilation goes on for modules properties, it cannot for viewportSize since the value cannot be figured out. An explicative error message is produced about styles.

Template type checking

A template and expressions

<account-view
    [account]="getAccount(user.id, 'primary') | async">
</account-view

they are compiled into TypeScript code becoming type check blocks. So blocks are sent to TypeScript compiler and possible errors are then return in the context of the template.

function typeCheckBlock(ctx: AppComponent) {
    let cmp!: AccountViewCmp;
    let pipe!: ng.AsyncPipe;

    cmp.account /*273,356*/ = (pipe.transform(
        ctx.getAccount((ctx.user /*311,315*/).id /*311,318*/,
        "primary" /*320,329*/)
        /*300,331*/)
    /*300,338*/) /*289,339*/;
}

How is it possible to return an error in the context of a template? Well, code is translated with additional offset comments that allows then the contextualization.

Example

import { Component } from '@angular/core';

@Component({
  selector: 'app-root',
  template: `
    <div style="text-align:center">
      <h1>
        Welcome to {{ title }}!

        <tr *ngFor="let hero of heroes">
            <td>{{hero.name}}</td>
        </tr>
      </h1>
    </div>
  `,
  styleUrls: []
})
export class AppComponent {
  @Input() title = 'Angular';

  heroes = 'fake array';
}

Angular version 9 with Ivy enabled maintains the behavior of the fullTemplateTypeCheck flag and introduces a strict mode with the flag strictTemplates that goes beyond the Angular 8 type checker. Activate the strict mode in the tsconfig.json:

...
"angularCompilerOptions": {
    "fullTemplateTypeCheck": true,
    "strictTemplates": true,
    ...
}

Re-run the npm run ngc and get a full better error reported by the compiler:

src/app/app.component.html:7:13 - error NG2339: Property 'name' does not exist on type 'string'.

7       <td>{{hero.name}}</td>
              ~~~~~~~~~

  src/app/app.component.ts:5:16
    5   templateUrl: './app.component.html',
                     ~~~~~~~~~~~~~~~~~~~~~~
    Error occurs in the template of component AppComponent.

More details available on the template type-check page.

Conclusions

The Angular 9 Ivy rendering architecture introduces a new compiler and a new rendering engine not only to exploit the incremental DOM technique but also a more powerful compiler.

Ivy is enabled by default, --aot is the default way of developing since the new compiler is faster than the previous one. Having AoT mode enable by default reduces also the risk of discrepancies between development and production code.

Ivy compiler goes even further, it has a better type-check making the reported error much more explicative and easier for the developer to identify the root cause of the issue.

We all are looking forward to see the new compiler applied to Angular Elements.

References

Angular Compiler Architecture

Tools

Angular Ivy: a detailed introduction

2019-11-10T00:00:00+00:00

Angular Ivy is the new rendering architecture that comes, by default, with version Angular 9. The Angular rendering architecture is not new to a completely revamp, Angular 2.0, Angular 4.0 and now Angular 9.0 have introduced new compilers and runtime engines.

Currently Angular stable version is 8.3.9 and Angular 9 is in RC4.

Slides

This post comes along with a presentation written in markdown, rendered via reveal.js and available on GitHub.

Lingo

Rendering architecture: compiler and runtime engine pipeline that allows an Angular application to be executed.
Runtime rendering function set/instruction set: set of JavaScript functions understandable by runtime, templates and decorators are transformed into sequence of instructions.
Virtual DOM and incremental DOM: techniques to create and update a component in the DOM.
View Engine: rendering architecture introduced in Angular 4,
angular.json is the workspace or build configuration file.
tsconfig.app.json is the project configuration file.
.ngfactory.js suffix for decorator factory files, class decorators like @Component are translated by the compiler into external files.
Locality: compiler should use only information from the component decorator and its class.
Global compilation: compilation process requires global static analysis to emit the application code.

Rendering Architecture

What is a rendering architecture? It is the pair compiler:runtime. Angular framework is composed by two main parts:

compiler to transform templates written in Angular declarative syntax into JavaScript instructions enriched with change detection;
runtime to execute the application code produced by the compiler.

Currently, Angular 8 uses as rendering architecture called View Engine:

View Engine has been introduced with Angular version 4 and still used in version 8, but some limitations has been identified
- no tree-shakable: both the Hello World application and a very complex one are executed by the same and full runtime. If the internationalization module is not used, for instance, it is part of the runtime anyhow, basically the runtime it cannot be tree-shakable;
- no incremental compilation: Angular compilation is global and it involves not only the application but also the libraries.
Ivy will the new default rendering engine starting from version 9 and should solve the View Engine current issues:
- simplify how Angular works internally;
- three-shakable the Hello World application does not required the full Angular runtime and wil be bundled in only 4.7 KB;
- incremental compilation is not possible so the compilation is faster than ever and --aot can be now used even during development mode (advice from Angular team).

Ivy is an enabler.

Igor Minar - Angular team

The incremental DOM is the foundation of the new rendering engine.

Incremental DOM vs. Virtual DOM

Every component gets compiled into a series of instructions. These instructions create DOM trees and update them in-place when the data changes.

Viktor Savkin - Nrwl

Each compiled component has two main set of instructions:

view creation instructions executed when the component is rendered for the first time;
change detection instructions to update the DOM when the component changes.

Change detection is basically a set of instructions added at compile time. The developer life is made easier since he is aware only of the variable binding in the Angular template declarative.

Incremental DOM enables better bundle size and memory footprint so that applications can perform really well on mobile devices.

Virtual DOM

React and Vue are based on the concept of Virtual DOM to create a component and re-render it when change detection happens.

Render the DOM is a very expensive operation, when a component is added to the DOM or changes happen, the repaint operation has to take place. Virtual DOM strategy aims to reduce the amount of work on the real DOM and so the number of time the user interface needs to be repainted.

Tip The end user sometimes does not realize the complexity behind the rendering of a user interface. A simple click can generate HTTP requests, changes in the component, changes in other components and so on. Single change for the user can be a complex set of changes that must be applied into the DOM.

DOM manipulations happens every time a new component is going to be added, removed or changed from the DOM, so instead of operating directly on the DOM it operates on a JSON object called Virtual DOM. When a new component is added or an existing one removed, a new Virtual DOM is created, the node added or removed and the difference between Virtual DOMs is computed. A sequence of transformations is applied to the real DOM.

React advices to use JSX, a syntax extension to JavaScript, to define React elements. JSX is not a template language. A template is an enriched JavaScript expression that is interpreted at runtime. Plain JavaScript can also be used instead of JSX.

Virtual DOM technique has some disadvantages:

create a whole tree every time a change happens (add or remove a node), so the memory footprint is quite important;
an interpreter is required as long as the diff algorithm to compute the difference among the Virtual DOMs. At compile time it is not known which functionalities are required to render the application, so the whole thing has to be shipped to the browser.

Incremental DOM

It is the foundation of the new rendering engine. Each component template gets compiled into creation and change detection instructions: one to add the component to the DOM and the other one to update the DOM in-place.

Instructions are not interpreted by the Angular runtime, the rendering engine, but the instructions are the rendering engine.

Viktor Savkin - Nrwl

Since the runtime does not interpret the template component instructions, but the component references instructions it is quite easy to remove those instructions that are not referenced. At compile time the unused instruction can be excluded from the bundle.

The amount of memory required to render the DOM is proportional to the size of the component.

Tip The compiled template component references some instructions of the Angular runtime which holds the implementation.

Enable Angular Ivy

Ivy can be enable in an existing project with latest Angular version but also directly scaffold a project with Ivy.

Enable Ivy in an existing project

Having an existing Angular (8.1.x) project run:

$ ng update @angular/cli@next @angular/core@next

both the Angular core and the CLI will be updated at the latest release candidate. One interesting thing to notice is the "aot": true in the angular.json workspace configuration file:

AOT compilation with Ivy is faster and should be used by default.

Angular docs

Then add the angular compiler options in the tsconfig.app.json:

{
  "compilerOptions": { ... },
  "angularCompilerOptions": {
    "enableIvy": true
  }
}

New project with Ivy

To start a new project with Ivy run:

$ new my-app --enable-ivy

Disable Ivy

To disable Ivy:

in angular.json set "aot": false;
in tsconfig.app.json remove the angularCompilerOptions option or set "enableIvy": false.

Angular Ivy Compilation

Consider the following Hello World Angular component:

import { Component, Input } from '@angular/core';

@Component({
  selector: 'app-root',
  template: `
    <div style="text-align:center">
      <h1>
        Welcome to !
      </h1>
    </div>
  `,
  styleUrls: []
})
export class AppComponent {
  @Input() title = 'Angular!';
}

it gets compiled into the following code:

class AppComponent {
  constructor() {
    this.title = 'Angular!';
  }
}
AppComponent.ngComponentDef = defineComponent({
        selectors: [['app-root']],
        factory: function() { return new AppComponent();}
    },
    template: function(flags, context) {
        if (flags & 1) {
            elementStart(0, "div", 0);
            elementStart(1, "h1");
            text(2);
            elementEnd();
            elementEnd();
        } if (flags & 2) {...}
    },
    directives: [...]
  });

The Angular decorators are compiled into static fields in the decorated class. So @Component becomes ngComponentDef static field. Back to View Engine, the ngc compiler produces .ngfactory separated files for each component and modules. With Ivy the code produced by the compiler is move into component class static fields.

The elementStart(), elementEnd(), etc are the component referenced instructions, every component is its own factory, the framework does not interpret the component.

All the not referenced instructions at compile time are removed from the final application bundle.

Tip The View Engine runtime is a single monolith interpreter that is not tree-shakable and has to be entirely shipped to the browser. Differently, Angular Ivy runtime is an instruction set that is a set of rendering functions like an assembly language for templates.

What Angular Ivy enables

Angular Ivy is an enabler. Simplifying how Angular works internally and the compilation process resolves current View Engine limitations and makes Angular easily extensible to new features.

The new Ivy engineering has been driven by three main goals: three-shaking, locality and flexibility.

Tree-shaking

Tree-shaking is the operation of removing dead code from the bundle so if the application does not reference some of the runtime rendering function, they can be omit from the bundle making it smaller.

Dead code comes from libraries, Angular included. Angular CLI is powered by Webpack uglify plugin as tree-shaker that, in turn, receives information from Angular Build Optimizer Plugin about which code is used and which not. Angular compiler simply does not emit those instructions, the plugin can gather information about component referenced instructions so can instruct Uglify about what to include/exclude in/from the bundle.

While the @angular/core framework is tree-shakable, the View Engine runtime is not, it cannot be broken into small pieces and this is mainly due to the static Map<Component, ComponentFactory> variable.

Incremental compilation

The Angular 8 compilation pipeline started by ng build prod --aot is composed by five phases where the tsc and the ngc generates the template factories. ngc compiles the libraries as well. Ivy enables Incremental compilation that is libraries can be compiled and deployed on npm.

Locality

Currently Angular relies on global compilation. The compilation process requires a global static analysis of the entire application to combine different compilation outputs (application, libraries from the monorepo and libraries from npm) before emitting the bundle. Moreover it is really complex to combine AOT libraries into a JIT application.

Tip The compiler should use only information provided by component decorator and its class and nothing else. This simplifies the overall compilation process, no more component.metadata.json and component.ngfactory.json that requires complex management in the compilation pipeline.

Locality is a rule. Ivy compilation introduces the concept of component/directive public API: an Angular application can safely refer to components and directives public API, no more needed to know much about dependencies since extra information are added to .d.ts component files.

Ivy library compilation example

Add a library to the monorepo where your application is running ng generate library mylib.

Compile the library with ng build mylib, the following files are produced:

├── bundles
├── ...
├── lib
│   ├── mylib.component.d.ts
│   ├── mylib.module.d.ts
│   └── mylib.service.d.ts
├── mylib.d.ts
├── package.json
└── public-api.d.ts

Notice as well that this new message is displayed in version 9 due to Ivy activation:

Building Angular Package
******************************************************************************
It is not recommended to publish Ivy libraries to NPM repositories.
Read more here: https://next.angular.io/guide/ivy#maintaining-library-compatibility
******************************************************************************

Generated component

This is the component generated by the Angular CLI:

import { Component, OnInit } from '@angular/core';

@Component({
  selector: 'lib-mylib',
  template: `
    <p>mylib works!</p>
  `,
  styles: []
})
export class MylibComponent implements OnInit {

  constructor() { }

  ngOnInit() { }
}

Compiled library code

The metadata file mylib.metadata.json is not generated anymore, metadata are now part of the definition files.

Definition file of the component:

import { OnInit } from "@angular/core";
import * as i0 from "@angular/core";
export declare class MylibComponent implements OnInit {
  constructor();
  ngOnInit(): void;
  static ɵfac: i0.ɵɵFactoryDef<MylibComponent>;
  static ɵcmp: i0.ɵɵComponentDefWithMeta<MylibComponent,"lib-mylib",never,{},{},never>;
}

Definition file of the module:

import * as i0 from "@angular/core";
import * as i1 from "./mylib.component";
export declare class MylibModule {
    static ɵmod: i0.ɵɵNgModuleDefWithMeta<MylibModule, [typeof i1.MylibComponent], never, [typeof i1.MylibComponent]>;
    static ɵinj: i0.ɵɵInjectorDef<MylibModule>;
}

and definition file of the service:

import * as i0 from "@angular/core";
export declare class MylibService {
    constructor();
    static ɵfac: i0.ɵɵFactoryDef<MylibService>;
    static ɵprov: i0.ɵɵInjectableDef<MylibService>;
}

Add a property to the component

Add to the library component an input field:

@Component({
  selector: 'lib-mylib',
  template: `
    <p>Please input your phone</p>
    <input #phone placeholder="phone number" />
  `,
  styles: []
})
export class MylibComponent implements OnInit {

  @Input('phone-number') private phone: string;

  constructor() { }

  ngOnInit() {
  }
}

The alias phone-number will be added to the input property providing additional metadata for the public API. The compiler generates the following definition file:

import { OnInit } from '@angular/core';
import * as i0 from "@angular/core";
export declare class MylibComponent implements OnInit {
    private phone;
    constructor();
    ngOnInit(): void;
    static ɵfac: i0.ɵɵFactoryDef<MylibComponent>;
    static ɵcmp: i0.ɵɵComponentDefWithMeta<MylibComponent, "lib-mylib", never, { 'phone': "phone-number" }, {}, never>;
}

Decorator that marks a class field as an input property and supplies configuration metadata. The input property is bound to a DOM property in the template. During change detection, Angular automatically updates the data property with the DOM property’s value.

Input decorator - Angular docs

The property phone-number is the name part of the public API while phone is the private name, an implementation detail. Since it can change, the code must be compile every time to emit, in case, an error if there is a property name mismatch. For this reason current Angular version must rely on global compilation.

Angular Ivy instead relies on the public API, so library code can be compiled and safely shipped to npm.

Browser property

The input property is bound to a DOM property in the template.

Basically

When the browser loads the page, it “reads” (another word: “parses”) the HTML and generates DOM objects from it. For element nodes, most standard HTML attributes automatically become properties of DOM objects.

Attributes and properties - javascript.info

Angular compiler transforms the decorators and the templates into JavaScript instructions not only to create element into the DOM but also extra content properties and attributes used by runtime to “keep alive” the application.

Flexibility

Angular Ivy is more flexible that View Engine, if new features are introduced in Angular new instructions will be implemented in the set. Ivy is easier to be extended and optimized.

Angular Ivy Build Pipeline

The compilation of anf Angular application is just half of the process since the libraries the application depends on have to be make compatible with the new runtime.

ngcc (Angular compatibility compiler) is a new compiler that convert and compile the libraries. Libraries compatible with ViewEngine, the previous rendering engine of Angular, are converted into Ivy instructions so that the “library can participate in the Ivy runtime” and be fully compatible.

The new compiler has been implemented to make libraries compatible with the new format without obliging maintainers to rewrite important parts of them and, moreover, not all the applications need to be compatible with Ivy.

In Angular version 9 Ivy is enable for application only and ngcc is used to convert existing libraries making them Ivy compatible. Over time application will start to become more and more Ivy compatible and so the libraries, then ngcc will not be anymore necessary. Libraries can be converted on the fly into Ivy compatible libraries during the build or installation process.

Incremental transition from version 9 to version 11 will make ngcc only required for some few cases:

Angular version	ngcc
9	app on Ivy (opt-out) and libraries VE compatible
10	stabilize Ivy instruction set, libraries ship Ivy code
11	`ngcc` backup for obsolete libraries or not updated yet

ngcc-validation project is the way Angular team test the libraries compatibility.

Component lazy loading feature

Angular is enabler, it will allow more improvement about performance not only for the build but also for the application. Since version 2 Angular has a component lazy loading feature but just at router level: .italic-red-text}. Lazy loading at component level requires a lot of boilerplate code and some patches to make it work.

With Angular Ivy will be much simpler. Consider the following example: click on an image, lazy load the bundle and add the component to the view. Lazy loading improves the speed of an application. Ideally{. .italic-red-text} it will be:

@Component(...)
export class AppComponent{
  constructor(
      private viewContainer: ViewContainer,
      private cfr: ComponentFactoryResolver) {

    // lazy click handler
    async lazyload() {
      // use the dynamic import
      const {LazyComponent} = await import('./lazy/lazy.component');
      this.viewContainer.createComponent(LazyComponent);
    }
  }
}

View Engine obliges to pass via the ComponentFactoryResolver to resolve the lazy component into a factory and to load it:

this.viewContainer.createComponent(this.cfr.resolveComponentFactory(LazyComponent));

Bundle size

To evaluate the bundle size improvement, the Angular team uses a metric the Hello World application. Building with Angular Ivy, the final minimized bundle is ~4.5kB and ~2.7kB with Closure Compiler.

Angular Elements can be then bundled more efficiently and, moreover, Ivy is ready for future bundlers/optimizers.

Debugging

A new API has been added to the global ng object. In the ChromeDevTools just open the console and type ng to see the new options:

Consider to have a <mat-drover></mat-drover> component from the Angular Material library, it is possible to directly act on the component from the console (thanks to Juri Strumpflohner for the example in his tutorial):

// grab the component instance of the DOM element stored in $0
let matDrawer = ng.getComponent($0);

// interact with the component's API
matDrawer.toggle();

// trigger change detection on the component
ng.markDirty(matDrawer);

From the Elements tab just select the element of the debug action, a $0 will appear close to it, it can be used as selector/placeholder for the element in the console.

NgProbe will not be probably supported anymore:

In Ivy, we don’t support NgProbe because we have our own set of testing utilities with more robust functionality.

We shouldn’t bring in NgProbe because it prevents DebugNode and friends from tree-shaking properly.

Platfrom browser - Angular source code

Conclusions

Angular team has done an amazing job, it was really a pleasure to attend the Angular Connect 2019 and see the improvement done on the new rendering architecture introduced last year.

Development can be done now with aot compilation enabled by default to avoid possible mismatches between the development and the production environment.

Another interesting point is Angular Elements. I think the project can now really speeds up thanks to the new compiler and rendering engine. Currently it is not possible to create a library project and compiler it as web components, this will really a killing feature. Moreover the generated web components have “too much Angular inside”, they are a bit too big, Ivy should reduce the amount of the framework that wraps an Angular component.

Really impressive is the lazy loading that could be achieved in a very simple manner, powerful but keeping the readability of the code simple.

References

Angular 6 Elements

2018-09-15T00:00:00+00:00

Angular Elements library provides a method to extend the HTML tag library writing components that can be used with any framework. The new components can be used everywhere without any Angular knowledge and regardless the technology used to implement a web site:

CMS
HTML and VanillaJS
React, Vue, etc
legacy AngularJS application as an alternative to ngUpdate for the migration.

Angular Elements project has been revealed at the AngularConnect 2017 by Rob Worlmald

Here's the slides from my Angular Elements talk @ #AngularConnect - https://t.co/4kuySZ9zMs
— Rob Wormald (@robwormald) November 8, 2017

Presentation. A presentation about Angular Elements based on this post is also available.

World of Custom Components

Want to know the syntax difference between native Web Components, @polymer, @stenciljs and @Angular Elements? I created a repo that implements the same Todo list with those technologies
Repo: https://t.co/dWlM4Hsz4P
Demo: https://t.co/5eo8NrxZQb #TodoMVC pic.twitter.com/RMoNQOuBmV
— Julien Renaux 👨‍💻 (@julienrenaux) January 29, 2018

Each framework provides a specific tailored mechanism to smoothly extend the HTML tag library w.r.t. its own way to configure, interact and update the DOM element.

For instance in AngularJS, the developer can implement new HTML tags via the directive approach: a powerful technique to link together a controller and a template encapsulating everything in a new DOM element.

Framework specific components libraries cannot be used neither by another framework nor in a CMS nor in a website implemented in HTML and JavaScript with standard Web API.

Nitty Gritty. Web Components allow to extend the HTML developing new cross browser DOM elements. Encapsulation and private features enpower reusability avoiding collision with other elements (Shadow DOM).

Web Components

Web components are a set of Web Platform APIs to create new custom, reusable, encapsulated HTML tags that can be used everywhere.

Developers can then extend HTML tag library writing new elements that encapsule style and behavior.

Web Components include four different technologies:

Custom Elements: JavaScript APIs to define new DOM elements.
Shodow DOM: JavaScript APIs to render and attach and encapsulated Shadow DOM tree to an element whose features stay private and not collide with the rest of the document.
HTML templates: write HTML templates that can be reused by custom elements using <template> and <slot> tags.
HTML Imports: how to include and reuse HTML templates.

Pro Tip. Angular Elements are powered by Custom Elements.

Watch out. Web Components specification has moved from V0 to V1.

Extend the HTML with v1 Custom Elements

Let’s introduce an example to show how to implement a new custom element.

Custom Element. Web API that allow to extend the HTML tag system creating new custom DOM elements that encapsulate functinalities away from the rest of the code. It refers to encapsulation principle of the software engineering.

To define a new v1 Custom Element:

create a ES6 class extending HTMLElement;
register the new class with the browser via the CustomElementRegistry Web API: use [window.customElements] property of the Window interface to get an instance of it;
use lifecycle hooks or reactions
- constructor: initialize the state, set up event listeners or create a Shadow DOM
- connectedCallback: called when the element is attached to the DOM, useful for setting up, fetching some resources or rendering
- disconnectedCallback: called when the element is removed from the DOM, useful for cleaning up
- attributeChangedCallback(attrName, oldVal, newVal): called when an attribute registered in the observedAttributes property has been added, removed, updated or replaced
- adoptedCallback(): called when the element has been moved into a new document (document.adoptNode(el))
- more examples
events can be associated, as any other DOM element, using EventTarget method addEventListener.

Shadow DOM

Shadow DOM is more or less like a scoped subtree inside the Custom Element that helps to build a new DOM element.

class RocketParagraph extends HTMLElement {
  constructor() {

    super()

    const shadowRoot = this.attachShadow({ mode: 'open' })
    shadowRoot.innerHTML = `<link rel="stylesheet" href="fontawesome.com/all.css">`

    const span = document.createElement('span')
    const icon = document.createElement('i')

    ...

    shadowRoot.appendChild(span)
    shadowRoot.appendChild(icon)
  }
}
window.customElements.define('rocket-paragraph', RocketParagraph)

A Shadow Root is the document fragment that gets attached to a “host” element. The act of attaching a shadow root is how the element gains its shadow DOM.

Shadow Root in the Chrome debugger.

Building Libraries

There are many libraries that help to build custom Web Components providing features such as:

boilerplate to define the project scaffolding
- Polymer Boilerplate
- ValillaJS Boilerplate
- etc
polyfills to have web components work in all major browsers
automatic build systems
- Yeoman generator to automate scaffolding and building web components.
- Polymer CLI to automate scaffolding and build Polymer-based web components.

Once created the component it can be published and the downloaded and installed via a package manager like Bower.

Look for a component, for instance emoji-rain, install and use it in your web site.

More examples are available on the Web Components Github MDN repo.

Angular Elements

Angular Component on the inside, standards on the outside. Rob Wormald

Pro Tip. Angular Elements are Angular Components packed as Custom Elements, part of the Web Components, a suite of Web platform APIs.

Angular Components act as a hosts for Custom Elements mapping properties from the Angular world to the Web Platform API standard:

@Input mapped to properties
@Output mapped to events
@HostBinding mapped to observed attributes
lifecycle hooks mapped to Custom Elements lifecycle hooks

Whereas Inputs rely on the Custom Elements lifecycle hook, Outputs rely on the Custom Event API (spec). Both APIs are entirely separate from each other.

Pro Tip. Angular Elements are self-bootstrapping, they host an Angular Component inside a Custom Element and their usage is not anymore tied to a specific framework.

Dependency Injection. Angular Elements also let us take advantage of Angular’s dependency injection. When we create a custom element using Angular Elements, we pass in a reference to the current module’s injector. This injector lets us share context across multiple elements or use shared services.

Content Projection. We also get content projection (transclusion) with Angular Elements, but not with dynamic content projection yet. Content projection we’ll have the ability to use slots and the shadow DOM as browser support increases.

PR for basic Shadow DOM v1 support in Angular - should make Elements more flexible, as you can now use <slot> elements to do basic content projection. https://t.co/SBwNBQmwwe
— Rob Wormald (@robwormald) June 30, 2018

Angular Elements: not production ready yet

Angular Elements is one of the Angular Labs projects where new experimental ideas are developed. The first release became part of Angular 6 and a more refined version should arrive along with Angular 7. For the time being Angular Elements is not ready for production yet:

element bundle size: even if a small widget is delivered a great amount of Angular code comes along with. Angular Yvi, next Angular compiler generation, should solve this problem removing from the bundle unused Angular code;
laborius setup: the set up required to use Angular Elements is a bit cumbersome, better way should come;
browser support: native browser support for Custom Elements has been not fully implemented everywhere as can be seen from this table. For the time being:
- Chrome 54 has Custome Elements v1;
- Safari 10.1 has Custom Elements v1;
- Edge has begun prototyping;
- Mozilla has an open bug to implement;
- use @webcomponents/webcomponentsjs loader to asynchronously load only the necessary polyfills required by the browser.

Angular Elements In Action

The code used in this post is available on GitHub. Following all the steps to reproduce the full workflow if you start from scratch.

Angular Elements Component Description

The Angular Elements component will allow the user to select a CSS shape and, once the button will be pressed, an alert message will be displayed with the info of the selection made.

The goal is to show how the input/output between an Angular Elements component and the vanilla JavaScript of the hosting page works. Basically the selection is captured by the component, the action connected with the submit button generates an event that communicate the selection made by the user to the external world.

The final result: the user can select the desired CSS shape and then press the button submit.

Setup the Project

npm install -g @angular/cli@latest
ng new angular-geometry-elements
cd angular-geometry-elements
ng add @angular/elements --project=angular-geometry-elements

The latest command add both Angular Elements and the polyfill called document-register-element to the configuration file angular.json.

Create the Component

Open and modify the file app.component.ts as follows:

import { Component, ViewEncapsulation, Input, Output, EventEmitter, OnInit } from '@angular/core';
import { FormControl, FormGroup } from '@angular/forms';

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css'],
  encapsulation: ViewEncapsulation.Native
})
export class AppComponent implements OnInit {

  @Input() defaultValue = 'square';

  @Output() selectedValue = new EventEmitter<string>();

  form: FormGroup;

  ngOnInit(): void {
    this.form = new FormGroup({
      shape: new FormControl(this.defaultValue)
    });
  }

  submit() {
    this.selectedValue.emit(this.form.value);
  }
}

ViewEncapulation: change the encapsulation strategy w.r.t. the styles are applied to the component:
- Native: use the real Shadow DOM if natively available, otherwise the web-components polyfills are required to shim the behavior.
- Emulated: scoped styling emulating the Shadow DOM.
@Input for a property with a default value.
Some other properties.

In the file app.module.ts rename bootstrap to entryComponents to avoid the component bootstrap along with the module:

import { BrowserModule } from '@angular/platform-browser';
import { NgModule, Injector } from '@angular/core';

import { FormsModule, ReactiveFormsModule } from '@angular/forms';

import { createCustomElement } from '@angular/elements';

import { AppComponent } from './app.component';

@NgModule({
  declarations: [
    AppComponent
  ],
  imports: [
    BrowserModule,
    FormsModule,
    ReactiveFormsModule
  ],
  providers: [],
  entryComponents: [AppComponent]
})
export class AppModule {

  // The bridge between Angular and the Web API.
  constructor(private injector: Injector) {
    const el = createCustomElement(AppComponent, {injector});
    customElements.define('geometry-elements', el);
  }

  ngDoBootstrap() {}

}

Now focus on the constructor of the module:

constructor(private injector: Injector) {
  const el = createCustomElement(AppComponent, {injector});
  customElements.define('geometry-elements', el);
}

this function returns a special class that can be used with the Custom Element Web API to define a new custom element as the pair selector and ES6 class.

Pro Tip. The Angular Component now act as a host for the Custom Element.

Build and Pack the Angular Element Component

So fare there is not a straight forward way to pack an Angular Element component, here some approaches proposed by Sam Julien:

Use a bash script, like in this article by Tomek Sułkowski.
Use Gulp, like in this made-with-love component by Nitay Neeman.
Use Manfred Steyer’s CLI tool ngx-build-plus.
Use Node with a script like in this article by Sebastian Eschweiler.

Following the Sam Julien’s advice the Node approach is choosen:

npm install --save-dev concat fs-extra

Create a file called elements-build.js at the root of the project with the following script:

const fs = require('fs-extra');
const concat = require('concat');

(async function build() {
  const files = [
    './dist/angular-geometry-elements/runtime.js',
    './dist/angular-geometry-elements/polyfills.js',
    './dist/angular-geometry-elements/scripts.js',
    './dist/angular-geometry-elements/main.js'
  ];

  await fs.ensureDir('elements');
  await concat(files, 'elements/geometry-elements.js');
  await fs.copyFile(
    './dist/angular-geometry-elements/styles.css',
    'elements/styles.css'
  );
})();

The script will combine what the Angular CLI has generated in a single file.

In the package.json add

"build:elements": "ng build --prod --output-hashing none && node elements-build.js"

Then create the file index.html in the previously created folder elements to have a page where to test the new created component:

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>Angular Elements</title>
    <base href="/">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <link rel="icon" type="image/x-icon" href="favicon.ico">
    <link rel="stylesheet" href="styles.css"></head>
  </head>
  <body>
    <geometry-elements id="el" default-value="circle"></geometry-elements>
    <script type="text/javascript" src="geometry-elements.js"></script>
    <script>
      document.getElementById("el").addEventListener("selectedValue", function(value){
        alert("Output from the component " + JSON.stringify(value.detail));
      })
    </script>
  </body>
</html>

Fireup the build

npm run build:elements

The output of the build is put in the elements folder:

.
├── angular-geometry-elements.js
├── index.html
└── styles.css

Note. Style file is empty because of the Native strategy choice.

Test The Angular Elements Component

Install the static-server to serve the elements/index.html

npm install -g static-server

From the afrementioned folder type

static-server

and connect to localhost:9080.

Check The Registered Element

Following this advice, in the browser console type:

String.prototype.wasRegistered = function() { 
  switch(document.createElement(this).constructor) {
    case HTMLElement: return false; 
    case HTMLUnknownElement: return undefined; 
  }
  return true;
}
'element-cake'.wasRegistered()
//⇒ true
'element-my-cake'.wasRegistered()
//⇒ false
'xx'.wasRegistered()
//⇒ undefined

Browser Support

Recent News

Great to hear that! Firefox 63 are getting Shadow DOM and Custom Elements.https://t.co/Hgp7HXx8r5
— Hayato (Shadow DOM) (@shadow_hayato) August 14, 2018

We are planning on deprecate and remove Shadow DOM v0 and Custom Elements v0 from Blink. Shadow DOM v0 and Custom Element v0 are NOT *Web*. They are only available on Google Chrome.
If you are still using v0 APIs, please consider to migrate to v1 seriously.
— Hayato (Shadow DOM) (@shadow_hayato) July 11, 2018

Little hack for enabling shadow DOM V1 and slots in Angular Elements, till we fix https://t.co/Q3F1LoUcGP - https://t.co/vh4TRGbQTP
— Rob Wormald (@robwormald) June 10, 2018

Polyfill vs. Shim

2016-10-25T18:47:00+00:00

With the new ES6 specification, people talk more and more about polyfill and shim, sometimes as the term are interchangeable but, are they the same thing?

Polyfills and shims

There are many definitions and interpretation about what is Polyfill and shim. Agree that both are libraries there are some differences between them.

Shim:

the API exists in major browser;
has its own API;
is more about fixing some functionality and aligning browser behavior with another browser;
is strictly tailored for an old environment;
is a graceful degradation.

Watch out! Saying “has its own API” doesn’t mean it exposes a broader API, more methods, but just that all the methods from the API are reimplemented. The shim intercepts all the API calls and provides its own implementation so that all the browsers have the same behavior.

Polyfill:

is a shim for a browser API;
is related to browsers;
implements missing feature in an API;
is something that you could drop in and it would silently work;
brings future EcmaScript features back in time to old and latest browsers (regressive enhancement).

In a nutshell

Respoke has created an interesting flow chart to decide if a certain library is a shim or a polyfill. The demarcation between them is not always really sharp. Try to answer the following question and make a decision:

Does your library fix some functionality and/or normalize a JavaScript API across the major browsers?
Does the JavaScript API exist in some major browsers?
Does your library implement the JavaScript API where it does not exist?

Polyfill, back to 2010

The term polyfill comes from a 2010 blog post by Remy Sharp where Remy Sharp was looking for a new term or word:

I wanted a word that meant “replicate an API using JavaScript (or Flash or whatever) if the browser doesn’t have it natively. … I wanted something you could drop in and it would silently work.

The term shim doesn’t work for him because:

I knew what I was after wasn’t progressive enhancement because the baseline that I was working to required JavaScript and the latest technology. So that existing term didn’t work for me.

So Remy gives the following definitions:

A polyfill, or polyfiller, is a piece of code (or plugin) that provides the technology that you, the developer, expect the browser to provide natively. Flattening the API landscape if you will.

Poly meaning it could be solved using any number of techniques - it wasn’t limited to just being done using JavaScript, and fill would fill the hole in the browser where the technology needed to be. It also didn’t imply “old browser” (because we need to polyfill new browser too).

Shim, to me, meant a piece of code that you could add that would fix some functionality, but it would most often have its own API.

Other definitions

I like classify polyfilling as a form of Regressive Enhancement

Alex Sexton

A shim that mimics a future API providing fallback functionality to older browsers.

Paul Irish

A polyfill is code that detects if a certain “expected” API is missing and manually implements it. E.g.
if (!Function.prototype.bind) { Function.prototype.bind = ...; }

A shim is code that intercepts existing API calls and implements different behavior.

The idea here is to normalize certain APIs across different environments. So, if two browsers implement the same API differently, you could intercept the API calls in one of those browsers and make its behavior align with the other browser. Or, if a browser has a bug in one of its APIs, you could again intercept calls to that API, and then circumvent the bug.

Sime Vidas

A shim is a library that brings a new API to an older environment, using only the means of that environment.

A polyfill is a shim for a browser API. It typically checks if a browser supports an API. If it doesn’t, the polyfill installs its own implementation. That allows you to use the API in either case.

Axel Rauschmayer

Polyfills and shims libraries

Paul Irish provides a list of HTML5 cross browser polyfills and shims.

Future of Declaration Files

2016-09-11T09:15:37+00:00

When I have started to use TypeScript I have looked at Definitely Typed to see how many people where writing declaration files (.d.ts) and then guidelines from Microsoft to see how to do it. There was then the utility tsd, but now many stuff has changed.

Evolution at Glance

TSD Package Manager

TSD is a package manager to search and install TypeScript definition files directly from the community driven >DefinitelyTyped repository.

TSD

A tool has to be installed via npm to fetch the package and start to configure a file named tsd.json that holds all the declaration (or definition) files. Basically tsd is like npm with similar commands able to manage defintion dependencies used by TypeScript compiler to transpile .ts files and by type guard to perform runtime check.

A type guard is some expression that performs a runtime check that guarantees the type in some scope.

TypeScript

Typings

Some months go, looking at the GitHub repository, I have discovered the project has been deprecated in favor of typings. So typings is an evolution of tsd with some advantages explained here by blakeembrey.

Basically the previous tool has been improved into a more generic package manager that acts like npm, then it still requires to be installed. Old projects have to be updated to migrate from tsd if previously used. Slighter different syntax needs to be learnt.

@Types

Recently I was experimenting Angular2 Command Line Interface and I have noticed that generating a new project the folder typings was not created, something missing? Not at all, TypeScript 2.0 has improved once again and a new property types has been kicked in the tsconfig.json.

So looking at the tsconfig.json schema definition we can find

types: {
  "description": "Type declaration files to be included in compilation. Requires TypeScript version 2.0 or later.",
  "type": "array",
  "items": {
    "type": "string"
  }
}

The evolution is explained in this post from Daniel Rosenwasser Program Manager on TypeScript.

In a nutshell

Getting type declarations in TypeScript 2.0 will require no tools apart from npm. The future of declaration files

Really a good news! we can rid off some extra entities in our project such as:

another package managers then npm;
json file for type declarations;
configure tsconfig.json to exclude files from typings folder;
the famous ///reference at the beginning of each .ts file.

Usage

Very simple:

npm install --save @types/lodash

From there you’ll be able to use lodash in your TypeScript code with no fuss. This works for both modules and global >code.

The future of declaration files

No more additional configuration or references at the beginning of each file.

Type Search

Please refer to TypeSearch search engine to find a type definition and how to install it via npm.

At first sight I could say the project is cleaner, less utilities and, may be, a more consistent way to manage .d.ts files. I hope a definitive path has been defined (at least for a couple of years) in order to have all the dependencies centralized in one place.

How to test 1: find errors to make the application works

2015-03-07T00:00:00+00:00

This is an experiment tutorial to better learn some 101 practices and how testing can be a better replacement of developing by debugging.

The goal of the experiment-project is to find, to correct bugs and make the application works by writing tests.

A bit of theory and jargon

SUT: the System Under Test, in a unit test it could be a class.

Test class: the client of the interface exposed by the SUT.

test fixture: the context under which a test is run, it is a baseline under which running tests. It is a set of operations to bring the System Under Test (SUT) in a given state, run the test and verify the expected outcome. Test fixture and the expected outcome are tightly coupled, the test fixture allows the test to be repeatable for ever and ever. Method setUp() or initialization is used to create fixture while tearDown() or distruction is invoked to restore the original state.

A test has four phases:

set up to apply the test fixture;
exercise to interact with the API exposed by the SUT;
verifiy to check if the expected outcome has been matched;
tear down to return to the original state tearing down the test fixture.

Clone the project

Clone the project from Github and import it in you preferred IDE, IntelliJ IDEA in my case.

Project Structure

├── README.md
├── exercise-api
│   ├── pom.xml
│   └── src
│       └── main
│           └── java
│               └── com
│                   └── contrastofbeauty
│                       └── tutorial
│                           └── api
│                               ├── collectors
│                               │   └── Collector.java
│                               ├── domain
│                               │   ├── AcknoledgeService.java
│                               │   └── Callback.java
│                               └── services
│                                   └── Service.java
├── exercise-to-be-corrected
│   ├── pom.xml
│   └── src
│       ...
│
├── exercise-working
│   ├── pom.xml
│   └── src
│       ...
│
└── pom.xml

The maven project is composed by four modules:

exercise-api: just collection of interfaces used in the other two modules;
exercise-working: working application with a test suite;
exercise-to-be-corrected: application with bugs where the test suite has to be created in order to find, correct bugs and make it work;
exercise-tdd: implementation of some classes by tdd whose description is in the next tut.

In the module exercise-to-be-corrected run the class RunMeToBeCorrected.java and see that it fails some some reason, may be a NullPointerException is thrown. We do not care so much because we do not want to make it work by a common pattern running/debugging rather we want to implement missing tests demonstrating how they can help us to find and correct bug make our application more robust.

Application Description

The application is mainly composed by a fake Cloud Service able to post user’s tweets in batch by means of a Tweet Collector. Here the UML source.

The workflow

The CloudService has a certain numbers of different type collectors (hence the association in the UML diagram), in this case just with one able to collect tweets.
A user who wants to post tweet in batch has to open a connection invoking the method service.openConnection() of the service.
The user then starts to post tweets using the method service.saveObject().
Once done service.saveObjectCompleted() method will be called to tell the service that the user session is finished.

The workflow in details

The CloudService has a TweetCollector and a Callback implementation instance (Another approach is to leave the service the initialization of the callback function and substitute the association with a composition in the UML diagram).
The Callback implementation instance will be set into each added collector by the service.
Every time a user wants to save a tweet through the service, the collector will stock it in a list as a sort of buffer.
Once the collector has stocked a certain amount of tweets, for instance 500, it will generates a task.
A task has a collection of tweets and the implementation of the method call() which will be called by one of the thread of the thread pool. The method specifies how the list of tweets should be treated, for instance: saved, destroyed, sent to Twitter, printed or something else.
A collector uses the Callback instance to add a task to a processing list.
The list of active tasks will be then processed by a thread pool.

How to initiliaze the service and run it

public class RunMeWorking {

    public static void main(String[] args) {

        Callback callback = new CallbackImpl();
        Service service = new CloudService(callback);

        service.addCollector(new TweetCollector());

        service.openConnection(1L);
        service.saveObject(new Tweet("I am Felix the awesome cat."), 1L);
        service.saveObjectCompleted(new AcknoledgeServiceImpl(), 1L);
    }
}

Actually the service does not post anything to Twitter, it is just a way to show a bit of java concurrency and how to write unit tests when dealing with concurrent structures like Future.

Let’s correct the application

Let’s create tests for the application starting from TweetCollector.java class, press cmd + shift + T on Mac or crtl + shift + T on Windows and choose methods you want to test, then Idea will create the test class in the right place

src/test/java/com/contrastofbeauty/tutorial/collectors/TweetCollectorTest.java.

Tip Make sure the path src/test/java exists, otherwise Idea will create the test class together with the source one.

TweetCollector Class

TweetCollector is specific implementation of Collector interface able to stores an object of a specific type in a List by userId when its method accept() is invoked (e.g. by the service cloud).

Once the tweet collector has collected a certain number of tweet objects, flush() method is called, a TweetTask object is created and passed an parameter to callbackFunction.addTask(). The task in then put in a queue to be processed. A task is a computation unit, it owns:

call() method: called by a thread when the task is available in the task queue;
processing list: a set of object on which the aforementioned method will work.

Once the task is added to the queue the list is emptied, ready to accept new object to be processed.

TweeterCollectorTest

Populate the setUp() method creating a new TweetCollector and annotate it with @Before in order to get it run automatically before any method annotated with @Test as specified by JUnit doc. Every time a test method is run, the setUp() method is invoked to have a fresh and clean collector object.

public class TweetCollectorTest {

    private Collector collector;

    @Before
    public void setUp() throws Exception {
        collector = new TweetCollector();
    }
}

Method accept() - 3 errors

Let’s start working on the method accept(Object object, long userId). Create, or rename the method if already created by the IDE, testAcceptGoldenPath() and add the following assertion:

@Test
public void testAcceptGoldenPath() throws Exception {
    assertTrue(collector.accept(new Tweet("foo tweet"), 1L));
}

In the first test we want to test the standard scenario when everything run smooth, no exceptional situations. So for this reason it has been added the suffix GoldenPath, an alternative to Happy Path. After that some borderline scenario tests will be added. So run the test!

Error 1

Issue: a NullPointerException is thrown.

Solution: the object processingList has not been initialized. Add the initialization in the constructor for instance and rerun the test.

public TweetCollector() {
    processingList = new HashMap<>();
}

Error 2

Issue: another NullPointerException is thrown because the data structure is accessed but not initialized for a given user:

processingList.get(userId).add((Tweet) object);

Solution: check if a given userId already exists in the map, if not create and add to the map the pair and rerun the test.

if (processingList.get(userId) == null) {
    processingList.put(userId, new ArrayList<Tweet>());
}

Error 3

Issue: an AssertionError is thrown. All the NPE have been fixed but it seems the collector has not accepted the Tweet object.

Solution: a missing return true prevent the method to behave correctly. Add the aforementioned statement and rerun the test. Now the test passes and this is the complete code for the method accept:

@Override
public boolean accept(Object object, long userId) {

    if (object instanceof Tweet) {

        if (processingList.get(userId) == null) {
            processingList.put(userId, new ArrayList<Tweet>());
        }

        processingList.get(userId).add((Tweet) object);

        if (customBufferSize != 0) {
            if (processingList.get(userId).size() == customBufferSize) {
                flush(userId);
            }
        } else if (processingList.get(userId).size() == PROCESSING_LIST_BUFFER_SIZE) {
            flush(userId);
        }

        return true;
    }

    return false;
}

Additional tests

The method now seems correct but has been tested in case of a different obejcts? Add a new test method which will be part of the automatic test suite we are going to create. This automatic test suite will help us during phases such as refactoring, improving code readability and method evolution.

@Test
public void testAcceptObjectNotAcceptedBecauseDifferentType() throws Exception {
    assertFalse(collector.accept(new Object(), 1L));
}

Our automatic net of tests starts to take shape.

Method flush() - 1 error

The method flush() does not have a return type, so how can we test the correct behavior?

Idea: indeed, we want to test the correct behavior! we need to find a way to check if the behavior of the method is the expected one so if it follows the right path. When flush() is invoke we expect the creation of a TweetTask object and the invocation of addTask() method.

Let’s try step by step. If not done yet by the IDE, create method testFlushGoldenPath() and invoke the method flush(), then run the test!

@Test
public void testFlushGoldenPath() throws Exception {
    collector.flush(USER_ID);
}

Error 1

Issue: a NullPointerException is thrown. This is not a good behavior, the map has not been initialized if the method will be directly called.

Pre-solution: check if in the map exist a List for the given userId, if not simply exit from the method execution (other solutions are acceptable, depends on requirements), rerun the method.

if (processingList.get(userId) == null) {
    return;
}

The test passes, but no verification is done, so the test is pretty useless. We want to verify that if there is an item, a new TweetTask is created and the method callbackFunction.addTask() is invoked.

Idea: the callbackFunction has not been set yet so a possible NPE could arise. May be in the future it will be inject by DI. So we can use Mockito to create a mock object and verify if the method addTask() has been called.

Solution: mock a Callback.class class in order to make a verification on an expected behavior and change a bit the setUp() method to initialize mocks via annotations. Moreover set the callback and add at least one tweet object just to reproduce a small common scenario.

@Mock
private Callback callbackFunctionMock;

@Before
public void setUp() throws Exception {
    MockitoAnnotations.initMocks(this);
    collector = new TweetCollector();
}

@Test
public void testFlushGoldenPath() throws Exception {

    collector.setCallbackFunction(callbackFunctionMock);
    collector.accept(new Tweet("foo tweet"), USER_ID);
    collector.flush(USER_ID);

    verify(callbackFunctionMock, times(1)).addTask(any(TweetTask.class), anyInt());
}

So we set the callback function, we invoke the accept() method to put one tweet into the list for a given user and finally we call the flush() method. A mock object is useful to verify behaviors for instance if a given method has been called. In our case we want to verify if the method addTask() has been called exactly one time. any(java.lang.Class) and anyInt() belongs to org.mockito.Matchers library. A matcher is an entity that helps to match parameters and arguments. For instance, in the test, we need to mach method call parameters, addTask(java.util.concurrent.Callable task, long userId), but we are not interested to pass specific arguments so we use matchers to say whatever object implementing Callable class is fine.

Why? Well, we are not directly call the addTask, we just call a method from a public interface that, under certain condition, should call addTask with a newly created object. We do not control and we do not want to control the creation of the object, just let it be, but we want to verify if the method has been invoked with some instances of a given class. In Mockito if we use a matcher for an argument, all the other arguments must be substituted with matchers.

Best practice Name a mock variable with mock prefix in order to recognize at first sight which variable is a reference to a real object or to a mock. Other patterns are possible, chose the one you like and keep the whole project consistent.

Attention A test class is not just a test on methods but it is a test to verify the correct behavior of the all unit.

Alternative solution with Mockito.spy()

A mock object has been used, but another kind of test double exists, the spy. A spy is a stub (state verification) able to record calling information, it is a “partial mock”. Instead of doing the check on processingList.get(userId) at the beginning of the class it is possible to refactor the tweet creation in a separate method as

@Override
public void flush(long userId) {

    // create a new processing task
    if (callbackFunction != null) {
        TweetTask tweetTask = getTweetTask(userId);
        if(tweetTask != null) {
            callbackFunction.addTask(tweetTask, userId);
        }
    } else {
        throw new IllegalArgumentException("Callback function must be set by the service.");
    }
}

protected TweetTask getTweetTask(long userId) {
    TweetTask tweetTask = new TweetTask(new ArrayList<>(processingList.get(userId)));
    processingList.get(userId).clear();
    return tweetTask;
}

and the corresponding modified test using Mokito.spy() to avoid the invoke on method accept()

@Test
public void testFlushWithSpyGoldenPath() throws Exception {

    TweetCollector spyOnTweetCollector = Mockito.spy(TweetCollector.class);

    spyOnTweetCollector.setCallbackFunction(callbackFunctionMock);
    doReturn(new TweetTask(new ArrayList<Tweet>())).when(spyOnTweetCollector).getTweetTask(USER_ID);

    spyOnTweetCollector.flush(USER_ID);

    verify(callbackFunctionMock, times(1)).addTask(any(TweetTask.class), anyInt());
}

Best practice Name a spy variable with spyOn prefix in order to recognize at first sight which variable is a reference to a real object, to a spy or to a mock.

Attention We do not have invoked accept() method

Another alternative solution without Mockito but with @Override

Just to make a simple comparison, the above test could have been written without Mockito as follows:

@Test
public void testFlushWithOverrideGoldenPath() throws Exception {

    final AtomicBoolean taskAdded = new AtomicBoolean();

    Callback callbackFunction = new CallbackImpl(){
        @Override
        public void addTask(TweetTask tweetTask, long userId) {
            taskAdded.set(true);
        }
    };

    TweetCollector collector = new TweetCollector(){
        @Override
        protected TweetTask getTweetTask(long userId) {
            return new TweetTask(new ArrayList<Tweet>());
        }
    };

    collector.setCallbackFunction(callbackFunction);
    collector.flush(USER_ID);

    assertTrue(taskAdded.get());
}

The use of Mockito makes the code more concise, easy to read, maintain and understand. Moreover the behavior verification is easy to understand than using a variable and check its value. By using ‘verify()’ we directly verify if the method has been called and how many times, it is clear that we are doing behavior verification.

Additional tests

Another test can be written to test the exception along with the message in case the callbackFunction is not set.

@Test
public void testFlushExceptionThrownWithNullCallbackFunction() throws Exception {
    exception.expect(IllegalArgumentException.class);
    exception.expectMessage("Callback function is null, it must be set by the service.");
    collector.accept(new Tweet("foo tweet"), USER_ID);
    collector.flush(USER_ID);
}

Exception messages are really important in order to immediately find the root cause of the issue.

CloudService Class

Once the service is created, once or more collectors are added in order to provide batch processing to one or more social networks or other services. A user has to open a connection to the service through the method openConnection() and then start to post tweets invoking saveObject(). Once done the user calls method saveObjectCompleted() to post latest tweets and close the connection.

CloudServiceTest Class

Create the class as done for the previous example and initialize the System Under Test (SUT). The test class is the API client exposed by the SUT. The setUp() method properly initializes the SUT in order to be ready for the test method.

public class CloudServiceTest {

    private CloudService cloudService;

    private Callback callbackFunction;

    @Before
    public void setUp() throws Exception {

        callbackFunction = new CallbackImpl();
        cloudService = new CloudService(callbackFunction);
    }
}

Method addCollector() - 2 errors

Create the first test for method addCollector()

@Test
public void testAddCollectorGoldenPath() throws Exception {

    cloudService.addCollector(new TweetCollector());

    assertEquals(1, cloudService.getCollectorSize());
}

Error 1

Issue: a NullPointerException will be thrown when CloudService.getCollectorSize() is called in the assertion.

Solution: the object processingCollectors has not been initialized. Add the initialization in the constructor, for instance, and rerun the test.

public CloudService(Callback callback) {

    processingCollectors = new HashMap<>();
}

Error 2 - Test Driven Development (TDD)

Spot this error is not easy, it is a missing code. As it has been seen before, a collector uses a callback function to post a batch processing. When a collector is added the callback function must be set by the service. The collector in this situation acts as a collaborator and we want to verify that a specific method is invoke on the collaborator itself. Create the test before and implementation later on to make the test passes (TDD).

In order to verify a method call, i.e. indirect output, we use Mockito, so add tweetCollectorMock variable and init mocks inside of the setUp() (initialization of the test fixture) method adding MockitoAnnotations.initMocks(this) and run the test

@Mock
private TweetCollector tweetCollectorMock;

@Before
public void setUp() throws Exception {

    MockitoAnnotations.initMocks(this);
    callbackFunction = new CallbackImpl();
    cloudService = new CloudService(callbackFunction);
}

@Test
public void testAddCollectorVerifyCallbackFunctionAddedWithMockito() {

    cloudService.addCollector(tweetCollectorMock);
    verify(tweetCollectorMock, times(1)).setCallbackFunction(any(Callback.class));
}

Issue: an error is thrown Wanted but not invoked:... that means the method setCallbackFunction() has not been called.

Solution: in the method addCollector() add the call to set the callback function and rerun the test.

@Override
public void addCollector(Collector collector) throws RuntimeException {

    processingCollectors.add(collector);

    collector.setCallbackFunction(callbackFunction);
}

Now the test passes. The addCollector() function has been corrected by tests. The tests have been left separated for two reasons:

test two different aspects of the same function
how Test Driven Development (TDD) helps building a solid net of automated tests and a more reliable code, tests come directly from requirements.

Attention! TDD helps creating a tests net, tests represent the requirements, the documentation as code that can be read to understand the behavior of the class. The code satisfying the tests is consistent with the requirements, its behavior is something to rely on for the next development iteration.

Error 2 - Alternative solution with @Override

Instead of using a mocking framework, it is possible to “intercept” the call of the method and register a status that can be used for verification later on as following:

@Test
public void testAddCollectorVerifyCallbackFunctionAddedWithOverride() {

    final AtomicInteger functionCalled = new AtomicInteger();

    cloudService.addCollector(new TweetCollector(){

    @Override
    public void setCallbackFunction(Callback callback) {

            functionCalled.set(1);
        }
    });

    assertEquals(1, functionCalled.get());
}

The use of a mock object and a mocking framework as Mockito allows to have a more concise and readable code. Methods and matchers such as verify, times() and any() make the comprehension of the test easier. Moreover using a mock for a collector makes clearer that in this context it a collaborator and not the object of the test, it is even closer to a documentation purpose.

Method openConnection() - 1 error

Let’s create a new test and run it

@Test
public void testOpenConnectionGoldenPath() {
    cloudService.openConnection(USER_ID);
    assertTrue(cloudService.isUserConnected(USER_ID));
}

Error 1

Issue: a NullPointerException will be thrown.

Solution: the object processingFutureList has not been initialized. Add the initialization in the constructor for instance and rerun the test.

public CloudService(Callback callback) {

    processingFutureList = new HashMap<>();

    processingCollectors = new ArrayList<>();
}

Now the test passes.

Additional test

To build a net of automatic tests is interesting to test different aspects of a method so we can add the following one

@Test
public void testOpenConnectionUserHasNotOpenConnection() {
    assertFalse(cloudService.isUserConnected(USER_ID));
}

Method saveObject() - 0 errors

This method does not have errors anymore, but one or more test must be written to build up an efficient test suite. It is a small workflow able to test the interface exposed by the class.

Remember! Test the behavior of all the unit not just single and separated methods, the focus is on the all unit.

@Test
public void testSaveObjectWithOverrideGoldenPath() throws Exception {

    final AtomicBoolean accepted = new AtomicBoolean();

    Collector tweetCollector = new TweetCollector(){
        @Override
        public boolean accept(Object object, long userId) {

            if (object instanceof Tweet) {
                accepted.set(true);
                return true;
            }

            return false;
        }
    };

    cloudService.addCollector(tweetCollector);

    cloudService.openConnection(USER_ID);

    cloudService.saveObject(new Tweet("foo tweet"), USER_ID);

    assertTrue(accepted.get());
}

Once again pay attention on the difference between using override and Mockito to test indirect output as follows

@Test
public void testSaveObjectWithMockitoGoldenPath() throws Exception {

    Collector collectorMock = mock(TweetCollector.class);
    when(collectorMock.accept(any(Tweet.class), anyInt())).thenReturn(true);

    cloudService.addCollector(collectorMock);

    cloudService.openConnection(USER_ID);

    cloudService.saveObject(tweetMock, USER_ID);

    verify(collectorMock, times(1)).accept(any(Tweet.class), anyInt());
}

Once again the use of Mockito allows to have a more concise and readable test. The Mockito.verify() method highlights the test of the indirect output.

Additional class behavior tests

Additional test 1

It is interesting to enrich the test suite with some other tests, may be a bit redundant but useful to highlight if the code correctly answer to borderline scenarios.

@Rule
public ExpectedException exception = ExpectedException.none();

@Test
public void testSaveObjectObjectNotAcceptedThrowException() throws Exception {

    exception.expect(IllegalArgumentException.class);
    exception.expectMessage("Entity of type " + new Object().getClass() + " cannot be accepted.");

    cloudService.addCollector(new TweetCollector());

    cloudService.openConnection(USER_ID);

    cloudService.saveObject(new Object(), USER_ID);
}

Additional test 2

The name of the method makes the test self-explanatory.

@Test
public void testSaveObjectUserNotConnected() throws Exception {
    exception.expect(IllegalArgumentException.class);
    exception.expectMessage("User with id " + USER_ID + " has not open any connection, please open a connection " +
        "before trying to save.");

    cloudService.saveObject(mock(Callable.class), USER_ID);
}

Additional test 3

@Test
public void testSaveObjectCompletedUserNotConnectedThrowException() throws Exception {
    exception.expect(IllegalArgumentException.class);
    exception.expectMessage("User with id " + USER_ID + " has not open any connection, please open a connection " +
        "before trying to save.");

    cloudService.saveObjectCompleted(null, USER_ID);
}

Additional test 4

@Test
public void testSaveObjectCompletedGoldenPath() throws Exception {

    doNothing().when(acknoledgeServiceMock).sendAckSuccess();

    TweetCollector spyOntweetCollector = spy(new TweetCollector());
    // if you call the real method there will be an exception, use doReturn for stubbing
    doReturn(tweetTaskMock).when(spyOntweetCollector).getTweetTask(USER_ID);
    when(tweetTaskMock.call()).thenReturn(1L);

    cloudService.addCollector(spyOntweetCollector);

    cloudService.openConnection(USER_ID);

    cloudService.saveObject(new Tweet("foo tweet"), USER_ID);

    cloudService.saveObjectCompleted(acknoledgeServiceMock, USER_ID);

    verify(tweetTaskMock, times(1)).call();
    verify(acknoledgeServiceMock, times(1)).sendAckSuccess();
}

Additional test 5

@Test
public void testSaveObjectCompletedExceptionThrown() throws Exception {
    doNothing().when(acknoledgeServiceMock).sendAckSuccess();

    TweetCollector spyOntweetCollector = spy(new TweetCollector());
    // if you call the real method there will be an exception, use doReturn for stubbing
    doReturn(tweetTaskMock).when(spyOntweetCollector).getTweetTask(USER_ID);
    when(tweetTaskMock.call()).thenThrow(InterruptedException.class);

    cloudService.addCollector(spyOntweetCollector);

    cloudService.openConnection(USER_ID);

    cloudService.saveObject(new Tweet("foo tweet"), USER_ID);

    cloudService.saveObjectCompleted(acknoledgeServiceMock, USER_ID);

    verify(tweetTaskMock, times(1)).call();
    verify(acknoledgeServiceMock, times(1)).sendAckFailed(any(RuntimeException.class));
}

kii web application

2014-12-11T18:32:27+00:00

The current tut explore how to create a simple mobile application with HTML5, CSS3 and JavaScript based on the new Kii Javascript SDK. Basically Kii Corporation offers a rich cloud mobile backend to help the development of a mobile application providing some services out of the box such as user registration and login. Then the application will be wrapped by Apache Cordova to obtain a deployable app for iOS and Android in a hybrid fashion.

Preview of the application in Plunker

A preview of the web application that is going to be implemented can be seen and tested in Plunker using jon/dohh as username/password or just registering a new user. Password must be at least 4 charaters and special ones are not allowed

In order to make it work I had to brutally copy/paste the content of the Kii Javascript SDK in a file because Plunker does not allow file upload.

Build steps

1. Create a Kii cloud application

First of all create a Kii account and then a new Kii application, put the name and select the HTML5 logo. Now download the Kii Javascript SDK.

2. Clone Github repository

Clone the repository containing the simple application

$ git clone https://github.com/blackat/kii-sdk-js-101.git

The application is the same application has been implemented in Plunker with the following structure:

├── css
│   ├── bootstrap.min.css
│   └── style.css                   # custom style
├── fonts
│   └── CherryCreamSoda.ttf         # a fancy font
├── index.html                      # single page of the web application
├── js
│   ├── angular.min.js
│   ├── angular.min.js.map
│   ├── kiisdk.js                   # Kii JavaScript SDK
│   └── script.js                   # custom script for Angular controllers
├── package.json
└── server.js                       # server side configuration

In the index.html a minimal form has been implemented to allow login and registration of the user. Just to make the UI a bit more fancy and easy to develop, Boostrap and AngularJS frameworks have been added.

2.1 Kii JavaScript SDK

The SDK has been used to perform login and registration action.

``` javascript js/script.js https://github.com/blackat/kii-sdk-js-101/blob/master/js/script.js $scope.login = function () {

KiiUser.authenticate($scope.user.username, $scope.user.password, {
    // Called on successful registration
    success: function (theUser) {

    },

    // Called on a failed authentication
    failure: function (theUser, errorString) {

    }
}) }; ``` Some details has been remove just to focus on the ``authenticate`` function. Once the credential has been sent to the cloud one of the two functions will be called according to the outcome of the authentication.

The registration function is a bit different, a new KiiUser object has to be created with the data coming from the form and then the function register has to be called. As for the login function two callbacks are available to manage possible process outcomes.

``` javascript js/script.js https://github.com/blackat/kii-sdk-js-101/blob/master/js/script.js $scope.register = function () {

// Create the KiiUser object
var user = KiiUser.userWithUsername($scope.user.username, $scope.user.password);

// Register the user, defining callbacks for when the process completes
user.register({
    // Called on successful registration
    success: function (theUser) {

    },
    // Called on a failed registration
    failure: function (theUser, errorString) {

    }
}); }; ```

3. Local test with node.js

In order to locally test in a browser the application node.js can be used. Download and install node.js. On mac use <a href=”http://brew.sh/”target=”_blank”>homebrew</a>

$ brew install node

once done use npm to install modules declared in package.json

$ npm install -g

where -g option means glabally, remove it to have all the modules installed locally in a folder named node_modules. Then, in the folder where the repository has been cloned, run the command

$ node server.js

and type in a browser tab http://127.0.0.1:5000/ to load the web page.

4. PhoneGap/Apache Cordova Installation

PhoneGap wraps a native web application written in HTML, CSS and JavaScript in a native application of a given platform such as iOS, Android, Windows8 and more.

Here it is possible to find a more detailed tutorial about hot to build a mobile application using PhoneGap.

PhoneGap vs Cordova

PhoneGap is a distribution of Apache Cordova. You can think of Apache Cordova as the engine that powers PhoneGap, similar to how WebKit is the engine that powers Chrome or Safari.

More here

Prepare the environment

Install Node.js if not yet done.

Install Apache Cordova

npm install -g cordova

Optional, to debug install Apache Ripple emulator

$ npm install -g ripple-emulator

Install Android SDK

$ brew install android

Install ‘‘actual SDK stuff’’

$ android

Install Genymotion to test the mobile application for Android devices.

Genymotion

Genymotion offers a better and more straight forward way to test an Android app than the native emulator. Once done register one or more virtual divices according to your preferences.

Pay attention about the specified API version of the virtual devices because it must match the one installed previously when android command has been run to install ‘‘actual SDK stuff’’, both are at version 21.

5. Create the Apache Cordova application

Step A - Create a Cordova project

Run the command

$ cordova create kii101phonegap com.kii.phonegap Kii101PhoneGap

This command will create a project named Kii101PhoneGap in the folder kii101phonegap.

Step B - Add project files

Copy the cloned repository content under the www into the project just created.

├── css
│   ├── bootstrap.min.css
│   └── style.css
├── fonts
│       └── CherryCreamSoda.ttf
├── index.html
└── js
    ├── angular.min.js
    ├── angular.min.js.map
    ├── kiisdk.js
    └── script.js

Step C - Add platforms

Switch to folder kii101phonegap and add the platforms against which the test will be done, for instance iOS and Android

$ cordova platform add ios
$ cordova platform add android

Step D - Build and emulate

Build the application and test it in iOS platform running

$ cordova build ios
$ cordova emulate ios

after the project have been built, an emulation window opens showing the developed application.

For Android it is a bit different, before start the Genymotion virtual device created in one of the previous steps, then in the command line type

$ adb devices

and see how the Genymotion virtual device is seen as a real one

List of devices attached
192.168.56.101:5555	device

so run the command

$ cordova run android

and the application will be deployed into the Genymotion device.

From the Kii Team

If you’re interested in more samples that use Kii Cloud please take a look at the GitHub page from the Kii team. Also, if you’re interested in other platforms, please check out their samples section available here.

Trobleshooting

Android target

Sometimes could happen that a prject has been created when a previuos android-sdk was installed, then having created a new device based on a more recent API the following error arises

ERROR: Error: Please install Android target 19 (the Android newest SDK).
Make sure you have the latest Android tools installed as well.
Run "android" from your command-line to install/update any missing SDKs or tools.

In the folder kii101phonegap/platforms/android of the Cordova application check the following files and keep android-sdk version consistent.

├── AndroidManifest.xml     # <uses-sdk> tag
├── local.properties        # location of the SDK. This is only used by Ant
├── project.properties      # project target