Best way to encapsulate modules for exporting?

Topics: General
Jul 16, 2013 at 1:21 PM
I have been playing with Typescript for a while now but am still slightly confused as to how exporting of modules works.

As my current approach is to made all modules internal and just export the classes, then if I use the ///reference syntax to include them I can use the exported classes via the module as a namespace.

So for example:
// foo/bar.ts
module Foo
{
    export class Bar
    {
        public DoSomething(){};
    }
}

// yellow/blue.ts

///<reference path="../../foo/bar.ts" />
module Yellow
{
    export class Blue
    {
        public Bar Instance;
    }
}
Now this worked ok for smaller projects, however now I am using typescript with a NodeJS based project I would like to be able to export my modules and them import (or require in v0.9) them so they are more modular and not a huge mess of interconnected files.

Now in the example above I may have 20 files within the 'Foo' module, and from what I can tell the 'export module' syntax only works on an individual file, so if I were to have 2 files within the 'Foo' directory which both were part of the 'Foo' module and both had the 'export module Foo' syntax it is implied that this wont work as there should only be one module exported, although I dont know if this is actually correct or not, I kept getting errors around this when I tried to compile this way with multiple files.

So I am wondering how I can lay my classes out in a namespace style fashion, but then just generate a single output for this module so node can just import it, I was thinking I could keep all the namespaces but remove the root "Foo" module and then somehow make a wrapper ts file which is the single export module declaration and somehow composes all the internal modules within itself to export them, if that makes sense... but the documentation around all of this stuff is sparse, so has anyone got any hints on best way to have it so I can have multiple classes/files within a module at varying depths within a namespace but compile it to a single file which can be imported like so:
import foo = require("Foo");
var something = new foo.Bar();
Coordinator
Jul 18, 2013 at 3:57 PM
There are a couple of points here.

Working with external modules, which you're doing by using 'import foo', moves us into the module loader style of code. We landed on a design here that tries to play to the style that people generally use with module loaders, which is that each separate bit of contained functionality is its own separately loaded module. This is in contract to treating them as a namespace. Instead, external modules each fit in separate files, and use top-level imports and exports to coordinate across this file boundary with the module loader.

TypeScript also has internal modules using the 'module' keyword, which you also show. These do tend to work a lot more like namespaces and are more of a convenience for grouping related functionality together. There is some similarities with external modules, in that you can export properties to make them visible outside of the module. But they're don't have any affect working with module loaders.
Jul 19, 2013 at 9:44 PM
Edited Jul 21, 2013 at 1:47 PM
Hi Jon, Grofit and fellow TypeScript enthusiasts,

I decided to use this thread as an opportunity to share a simple looking pattern that solves just this problem - of using internal modules (namespaces) with CommonJS (in Node.js). With this little "magic trick", my application can use the same (internal) modules both in the browser and in Node without any issues and almost no constraints or limitations. It is based on the assumption that an internal module is basically a regular Javascript object that is being continuously referenced and extended in multiple source files.

This "hack" may look quite trivial at first (though in practice it isn't really, as it's quite subtle) but actually depends on correct cycle resolution capability of the module loader to work correctly. It is also based on a second assumption that a CommonJS module is loaded only once and has a single, unique, object that is associated with it. This object is then used to provide a "context" and then further extended the same way that a browser based "Internal" Module object would. Here's the general layout of the pattern:
InternalModuleLoader.ts
-----------------------
require("./ModuleFile1");
require("./ModuleFile2");
require("./ModuleFile3");
require("./ModuleFile4");


ModuleFile1.ts
--------------
///<reference path='ModuleFile1.ts'/>
///<reference path='ModuleFile2.ts'/>
///<reference path='ModuleFile3.ts'/>
///<reference path='ModuleFile4.ts'/>

if (typeof require == "function")
    InternalModule = require("./InternalModuleLoader");

module InternalModule
{
    export class Class1
    {
    ..
    }
}

ModuleFile2.ts
--------------
///<reference path='ModuleFile1.ts'/>
///<reference path='ModuleFile2.ts'/>
///<reference path='ModuleFile3.ts'/>
///<reference path='ModuleFile4.ts'/>

if (typeof require == "function")
    InternalModule = require("./InternalModuleLoader");

module InternalModule
{
    export class Class2
    {
    ..
    }
}

...
Some remarks and caveats:
1: Module source files may load in an arbitrary order (depending on the "entry" file that's been loaded first). Code that exists inside the module scope but not deferred within a class or function may fail or not work as expected (in my app I never assume any order so that's not a problem). [Edit: this assertion appears to be incorrect - at least in the Node implementation - source files do load in the order they are required - see next post below for experimental results]

2: As I've mentioned, this pattern creates (a lot! of) cyclical references and depends upon the ability of the loader to resolve them (which Node seems to do correctly - using "partially done objects", but other loaders may not). It also assumes that once a CommonJS module has been "required" it will always maintain the same memory reference and never be reloaded again.

More detail from the Node manual:
(http://nodejs.org/api/modules.html#modules_caching)
Caching:
Modules are cached after the first time they are loaded. This means (among other things) that every call to require('foo') will get exactly the same object returned, if it would resolve to the same file.

Multiple calls to require('foo') may not cause the module code to be executed multiple times. This is an important feature. With it, "partially done" objects can be returned, thus allowing transitive dependencies to be loaded even when they would cause cycles.
Some more info on cycles can be found here:
http://nodejs.org/api/modules.html#modules_cycles


3: If trying to use the pattern in conjunction with CommonJS modules described in external module declarations (pretty much all of them), TypeScript will complain it doesn't allow mixing internal and external modules on the same source file. My current (and rather tedious) solution is to convert the "external" module declarations to "internal" ones (though I have two regular expressions that automate this). I then load the modules in the following way: "var http = <http> require("http");". (Perhaps after reading this the TypeScript team might reconsider the decision to be so strict on that matter. At the end it's a developer choice - if they want to mix, why not?)

4: Note that the variable for the internal module object was chosen to be the "loader" module object simply out of convenience. I could, as well have put it on the "global" object (or some other shared object) but I thought this would give even better encapsulation (though it might create a bit of confusion since the loader itself does not actually export anything).

.. some other "complications" I'm not aware of / never encountered ..


Overall it works very well for my needs and I'm generally satisfied with it. I'm interested to hear what the TypeScript team has to say about this (perhaps you'll consider integrating something of this sort into the compiler? or at least make it easier to implement?). I guess it's not the way they intended the module system to be used but this is indeed very simple and have saved me a lot (!) of headache (something others have described as the "module hell/confusion" and I have to agree on that).

Basically there are 3 different module systems that are not compatible with each other - to the point where the compiler even actively (and somewhat forcibly!) forbids any attempt to create some sort of compatibility layer between them. I doubt this is an optimal (or even, in practice, manageable) solution? Anyway this is my best attempt at trying to "bridge the gap", at least where it was important for my own work.
Jul 21, 2013 at 1:33 PM
Edited Jul 21, 2013 at 1:45 PM
Reading the Node module docs again (specifically about circular references and incomplete objects) I started to have a doubt on one of my assertions and it appears to be wrong. The module files do load in the order they are required, with the exception of the "entry" file (the file that's first to invoke the loader) - that will always be loaded last - and that makes sense anyway. Here is some experimental code I wrote to confirm this (this is also verified to compile correctly, of course):

ModuleLoader.ts:
///<reference path='References.ts'/>

require("./ModuleFile1");
require("./ModuleFile2");
require("./ModuleFile3");
require("./ModuleFile4");
Module1.ts:
///<reference path='References.ts'/>

if (typeof require == "function")
    InternalModule = require("./ModuleLoader");

module InternalModule
{
    export var test1 = 1;
}

console.log("ModuleFile1 loaded, content: " + JSON.stringify(InternalModule));
Module2.ts:
///<reference path='References.ts'/>

if (typeof require == "function")
    InternalModule = require("./ModuleLoader");

module InternalModule
{
    export var test2 = 2;
}

console.log("ModuleFile2 loaded, content: " + JSON.stringify(InternalModule));
Module3.ts:
///<reference path='References.ts'/>

if (typeof require == "function")
    InternalModule = require("./ModuleLoader");

module InternalModule
{
    export var test3 = 3;
}

console.log("ModuleFile3 loaded, content: " + JSON.stringify(InternalModule));
Module4.ts:
///<reference path='References.ts'/>

if (typeof require == "function")
    InternalModule = require("./ModuleLoader");

module InternalModule
{
    export var test4 = 4;
}

console.log("ModuleFile4 loaded, content: " + JSON.stringify(InternalModule));
References.ts:
///<reference path='ModuleFile1.ts'/>
///<reference path='ModuleFile2.ts'/>
///<reference path='ModuleFile3.ts'/>
///<reference path='ModuleFile4.ts'/>

declare var require;
Output when invoking Module4.ts as an "entry point":
ModuleFile1 loaded, content: {"test1":1}
ModuleFile2 loaded, content: {"test1":1,"test2":2}
ModuleFile3 loaded, content: {"test1":1,"test2":2,"test3":3}
ModuleFile4 loaded, content: {"test1":1,"test2":2,"test3":3,"test4":4}
Output when invoking Module2.ts as an "entry point":
ModuleFile1 loaded, content: {"test1":1}
ModuleFile3 loaded, content: {"test1":1,"test3":3}
ModuleFile4 loaded, content: {"test1":1,"test3":3,"test4":4}
ModuleFile2 loaded, content: {"test1":1,"test3":3,"test4":4,"test2":2}
So basically this shows that "internal" module source files can be loaded in the exact same way and order as they load in the browser. I've been using this pattern for about two months now with great success (and with more than 10 source files included). There are still some limitations imposed by the compiler that make this pattern difficult to work in conjunction with the more "standard" CommonJS language support. I hope that with the collaboration of the TypeScript team and users we can make this a more "standardized" method - rather than just a "hack" or "design pattern" - since the language is still in alpha and hasn't been stabilized, I think this might the right time to discuss this.