Compiling node.js project with both import and reference

Topics: General
Sep 17, 2013 at 8:44 PM
Edited Sep 17, 2013 at 10:12 PM
Hello,

I'm trying to create a simple node.js project with typescript, but I just can't structure my code into several files as I usually do. It seems that I would have to put everything in one big file to be able to compile it to usable javascript.

Used Typescript version: 0.9.1.1

Here is a simple test project of 3 files:

main.ts:
/// <reference path="config.ts" />

import my = require("myclass");

console.log("Main: " + Config.secret);

var m = new my.MyClass();
m.hello();
The main file references a config file and the myclass external module.

myclass.ts:
/// <reference path="config.ts" />
/// <reference path="node.d.ts" />

import util = require("util");

class MyClass { 
    hello() {
        util.log("Hello World: "+Config.secret);
    }       
}
Unfortunately this module cannot be referenced with a reference comment, because it automatically becomes an external module when I put the import ... require statement in it to use the util node.js module. But note that it also uses the config file.

config.ts:
module Config {
    export var secret = 42;
}
This file is referenced by the other two. They both use the secret constant.

Now the question is:
How can I compile this simple project with tsc to a working javascript code?

Let's see what I got:

1. Let's try to build each file separately
tsc main.ts --module commonjs
node main.js
main.js:4
console.log("Main: " + Config.secret);
ReferenceError: Config is not defined
So it compiles. I get: main.js, myclass.js, config.js. But they are not connected in any way which means that if I try to run this version the main.js won't see the config file. I would have to manually concatenate the two.

2. Let's ask typescript to put everything in one output file:
tsc main.ts --module commonjs --out main.js
node main.js
It also compiles. I get two files: main.js and myclass.js. But when I try to run it, nothin happens at all. Why? Let's see the generated main.js.

main.js:
var Config;
(function (Config) {
    Config.secret = 42;
})(Config || (Config = {}));
What?? The main.js contains the config file, just as I would expect I referenced it from main.ts, so it gets compiled to main.js. But where is the rest of the file? It's missing!

Let's see what we got in myclass.js.

myclass.js:
/// <reference path="config.ts" />
/// <reference path="node.d.ts" />
var util = require("util");

var MyClass = (function () {
    function MyClass() {
    }
    MyClass.prototype.hello = function () {
        util.log("Hello World: " + Config.secret);
    };
    return MyClass;
})();
Uh-oh. No config here. This wouldn't work either. This somehow didn't understand my wish to put referenced files where they need.

3. I'm out of ideas. I don't know any other method how I could compile this project to a working javascript program.

Let's see what I would expect from the compiler:

1. The easiest way and simplest way: The compiler would take each external modules, compile them separately and it should copy all the referenced files in each and every external module. In this case main.js and myclass.js should both contain the config file, so they can use it.

Like this:

main.js should be:
var my = require("myclass");

var Config;
(function (Config) {
    Config.secret = 42;
})(Config || (Config = {}));

console.log("Main: " + Config.secret);

var m = new my.MyClass();
m.hello();
myclass.js should be:
var util = require("util");

var Config;
(function (Config) {
    Config.secret = 42;
})(Config || (Config = {}));

var MyClass = (function () {
    function MyClass() {
    }
    MyClass.prototype.hello = function () {
        util.log("Hello World: " + Config.secret);
    };
    return MyClass;
})();
2. The smart way: The compiler shouldn't consider myclass.ts an external module just because I have imported a node.js module in it. It should be easily referenced with the reference comment in main.ts. And the resulting main.js should contain MyClass and also Config. The compiler should be able to understand that they both need the Config module so it would come first.

Something like this:

main.js should be if the compiler would be very smart:
var Config;
(function (Config) {
    Config.secret = 42;
})(Config || (Config = {}));

var util = require("util");

var MyClass = (function () {
    function MyClass() {
    }
    MyClass.prototype.hello = function () {
        util.log("Hello World: " + Config.secret);
    };
    return MyClass;
})();

console.log("Main: " + Config.secret);

var m = new MyClass();
m.hello();
So I could get a nice and working program. What do you think? Should I put everything in what huge app.ts file? Isn't there any way that I could structure my node.js app source code into several files? At the end of course I want one javascript file, or at least a file for each module, but I expect the compiled program to run and work properly in node.js.

Any ideas how I could achieve this?

Thank you!

ps: When I started my research about this topic I didn't completely understand why I couldn't reference a file that contains import ... require. I have wrote about my experiences here: https://typescript.codeplex.com/workitem/1690 What about using var ... require to use node modules?
Sep 25, 2013 at 1:45 AM
In Node.js every piece of code you load is a module - including the very first file you specify on the command line. Nothing you write contributes to the global scope. Everything should be compiled with --module commonjs.

When compiling for the global scope (i.e. simple browser files loaded via "script" tags, and NOT using the --module switch), you can use --out to join files together. However TypeScript does not support multi-file external modules. Each input source file should map to one external module when using --module.

It appears you are trying to join the config.ts file into your modules via /// references. Due to the above rule, this can't be done. This code either needs to be within the source file, or you need to load it as an external module (i.e. "import config = require('./config');" in the consumer, and "export var secret = 42;" in the config.ts).

Basically, you should not use /// reference tags in external modules for anything other than declarations of other external modules (i.e. as the node.d.ts file does to declare module "path", and "fs" etc....). It doesn't make sense to /// reference other implementation code due to our one-on-one source-to-module mapping, and the fact that in Node everything you write is an external module.

Hopefully this makes sense.
Sep 25, 2013 at 11:48 PM
Thank you for your explanation.
I'll try to put everything in an external module then if there is no other solution.

What about writing this in typescript:
var util = require("util");
Then the typescript compiler wouldn't consider my source file as an external module, so I could just reference everything and compile the program into one file with the --out option. I tried it and it works, but with this trick intellisense stops working in visual studio. Is there a way to make typescript write "var util = require("util");" to the output and still declare util as a module somehow?

I want this so badly because it doesnt seem right to create an external module for each class just because I want to write classes in separate files. Maybe a preprocessor could help that would merge ts files into one before compilation?
Sep 26, 2013 at 4:45 AM
I also found the compilation behavior of TypeScript with multiple associated files to be awkward in my project. It would be great to have a story closer to "partial classes" found in VB/C#. The solution I was able to come up with was keeping each of the files separate in the initial build (meaning that I have to reference each one separately in my tests html file), but to then combine them in the minification step using UglifyJS 2 (so I have separate HTML files that run the tests against the "built" JS files and the minified JS files).

I'm honestly allowing my core library file to grow to an unmanageable size mainly because I'm afraid of refactoring it due to the pain of how TypeScript 0.9.1.1 deals with many files declaring the same module name.

If anyone is interested, the Node.js script that I use to build my TypeScript project and minify the results is here:

https://github.com/ShamnaSkor/WafleProject/blob/master/Wafle/Wafle/build/build.ts

PS: I'm also using an "eval" trick on line 30 that I found on Stack Overflow to load the compiled wafleCore.js so I can pull out the version number which is stored in a variable - that's one way to use the outputted JS files without requiring them to be CommonJS modules.
Sep 26, 2013 at 6:05 PM
It seems that some functionality got lost when modules lost their interfaces: we can no longer refer to the interface of a string-named module (like module "util").

For non-string-named modules, we could use
declare module u {
    export function format(format: any, ...param: any[]): string;
    ...
}

var util:typeof u = require("util");
to get external modules imported without creating an external module, while still getting IDE functionality. But in the case of string-named modules, I don't know of a valid syntax to express the same idea.
Oct 2, 2013 at 6:57 AM
If you don't reference an imported module in a value position (i.e. only reference it for types) then it will get optimized away, and not imported at runtime. Thus if you declared module foo in foo.ts such as the below
declare module "foo" {
    export var x: number;
    export function y(a: string): string;
} 
You could then reference the declaration, import the module, use it only in the type position to declare a variable of its type (via the typeof keyword), and the actual import at the top-level would get optimized away, e.g.
/// <reference path='foo.ts'/>

import footype = require('foo');

// later...
var x: typeof footype = require('foo');
x.y; // etc...
Note however that in many cases trying to defer load like this is an anti-pattern. For module loaders such as RequireJS, it looks in the function for "require" calls (via toString) and still asynchronously loads the module as a dependency before entering the function (even if the 'require' call is in an "if" block that never runs). For Node, the loading is synchronous, so it does happen on demand, however unless there are a lot of modules that probably won't be needed, I still err on the side of trying to load dependencies on startup, and avoid the latency of the disk fetch when it comes to performing the processing needs it. But that's just me :-)
Jun 20, 2014 at 6:39 PM
This is how I'd done it (https://github.com/andraaspar/node-js-test):

1) I've rewritten node.d.ts to remove the quotes from module names.
declare module "http" {...}
became
declare module http {...}
2) I've removed unnecessary imports from node.d.ts, like this one:
import events = require("events");
3) I've captured a reference to the global object like this:
module illa {
    export var GLOBAL = (function() {
        return this;
    })();
}
4) Written code like this (https://github.com/andraaspar/node-js-test/blob/v1/src/Main.ts):
/// <reference path='node.d.ts'/>
/// <reference path='../lib/illa/Log.ts'/>

illa.GLOBAL.http = require('http');

var server = http.createServer(function(request, response) {
    response.writeHead(200, {'Content-Type': 'text/plain; charset=utf-8'});
    response.end('Árvíztűrő tükörfúrógép', 'utf-8');
});
server.listen(8888);

illa.Log.info('Yay!');
This can then be compiled into a single file (https://github.com/andraaspar/node-js-test/blob/v1/build/test.js), and keeps type checking intact. The only thing you've gotta be careful about is requiring all node modules you use.