Error in 0.8.2 w/ multiple files and imports

Topics: General, Language Specification
Jan 24, 2013 at 8:57 PM

I have a large project which, similar to the TypeScript compiler itself, is all emitted to a single out file via the --out parameter of tsc. I used to have imports inside the module declaration, but now in 0.8.2 that throws an error. But moving them out also throws an error about "dynamic modules".

Fails with "Import declaration of external module is permitted only in global or top level dynamic modules" for --out:

///<reference path='node.d.ts' />

module MyModule {
    import assert = module('assert');

    function doSomething() {
        assert.equal(1, 2, 'Fail!');
    }
}

And this fails with "Cannot compile dynamic modules when emitting into single file":

///<reference path='node.d.ts' />

import assert = module('assert');

module MyModule {

    function doSomething() {
        assert.equal(1, 2, 'Fail!');
    }
}
Is this regression intentional or a bug? Right now I am having to do "export var assert = require('assert');" and lose my type checking because of this issue.

Coordinator
Jan 25, 2013 at 5:32 PM

This is intentional.  The change in --out helps separate out the two ways of compiling a TypeScript application.

If you didn't use external modules and wanted to concatenate your sources, then you can use --out with a filename.  If you did use external modules, you likely want to control where they are compiled to, so we now support --out with a directory.

Does using --out with a directory help in this case?

Jan 26, 2013 at 1:53 AM

I need to specify a single file for all my TypeScript. I compile my single JS file the same way tsc.js is compiled. Now I see (after looking at io.ts) that I need to be literally calling require('nodemodule') to a var instead of what I previously did with import even though import saved my typing.

I guess my question is why the restriction? What is the harm of using external modules in a single emitted js?

Jan 27, 2013 at 8:37 AM

It could easily be done ... I am currently working on a builder which allows you to bundle your commonjs modules into a single file. It concatenates outputted js files and fixes require calls. Also merged declarations need some attention (duplicated references and paths). Maybe such feature should be implemented in compiler itself ...

Jan 27, 2013 at 11:50 PM

You can find this very easy to do with compiler, it's easy to add source units of text which are parsed on add and give a result script. Then run type check on the compiler. Then get the scripts and run them through the emitter. I have created my own emitter for another project I am working on. Extending the compiler is very easy. I've even extended the error reporter to swallow errors that are bugs in the current issue tracker. My biggest problem is working w/ the VS plugin which has supposedly been given some level of extensibility in 0.8.2, but nobody is providing details (ref: http://typescript.codeplex.com/discussions/429115).

Coordinator
Jan 28, 2013 at 4:30 PM

@cretz - we're planning on putting up a blog shortly discussing how to replace the language service for the VS plugin for users who are building from source.  Hopefully it will be up this week.

Also, to your question about external modules in a single file.  External modules assume that you're using a module loader.  In this case, you'd want your modules to be built into separate source files so that they can be loaded via the module loader.  Building all the modules into one .js file is counter to how these tools work.

I could see wanting to build external modules out of multiple source files, though we don't currently support this and might be a good feature request for the issue tracker, if it's not there already.

Jan 28, 2013 at 9:15 PM

Thanks Jon, awesome answer. I look forward to the blog post.

As for the external module issue, another option would be to let me do type assertions on modules, so I could var fs = <fs>require('fs'); which would allow me to resolve my typing issues. Sometimes I am on the end of the package manager where I am using dependencies and not necessarily publishing or making a package, so being able to leverage the module loader does help. So I don't necessarily want to build an external module, just a application that leverages them. I'm not so sure it warrants an issue in the tracker, as I see in src/compiler/io.ts y'all opt to use require(NAME) instead of module(NAME) for nodejs modules too.

Thanks again for the update.

Jan 29, 2013 at 11:58 AM
Edited Jan 29, 2013 at 12:00 PM

@jonturner

"External modules assume that you're using a module loader.  In this case, you'd want your modules to be built into separate source files so that they can be loaded via the module loader.  Building all the modules into one .js file is counter to how these tools work."

I think the assumption you are making here is not correct in all cases. For example, If I developed a library in typescript that may be divided in many classes, which I for convenience put in different files, I may want to compile it in one single .js file as well as also making it capable of being loaded via an AMD loader, and I may besides want to load dependencies of my library via AMD.

As a matter of fact we are currently developing a web framework in Typescript. The framework consists of a few dozen files, and is very modular and nice, but our release for javascript users is one single js file that is less than 15Kb, but that depends on "curl" and "underscore". Curl is the AMD loader itself, while underscore is loaded dynamically by our framework... I do not think we are doing things in such an ortodox manner, in fact we want to do it addering to common best practices as much as possible. So please reconsider the possibility of having single output files + support for AMD.

kind regards,

Manuel.

Jan 29, 2013 at 1:40 PM

@jonturner

After a quick discussion with my partner, I can add another argument to why this is incorrect:

Normally, when developing a web application using AMD modules, you keep them as separate modules during development, but you create one single file when deploying, for performance reasons. In this scenario, it is quite common that your web app depends on both external AMD modules and also global modules. With the new restriction that you are imposing in the typescript compiler you are making this kind of development pattern much more difficult (or imposible, since I have yet not found a workaround for this...).

Jan 29, 2013 at 1:48 PM

@jonturner

ok, so for what is worth, the workaround we found is just to import the external dependencies as globals. So if your typescript library depends on underscore, in nodejs something like this will work:

_ = require('underscore');

Quite hackish but it works...

regards,

 

Manuel.

Coordinator
Jan 29, 2013 at 6:42 PM

@manast - thanks for all the feedback, that's helpful.  You're right in that there are cases you might want to create one external module from multiple files.

We've been viewing the TypeScript compiler as just one of the tools you'll need in your toolchain.  Like you say, you might have a module loader and a few more libraries that you may load dynamically.  When you're in development you want to build things one way and another way when doing a release.  

Being able to handle the variety of build styles isn't something we'd want to build into the compiler.  Instead, making a concatenation a separate part of your toolchain is probably a better idea.  Such a tool could optionally take the .js files output by the TS compiler, concatenate them into the same .js file, and potentially also doing minification and/or obfuscation.  

Jan 29, 2013 at 7:59 PM

@jonturner

Actually we already implemented the concept that you are proposing, i.e., ts compilation, concatenation using requirejs, minimization with uglifyjs and zlib compression, it is implemented on our open source static file server Cabinet.

Still, we encounter some problems in how the module system is implemented in typescript. We have found some inconsistencies that makes things more difficult than they should be, for example, you need to use /// references if you want to take advantage of the modules system (yes you can use import, but modules will not be open anymore, you cannot span the same module across different files). And as soon as you use /// references you cannot compile typescript to generate AMD compatible modules... so it is a bit of a chicken and egg problem. Before 0.8.2 we could do workarounds, now the workarounds start to look like hacks (such as being forced to use global variables to expose external modules).

I think we may require a separate, more complete discussion, and I would certainly want to have it with you if you have time.

regards,

 

Manuel.


Jan 30, 2013 at 2:56 PM

I fully agree with manast.    I also struggle with /// <reference > versus  import.   I want to be able to split a library into several .ts files and compile to .js that can be used by requirejs.  When I use compilation with --out to single file, then I cannot even write "export" without getting the "dynamic modules" error. My only solution so far is to do postprocessing of the .js file (that breaks the sourcemap feature since it adds  some lines at the beginning and end:

 

define(["require", "exports"], function(require, exports) {

// -- output of typescript compilation --
var Main = (function () {
  //....
}

// -- end of output of typescript compilation

exports.Main = Main;
})

 

Is there any smarter way to do this?

Jan 30, 2013 at 8:32 PM

@wme

 

The workaround we found to solve the problem you have is to create a .ts file with all the dependent modules, and after all the references we added a shim that makes the resulting javascript code compatible with both AMD and CommonJS (nodejs), so here is how it looks like in our framework, where Gnd is our root module (note as well the windowOrGlobal hack we had to do because of the new restriction in typescript 0.8.2 of not letting us using the this pointer in modules anymore...):

 

/// <reference path="./lib/server.ts" />
/// <reference path="./lib/storage/socket-backend.ts" />
/// <reference path="./lib/storage/mongoose-backend.ts" />

declare var define;
declare var exports;
declare var window; // Browser
declare var global; // Node.js

var windowOrGlobal;
if(typeof window === 'undefined'){
  windowOrGlobal = global;
}else{
  windowOrGlobal = window;
}

(function (root, factory) {
  if (typeof exports === 'object') {
    for(var k in factory()){
      exports[k] = factory()[k];
    }
  } else if (typeof define === 'function' && define.amd) {
    // AMD. Register as an anonymous module.
    define(factory);
  } else {
    // Browser globals (root is window)
    root.returnExports = factory();
  }
}(windowOrGlobal, function () {
    return Gnd;
}));

Jan 31, 2013 at 6:53 AM
Thanks for the advice. So I can do without postprocessing by simply adding the define at the end of my main .ts file. This still seems like a workaround for simply writing "export".
/*export*/ class Main {
}

define(["require", "exports"], function(require, exports) {
  exports.Main = Main;
})
Feb 26, 2013 at 2:15 AM
jonturner wrote:
@cretz - we're planning on putting up a blog shortly discussing how to replace the language service for the VS plugin for users who are building from source.  Hopefully it will be up this week. Also, to your question about external modules in a single file.  External modules assume that you're using a module loader.  In this case, you'd want your modules to be built into separate source files so that they can be loaded via the module loader.  Building all the modules into one .js file is counter to how these tools work. I could see wanting to build external modules out of multiple source files, though we don't currently support this and might be a good feature request for the issue tracker, if it's not there already.
Feature request added.