Socl: Large scale TypeScript project

Topics: General
Dec 5, 2012 at 3:57 AM

Hi everyone... My team just launched a major re-design to our website today and the new site is 100% written in TypeScript on the browser side.  We have a little over 30,000 lines of TypeScript in our project so we think it's a great example of using TypeScript to develop web applications at scale.  Please give the site a try and if you have any questions about what went into developing a site of this scale in TypeScript please ask away...

Oh and to the TypeScript team... Thanks again for blog post... If it weren't for you guys we'd still be trying to get this thing out the door :)


Dec 8, 2012 at 9:14 AM

First of all, a great looking site and a nice concept.

One questions I have is that it looks like you went for the internal module approach (device.pkg.js has at least the signature of internal modules usage in there). What was the reasoning behind this and not going for external modules using for example AMD?

I'm often struggling in deciding what would be the best approach for some of my pet-projects and my gut feeling is telling me that internal modules are easier to start with, but could lead to less maintainable code since the change on name-clashes is bigger. Especially if a project uses only 1 module name. Overall internal modules feel more monolithic to me.

Also incremental compilation seems much harder to do with internal modules. Every time you have to compile all 30.000 lines of code (at least when you use the tsc command line tool). 

Your insight would be much appreciated!

// Peter

Dec 9, 2012 at 2:22 AM
Edited Dec 9, 2012 at 2:34 AM

Thanks... And way to jump straight to the tough questions :)

There's a couple of things going on with the way we're using TypeScript.  We, obviously, started using TypeScript fairly early on (like 9 months ago so ages ago) and a lot of pieces weren't in place yet.  One of which was the ability to generate a single .js file from multiple .ts files.  That lead to us building our own tooling for doing just that.  We have something called the "packager" that converts these pkg files we define (essentially JSON based make files) into .pkg.js & .pkg.min.js files.  The packager drives the compilation of all our hundreds of TypeScript files and does other things like converting our UI templates (based on jQuery Templates) into TypeScript files and compiling our LESS files into CSS.  Once the packager has compiled everything it then does a combine and optional minification step with the ultimate goal being that we want a single file as output the contains everything needed to run our site.  As you point out incremental compiles are essential for a project of this size and our packager has it's own simplistic form of of incremental compile logic which is largely why we're still using it versus moving to TypeScripts single output file model.

As you observed we do in fact currently use internal modules versus external modules.  The reason for this is pretty simple.  If you've seen any of my other post you might notice a bit of a theme. My initial approach to TypeScript was to try and model everything the way I would in C#.  I love JavaScript and I love C#.  TypeScript to me is the perfect marriage of both those worlds so I wanted to see how much of the C# world I could bring into JavaScript via TypeScript (it turns out a lot.)  The decision to use internal modules was simply my attempt to model the namespacing found within C# projects in TypeScript.  It also let us enforce some team wide rules like saying that the namespace of objects MUST reflect where the code for those objects are located in the source tree.  For instance, the Socl.Data.Post class can be found in the "Socl/Data/Post.ts" file which is nice.

You pointed out that incremental compilation is hard with internal modules and you're correct.  But we have this package concept which allows us to define exactly which files (and therefore which classes) reside in a package.  The incremental compile logic of our package works on the package boundary so the only time we need to recompile a set of files is when we detect that one of the files within a "package" has changed.  Our packager supports dependent packages and anytime a file changes it re-compiles all of the files within that package as well as any of the packages that were dependent upon that package.

with all of that said... I am in fact currently evaluating the move from internal modules to external modules for some of the reasons you point out.  The broader reason for potentially making this move though is that today we have a 100% typescript UI which talks to a 100% C# backend.  I'd like to see us move some of the pieces of our backend to node.js so that we can start realizing better code reuse between our UI and server pieces.  In the node.js world it's all about CommonJS modules so to sort of unify the experience across these two worlds I'm looking into adopting the use of external modules across our entire system.


Edit: I should probably note that what I'm calling Backend above our server guys call the frontend.  The backend to them is our database & indexes that the frontend servers talk to.  Given that our UI is effectively a 100% browser based app I consider anything in the cloud as backend.  So it's some of the bits in the middle I'd like to see moved to node.js :)

Dec 9, 2012 at 3:24 PM

Thanks for the extensive reply, very helpful!!! Good to know what was the reasoning behind some of the decisions and the challenges your team faced. One of the things I didn't really think about is that for people with a C# background, internal modules feel very natural. 

For me with a different background, external modules seem a more natural fit. However I feel that a main downside of using external modules is that a single file is a module and as a result you need many imports (even more so if you are going to provide typing info). And almost everywhere in your code your are going to see a module prefix. Also if using a class per external module approach, it provides awkward naming issues. What to call the module that contains the single class?

import mWidget = module("./core/widget"); 
var widget = new mWidget.Widget(); 
add(widget: mWidget.Widget) {...}

I tried a number of prefixes, like the "m" prefix in the above example, but overall it makes the code less readable. One way around this is to put multiple classes into a single external module file (lets cal it core). So you would get:

import core = module("./core"); 
var widget = new core.Widget(); 
add(widget: core.Widget) {...}

This looks and feels better and also reduces the number of required imports, since the "core" module contains a number of classes.  However I feel that this approach provides  the wrong granularity (from a version control and maintainability perspective). But again this just could be my Java background.

I also played around a bit with mixing internal and external modules, but that felt always "artificial", although perhaps I should give it another go. I guess some more experimenting is required before a optimal solution is found.

// Peter

Dec 10, 2012 at 2:28 PM

It is useful to keep in mind that TypeScript support for ES6 modules is incomplete, and even ES6 modules haven't quite settled down yet (the latest committee meeting introduced a re-design the details of which I haven't seen yet). Among the details still under discussion are internal vs external modules and modules that export a single object. Also, the existing spec provides for some abbreviations that TS doesn't support yet, so using TS external modules is more awkward than it will be. And finally, TS has types, which aren't covered in ES6 modules (so one can abbreviate module paths and module value imports, but not module type imports).

For those who have practical experience with TS modules, it might help to ask for module support priority to be increased (I did so in Roadmap comments), and to keep an eye on the ES6 modules spec, making sure that it covers your use cases. It is difficult to foresee uses of a new spec, even if based on existing systems.

Btw, in my experience (with reusing typescript compiler and services code), TS references-to-internal/import-of-external modules do not mix well - either choice seems to be infectious (apart from hacks like presenting an external module front for internal module code).

Dec 14, 2012 at 2:49 PM

Congrats on a huge launch!

Some things I see from an SEO perspective:

  • The title tag for all you pages is SOCL, this should be the name of the collage
  • Consider changing the urls to be more SEO friendly (title of the post as part the url rather than that random string).  This used to help with SERP placement but at a minimum helps clickthrough rates on search results
  • How are you handling Webcrawler crawling of this content? 

Best of luck, great start!


Dec 14, 2012 at 8:39 PM

Thanks Jon... Yes, the title thing is something we noticed right as we were getting ready to launch and it didn't make the cut list of bugs to fix.  We're not currently crawlable as you've noticed but we do have a task to add support for "ugly" urls to address the crawling issue.  We haven't decided if we want to do web snapshots or just implement an unstyled HTML version of the site to deal with crawling but again it's another task on a long list.