Making Private methods completely "private"

Topics: General
Jun 24, 2013 at 2:30 PM
This is what we enter:
class Greeter {
    greeting: string;
    constructor(message: string) {
        this.greeting = message;
    }
    greet() {
        return "Hello, " + this.greeting;
    }
    private privMessage:string;
    private fooMessage() {
        return this.privMessage;
    }
    setPrivateMessage(m:string) {
        this.privMessage = m
    }
    greet2() {
        alert ("HOLA! " + this.fooMessage());
    }
}

var greeter = new Greeter("world");
greeter.setPrivateMessage("abc");
greeter.greet2();

var greeter2 = new Greeter("world");
greeter2.setPrivateMessage("def");
greeter2.greet2();
This is what we get after compilation :
var Greeter = (function () {
    function Greeter(message) {
        this.greeting = message;
    }
    Greeter.prototype.greet = function () {
        return "Hello, " + this.greeting;
    };

    Greeter.prototype.fooMessage = function () {
        return this.privMessage;
    };
    Greeter.prototype.setPrivateMessage = function (m) {
        this.privMessage = m;
    };
    Greeter.prototype.greet2 = function () {
        alert("HOLA! " + this.fooMessage());
    };
    return Greeter;
})();

var greeter = new Greeter("world");
greeter.setPrivateMessage("abc");
greeter.greet2();

var greeter2 = new Greeter("world");
greeter2.setPrivateMessage("def");
greeter2.greet2();
this is what I want:
var Greeter = (function () {
    var privMessage;
    function Greeter(message) {
        this.greeting = message;
    }
    Greeter.prototype.greet = function () {
        return "Hello, " + this.greeting;
    };

    var fooMessage = function () {
        return privMessage;
    };
    
    Greeter.prototype.setPrivateMessage = function (m) {
        privMessage = m;
    };
    Greeter.prototype.greet2 = function () {
        alert("HOLA! " + fooMessage());
    };
    return Greeter;
})();

var greeter = new Greeter("world");
greeter.setPrivateMessage("Anshul");
greeter.greet2();

var greeter2 = new Greeter("world");
greeter2.setPrivateMessage("Anirudh");
greeter2.greet2();
Basically what I want is to make private methods completely "private". As it can be seen clearly what I want from above code.
So I want to implement this feature on TypeScript (not as default) but the one which can be enabled with a flag.
Coordinator
Jun 24, 2013 at 4:03 PM
This has come up in the past a few times on the forums (like https://typescript.codeplex.com/discussions/397651). The short of it is that closing over privates takes up more memory and is not the recommended codegen from a performance perspective, so we enforce the visibility restriction at the type system level, during compilation.
Jun 24, 2013 at 6:55 PM
Hi Jon!

I am working with the original poster of the suggestion (anshul) on this idea. I would like to answer your question.

I understand that this has been suggested in the past and am aware of the performance costs of this. But the reason for our concern is mainly 2 items:
  1. We are developing a JavaScript library using Typescript. We want to expose specific public methods in our API (much like in C#, Java, etc). But we also want to prevent people from using private functions as they might mistakenly use it as their IDE might auto-complete the code.
  2. We found that the compressing code happens a lot better when the private variables are not in the prototype of the class.
As a result of it, we decided to modify the compiler and add this behavior only when a certain flag is passed to the compiler like "--hide-private-members". This way we can have the best of both worlds - better performance for intensive stuff, and smaller filesizes and cleaner APIs for other stuff.

We just want to know whether the typescript team would be interested in merging this function, because we do not wish to maintain a personal fork. Instead we'd prefer to work with you and finalize on a design plan that will work well for the entire typescript community and implement it.

Thanks,
Anirudh
Coordinator
Jun 24, 2013 at 8:44 PM
Possibly, though I doubt before 1.0. The problem here is that opting into closing over private members is API-incompatible with the existing system, which might confuse users. That said, I can imagine people wanting API cleanliness more than perf, and opting to use this.
Jun 25, 2013 at 4:40 AM
Hi Jon!

I understand the pros-and-cons of making privates closed. The biggest downside would be that some other JS code was using your private variable (which was only private in typescript, not in JS) Now if this becomes the default behavior of the typescript compiler, this might make huge breakages to existing codebases in subtle and unpredictable ways.

We are already implementing a proof-of-concept patch to the compiler to do this. And we are ensuring that the default behavior remains the same and this behavior is only active when a flag is passed.

The questions we have for the typescript team are this:
  1. If you feel that this is beneficial to the typescript community, and that the code and tests are acceptable quality, will you be willing to merge this feature into the main compiler?
  2. Will it be okay if we post our design notes and details of our own going work for your feedback and advice?
The reason we want to merge this upstream, rather than maintain a personal branch is because we feel that since Typescript is moving fast it might be more complex to maintain down the line, and we're a small team and we don't have resources to maintain a complete fork of the compiler.

Thank you,
Anirudh
Jun 25, 2013 at 10:37 AM
Wow! This is a major breakthrough:
var Greeter = (function () {
    var privMessage;
    function Greeter(message) {
        this.greeting = message;
    }
    Greeter.prototype.greet = function () {
        return "Hello, " + this.greeting;
    };

    var fooMessage = function () {
        return privMessage;
    };
    
    Greeter.prototype.setPrivateMessage = function (m) {
        privMessage = m;
    };
    Greeter.prototype.greet2 = function () {
        alert("HOLA! " + fooMessage());
    };
    return Greeter;
})();
Now we can have private variables. Um...no... wait:
var greeter = new Greeter("world");
greeter.setPrivateMessage("Anshul");
greeter.greet2();

var greeter2 = new Greeter("world");
greeter2.setPrivateMessage("Anirudh");
greeter2.greet2();

greeter.greet2(); // What do you think this is?
with sincere apologies for the ugly sarcasm.
Jun 25, 2013 at 10:47 AM
nabog:

My guess from your tone is that you are not happy with this proposed change. If you can explain your concern more clearly I'll be happy to clarify them, or even discard our changes if you can explain to us why the benefits won't outweigh the costs.

Please note that we do not wish to change the default behavior of typescript, but rather add this as an optional feature which people who want this specific functionality can activate using a compiler flag.
Jun 25, 2013 at 12:16 PM
@skyronic, nabog's point was that your implementation of private variables is actually more of a static private, in that it gets shared between all instances of the class.
Jun 25, 2013 at 12:35 PM
Check this fiddle:

http://jsfiddle.net/rezhE/

Private variables are really private and only specific to the instance. The next challenge is that "this" keyword doesn't behave properly in private functions (basically they won't be able to access public methods) but we have found a way to fix that as well:

http://jsfiddle.net/gQUyy/1/

This requires minimal change to the codegen.
Jun 25, 2013 at 2:13 PM
That's amazing. Are you going to charge people to use your fork or is it going to be open source?
Jun 25, 2013 at 2:16 PM
nabog:

We will contribute the changes back to typescript. We're going to push to a fork on codeplex.

We will write it with TypeScript's contributor and code guidelines in mind, write unit tests and ensure that it's available to everyone. We'd be absolutely psyched if it can be merged before the 1.0 release of TypeScript.
Jun 25, 2013 at 2:17 PM
@skyronic Try out nabog's code, specifically, pay attention to the line where he says, "What do you think this is?". You will get the value "HOLA! Anirudh", when what you were probably expecting was "HOLA! Anshul". The value of privMessage is being shared between all instances of the class.

@nabog I don't think you need to be a jerk about it.
Jun 25, 2013 at 2:20 PM
Hi MgSam,

After further examination, you're right. The instance variables are being shared.

I apologize for not testing things out more thoroughly before posting here. I will try and see if I can find a solution. Otherwise we'll conclude our efforts in this direction.
Jun 25, 2013 at 2:33 PM
Edited Jun 25, 2013 at 2:33 PM
@skyronic There IS a way to simulate private members in JavaScript, it just uses a lot more resources as it requires you to not use the prototype and instead have the methods redefined on every instance.
var Greeter = (function () {
    function Greeter(message) {
        var privMessage;
        var fooMessage = function() {
            return privMessage;
        };
        this.greeting = message;
        this.greet = function() {
            return "Hello, " + this.greeting;
        };
        this.greet2 =  function () {
            alert("HOLA! " + fooMessage());
        };
        this.setPrivateMessage = function (m) {
             privMessage = m;
        };        
    }
    return Greeter;
})();

var greeter = new Greeter("world");
greeter.setPrivateMessage("Anshul");
greeter.greet2();

var greeter2 = new Greeter("world");
greeter2.setPrivateMessage("Anirudh");
greeter2.greet2();
greeter.greet2(); //This will output "HOLA! Anshul", as you'd expect

Jun 25, 2013 at 3:01 PM
Just a bit of tongue-in-cheek - no offense meant.

@skyronic, you could also try implementing private methods on your fork.

var Foo = (function () {
    function Foo() {

        // Ensure "this" context is provided
        getFoo.call(this);
    }

    // Private method
    function getFoo(){
    }

    return Foo;
})();

This at least partly addresses the problems you outline above.
Jun 25, 2013 at 4:04 PM
Edited Jun 25, 2013 at 4:05 PM
Honestly, if you're a developer not satisfied with private fields as a design time construct, then you need to do the leg work to do it yourself. There isn't anything you can do in JavaScript you can't do in TypeScript as far as I'm aware. Here's using TypeScript to reveal variables in the constructor:
class Greeter {
    getMyMessage: () => string;
    
    constructor(message: string) {
        this.getMyMessage = () => message;
    }
    greet() {
        return "Hello, " + this.getMyMessage();
    }
}
If you don't wish to have a getter/setter, you must assign all functions that use the "private member" within the constructor - just like JavaScript:
class Greeter {
    greet: () => string;
    
    constructor(message: string) {
        this.greet = () => "Hello, " + message;
    }
}
Jun 25, 2013 at 4:41 PM
@MgSam: Thanks for the explanation. I don't think that we have the skill to modify the compiler to output that form of private variables so we will conclude our efforts in this direction.

@nabog: Thanks for noticing the issue. We honestly don't mind any sarcasm (I'm not being sarcastic here :) )

@AndrewGaspar: yes, that makes sense but I think we found a simpler solution that we feel will work better (described below)

We are publishing a JavaScript library and our original requirement was to ensure that people don't accidentally end up calling the private functions instead. So we decided to have a coding convention like this:
class Greeter {
    private __privMessage:string;
    private __fooMessage() {
        return this.__privMessage;
    }
    setPrivateMessage(m:string) {
        this.__privMessage = m
    }
    public greet2() {
        alert ("HOLA! " + this.fooMessage());
    }
}
And we will have a script to build an index of symbols starting with __ and replace all of them with minified forms, thus ensuring that none of our users can accidentally depend on our private functions in production code.
Jun 25, 2013 at 6:10 PM

I like that obfuscation approach, that's something the TypeScript compiler could do without compromising performance.

Jun 25, 2013 at 6:23 PM
Edited Jun 25, 2013 at 6:25 PM
I like it too. It's consistent with established convention. I would vote for this feature being in the release.

I've been using the _ convention manually in my code for the benefit of javascript consumers and would be happy to have it automatic.
Jun 25, 2013 at 6:49 PM
@markrendle, @Grajkowski:

When you say "I like that obfuscation approach", do you mean the approach we're taking right now? Our plan is to convert:
class Greeter {
    private greeting: string;
    constructor(greeting) {
        this.greeting = greeting;
    }
    private getFullGreeting ()  {
        return "Hello, " + this.greeting;
    }
    greet() {
        return this.getFullGreeting();
    }
}

var greeter = new Greeter("world");

var button = document.createElement('button');
button.innerText = "Say Hello";
button.onclick = function() {
    alert(greeter.greet());
}

document.body.appendChild(button);
Effectively to:
var Greeter = (function () {
    function Greeter(greeting) {
        this.$0 = greeting;
    }
    Greeter.prototype.$1 = function () {
        return "Hello, " + this.$0;
    };
    Greeter.prototype.greet = function () {
        return this.$1();
    };
    return Greeter;
})();

var greeter = new Greeter("world");

var button = document.createElement('button');
button.innerText = "Say Hello";
button.onclick = function () {
    alert(greeter.greet());
};

document.body.appendChild(button);

I am pretty confident we can modify the typescript compiler to do this with minimal intrusion.

@Jon, what do you think of this approach?
Jun 25, 2013 at 8:08 PM
skyronic wrote:
When you say "I like that obfuscation approach", do you mean the approach we're taking right now?
Oh I actually misunderstood it a bit. Sorry. What I thought you were doing was having private variables automatically have "__" added to the name in the generated code.

ie:
class Alpha {
    private bravo: string = "";
    private charlie(): void {}
}
becomes:
var Alpha = (function () {
    function Alpha() {
        this.__bravo = "";
    }
    Alpha.prototype.__charlie = function () {
    };
    return Alpha;
})();
Like I said this has the advantage of being consistent with established conventions, while keeping the JavaScript readable. Sometimes I can't take advantage of the source mapping in my debugging so keeping the output readable is important to me.
Jun 25, 2013 at 8:19 PM
Grajkowski wrote:
Like I said this has the advantage of being consistent with established conventions, while keeping the JavaScript readable. Sometimes I can't take advantage of the source mapping in my debugging so keeping the output readable is important to me.
This was my original idea as well. I did a little digging and found that the same suggestion was put forth in this thread 1 4th message. I think this makes a lot of sense for typescript as well.
Jun 25, 2013 at 11:15 PM
This:
var Greeter = (function () {
    function Greeter(greeting) {
        this.__0 = greeting;
    }
    Greeter.prototype.__1 = function () {
        return "Hello, " + this.__0;
    };
    Greeter.prototype.greet = function () {
        return this.__1();
    };
    return Greeter;
})();
Don't use $, it's already over-used in JS, and the double-underscore is already a convention for private. But switching to numbers is an excellent way of saying "don't touch". Much like the impossible class names that the C# compiler generates for anonymous types and other stuff.

If you want to get really creative, JS will allow a whole bunch of mad Unicode characters in variable names.
Mar 31 at 8:27 AM
Edited Mar 31 at 8:48 AM
I am quite dismayed that TS would bother to implement and "support" a keyword "private" into TS, but not actually support it at runtime, purely because of TS authors' opinions about JS runtime performance. These concerns should be communicated to TS users but nonetheless applied using JS conventions. This is the primary reason why I have not adopted TS. In the nodeJS and AMD ecosystems you have modules being created out the wazoo today in 2014 with true private variables encapsulated away, and there is no real semantic in TS that respects the notion of private variables, except only by using JS conventions directly. At that point while I appreciate the value of type checking at compile time, I don't see the tradeoff in using the class patterns enjoyed by TS when encapsulation is a farce; I'll stick with JS except only when handshaking DTOs.

Prepending public properties with "warning" prefixes, or obfuscating property names, is not encapsulation. It's noise, and makes APIs ugly. Cya.
Jun 24 at 4:51 PM
I agree with @stimpy77, I'm trying to convince my superiors to use TypeScript at work due to the classes and the type checking, but not allowing to have real private atrbutes and members as with plain Node.js code, I find it really annoying... :-(

@skyronic idea was great, problem was that TypeScript classes are created by just using a function as closure and returning the constructed class, if they would have used a plain function it could have allowed to use real private members.
Jun 24 at 5:56 PM
Edited Jun 24 at 5:57 PM
I don't get it, are there any languages at all that enforce access visibility at run time? C# allows you to access private members via reflection. In C++ you can just cast your object to a byte array and poke around all you want, or even better is lay out another object with the same exact memory layout, but all of the members public and cast to that.

A variable hidden by scope and not by enforcement from the compiler is not really a private variable. I would call it a "hidden" variable and it's odd that everybody here is trying to conflate the two.

Private members for most languages are a development time check, not a runtime enforcement. If you're using TypeScript, use TypeScript throughout your code base and restricted access to private members will be enforced for you. If you're not, prepend an _ to your private members and be done with it.

Again, here are your hidden members if you really need them:
class Foo {
    
    publicFunction: () => string;
    
    constructor(bar: string) {
        var myHiddenVariable = bar; // truly hidden variable
        
        this.publicFunction = () => myHiddenVariable;
    }
}

var f = new Foo("Zap");
alert(f.publicFunction());
In my opinion, if this is what you're trying to do, it's even better than TypeScript automatically making decisions for you because it's explicit that every instance of this class Foo will have a member publicFunction that closes over the scope of the constructor. If the compiler were to change all private members into "hidden" members, then most functions in your class would have to contain pointers to the scope of the constructor and walk the scope to access private members. It would also make inheritance much more difficult because you wouldn't be able to override functions that access private members and then call that base class's implementation from within because both functions would have the same name and want to be attached to the same object.
Jun 24 at 10:25 PM
AndrewGaspar wrote:
I don't get it, are there any languages at all that enforce access visibility at run time? C# allows you to access private members via reflection. In C++ you can just cast your object to a byte array and poke around all you want, or even better is lay out another object with the same exact memory layout, but all of the members public and cast to that.
This are really advanced features that should not be put easily in the hands of inexperienced users. I don't find the usage of closures to hide private elements a problem, but instead an advance knowing how to use them correctly.


AndrewGaspar wrote:
A variable hidden by scope and not by enforcement from the compiler is not really a private variable. I would call it a "hidden" variable and it's odd that everybody here is trying to conflate the two.
As I said, I find here scope as a tool to improve accesibility management of the vars. I think in fact it's their purpose.


AndrewGaspar wrote:
Private members for most languages are a development time check, not a runtime enforcement. If you're using TypeScript, use TypeScript throughout your code base and restricted access to private members will be enforced for you. If you're not, prepend an _ to your private members and be done with it.
Exactly, development, but TypeScript generated code would not be only used by TypeScript, but also by Javascript users (this is our main intention to start using it, to easier development of Javascript libraries), and their "development" stage as has been with Javascript for twenty years is in fact during runtime, and this is something can be changed due its scripting and dinamic design. Only solution is, then, enforce private members on runtime, and the best tool is using closures. Using an underscore will not prevent users to do wrong things if they thing are brighter than the developers, so that's why we try to make clean APIs and this involve having private members and attributes.


AndrewGaspar wrote:
Again, here are your hidden members if you really need them:
class Foo {
    
    publicFunction: () => string;
    
    constructor(bar: string) {
        var myHiddenVariable = bar; // truly hidden variable
        
        this.publicFunction = () => myHiddenVariable;
    }
}

var f = new Foo("Zap");
alert(f.publicFunction());
That fix the problem with private members, but not with private methods, and also force you to set the "priviledged" (functions and methods that can access private variables defined in the constructor, in Javascript idiom) methods by hand inside the constructor. @Skironic had mostly solved the problem, only remaining thing are:
  • codegen identify the private instance members and put them inside the constructor
  • check a dependency call tree to know what methods (public and private) need access to private members and currently priviledged private methods and promote them to be also priviledged, adding them to the constructor
  • set remaining private methods as regular functions outside the constructor and call them with .call() or .apply() so they can themselves call public methods of the object (if neccesary).
This is how we do it at my work, and has memory usage at a minimum, only point is that we need to do it by hand and sometimes we forget about cleaning priviledged methods, but it's something codegen would do automatically. Anybody knows where its fork of @Skyronic so I could continue its work?
In my opinion, if this is what you're trying to do, it's even better than TypeScript automatically making decisions for you because it's explicit that every instance of this class Foo will have a member publicFunction that closes over the scope of the constructor. If the compiler were to change all private members into "hidden" members, then most functions in your class would have to contain pointers to the scope of the constructor and walk the scope to access private members.
As how I've explained before, this can be easily feasable, and it's how we are already doing it.
It would also make inheritance much more difficult because you wouldn't be able to override functions that access private members and then call that base class's implementation from within because both functions would have the same name and want to be attached to the same object.
Ok, I agree with you. Some points:
  • private methods will not be afected, since they are already private on their scope, so nobody knows about them, only the methods on their class, not child ones (basic class inheritance rules)
  • public members and methods will work as usual by prototype inheritance, no problem here
  • public priviledged methods with the same name in the base and the child classes will be more tricky here, since both are build in the constructor. Since you'll need anyway to call the base class constructor, you'll have the base one already there previously to overwrite it with the one of the child, so you could take a reference on a variable to it previously to do the asination of the new method and call the funcion on that variable later from inside the child one or any other place.