My Friend’s Boss Has Little Relevance To Me

posted by eric

trustmail.jpgSort by trust in TrustMail.In this paper Jennifer Golbeck and James Hendler outlines a “reputation network analysis” system for email filtering. The concept is simple: rate everyone you know on a scale from one to ten, and your email application will calculate a “reputation score” for each and every incoming mail message (using a local trust metric algorithm and some FOAF-magic).

Wouldn’t it be great if you could sort your inbox by “relevance”? I thought so too, but then I took a look at my own inbox while trying to think of a rating for each mail that was there. And guess what–it turned out to be quite hard. In fact, in some cases I couldn’t come up with a rating at all, and in other cases I could think of several different ratings for the same person.

Why? Well, we know already that reputation demands context. So whenever I thought of a rating, I had to artificially place the person in a context, which felt somewhat awkward, arbitrary and at times plainly wrong. Which context to choose? What if there were multiple? What if there was none at all?

This is also the reason why the system probably wouldn’t work well anyway. Reputation scores that are inferred by the algorithm are calculated without taking context into account. So let’s say my friend’s boss sends me a mail. I trust my friend, my friend trusts his boss. And yet, the mail from my friend’s boss has little relevance to me. Both reputation scores are accurate in themselves, but when we collapse the contexts around them, they loose their meanings.

So, we need a way of saying “I trust you in this particular context”. But then we run in to issues of etiquette and fingerspitzgefühl. By saying explicitly “I trust you in the context of work” I’m tacitly saying “I don’t trust you outside of work”. Apart from the high cognitive load, I’m actually being rather tactless.

Needless to say, given that I couldn’t even trust my own ratings of my own friends, I’m a bit skeptical of the Trustmail approach. The same critique is valid for many other reputation systems. Clay Shirky puts it rather harsh but well: “Almost all the work being done on reputation systems today is either trivial or useless or both, because reputations aren’t linearizable, and they’re not portable. […] The world’s best reputation management system is right here, in the brain”.

August 21st, 2006