The System of Reputation

posted by alex

Later today we are meeting up with RapLeaf CEO Auren Hoffman so here are some current pre-talk thoughts. When talking about reputation systems I think it is important to keep two things that aren’t mentioned that often in mind:

1) Input is unevenly distributed
2) Reputation need not be explicit

According to this talk there has been much research concerned with the functionality of reputation systems such as the ones on eBay or Amazon. Many studies show that the users inclinations to provide feedback is highly asymmetrical depending on if their experience is positive or negative. Users will give positive feedback on a deal gone right but will avoid giving negative feedback on deals gone bad. Why? Fear of retaliation.

Online reputation has become so important to people that the fear of upsetting your personal reputation stock value is simply higher than the drive to perform good for the community. Say that you just scammed me on Amazon and I just lost some money. The question is what I now have to gain by giving negative feedback about you? I have already lost my money and by giving you negative feedback, by means of reciprocation I am also gaining risk of your negative feedback retaliation and thus a decrease of my personal reputation score.

Conversely, if you actually didn’t scam me and the transaction went well, I might be inclined to give you a higher rating than necessary since you might reciprocate this by in return giving me a higher score (i.e. increasing my personal reputation stock value). So, we have a situation where we lack negative feedback and incentivize the polarization of positive feedback which naturally begs the question: Are reputation systems really useful at all?

Of course they are, but only to a certain extent. Moreover, one needs to take these system distortions into account when assessing the reputation value of people. On Amazon for instance, it seems to me that nobody that has a rating has one that is lower than say 90%. In other words, the 0-89 of the 100 unit scale are never used. In this scenario 91 is a terrible rating while 95 is OK and 99 is pretty good. Furthermore the rating can be seen as a proof of previous transactions that in itself provides a certain limited amount of trustworthiness.

Alex99reputationThe second issue I’d like to address is that in offline-life reputation is rarely explicated in the form of a numerical score. Yet, since the beginning of the net we’ve always had ways of hinting about a persons reputation anyway. Like I mentioned here, online, just like offline, we have certain social cues for reading people and assessing their character. In a forum setting most people gain reputation by the amount of posts they have made, how long they have been members, etc. In the days of social networking we can use connections to others to establish if a person is trustworthy or not. These cues aren’t explicit reputation cues or measurements but we still use them in this way–just like in real life. A numerical specific reputation value can be a quick and easy way to judge somebody but also, for the same reason, a highly motivating attribute to try to “game”.

Reputation will always be present, whether we have an explicit system for it or not. The question is–do we want it to be explicit and numerical? In the Amazon system, the inherent problems become a way of obscuring the explicitness of the system and translating it into an implicit cue about reputation. In other words, we are putting numbers on stuff to explain tricky phenomena and at the same time figuring out new ways of interpreting what these numbers really mean.