danah boyd has published a thoughtful and extensive rant about Facebook's slow-mo implosion of user trust, data privacy, and UI transparency:
A while back, I was talking with a teenage girl about her privacy settings and noticed that she had made lots of content available to friends-of-friends. I asked her if she made her content available to her mother. She responded with, "of course not!" I had noticed that she had listed her aunt as a friend of hers and so I surfed with her to her aunt's page and pointed out that her mother was a friend of her aunt, thus a friend-of-a-friend. She was horrified. It had never dawned on her that her mother might be included in that grouping.
Over and over again, I find that people's mental model of who can see what doesn't match up with reality. People think "everyone" includes everyone who searches for them on Facebook. They never imagine that "everyone" includes every third party sucking up data for goddess only knows what purpose. They think that if they lock down everything in the settings that they see, that they're completely locked down. They don't get that their friends lists, interests, likes, primary photo, affiliations, and other content is publicly accessible.
If Facebook wanted radical transparency, they could communicate to users every single person and entity who can see their content. They could notify then when the content is accessed by a partner. They could show them who all is included in "friends-of-friends" (or at least a number of people). They hide behind lists because people's abstractions allow them to share more. When people think "friends-of-friends" they don't think about all of the types of people that their friends might link to; they think of the people that their friends would bring to a dinner party if they were to host it. When they think of everyone, they think of individual people who might have an interest in them, not 3rd party services who want to monetize or redistribute their data. Users have no sense of how their data is being used and Facebook is not radically transparent about what that data is used for. Quite the opposite. Convolution works. It keeps the press out.
