{ on programming and the internets }


by Louis Brandy

Algorithms in Real Life: Gradient Descent in Social Media (3/3)

While the last two posts were about “usual” gradient descent, this one is about a concept that is very close by analogy. It’s not quite the same but I think the grouping is natural. In this case, the guy in charge is much more in control of the terrain than the direction of travel. This is about social online communities. For the purposes of our discussion, I’m refering specifically to any product that relies on the community for its content (usually with both community contribution, and community voting as a filter). In our analogy, the point we are trying to optimize is the community that develops around our product.

When your product depends on the crowd for content, you are quite literally on a slippery slope. Since the crowd creates the content and determines what is popular, the things that become popular also attract the type of people who like that content. There is positive feedback in this loop. If we think in terms of a gradient descent, the situation is changed. You don’t get to choose which way is downhill because the crowd pushes you there. Your site becomes like a ball on a hill. It’s going to go where the hill takes it. When a social site starts out, and is in its formative stage, the community is sort of like a ball on the top of a hill. With a tiny push in any particular direction, you can drastically alter the eventual outcome. In this case, the outcome is the particular community that develops. Small descisions early on can have drastic consequences. Once a community has its particular set of groupthinks, it’s extremely difficult to push that ball out of that valley and into some (hopefully) better valley.

From a founder’s perspective, the community that develops is of paramount importance. Good communities can dominate some niche, be extremely large, and be an extremely attractive crowd for advertisers. For one reason or another, though, founders need to think about how to best guide a ball to wherever it is they want to be.

The initial conditions help to determine that initial community. And from there, everything can follow.

Positive Reinforcement

It is important when trying to build a community based site to understand this principle. One particular community can all but remove the “opposing” community from existence. People like to go where other people agree with them. And there are certain issues that cause people to become extremely tribalistic and do their best to make opposing views as unwelcome as possible. These issues very quickly “break symmetry” and become dominated by a single viewpoint. There are obviously other reasons why certain viewpoints began with a large majority, but the fact that the positive reinforcements very quickly drives that ball downhill needs to be understood. If it is your business it might be in your interest to steer that ball away from certain valleys.

For example, Hacker News has done this by using moderation to simply declare a site-wide topic and stick with it. By and large, the community that has developed is there to read about that topic. They steered that ball early and attracted the type of community they wanted. Consequently, it has become largely self policing, now. Even still, the moderators have to step in occasionally.

reddit

There is another way to solve the same problem. A really good example of this comes from reddit.com. Many people who’ve been around reddit for any length of time remember how frequently and how pervasive the 9/11 conspiracy theories were on the site. They were common in the political reddits and often made their way on to the front page. Once a ball like that gets rolling, it becomes extremely difficult to stop. People who want to read conspiracy theory stuff can go there and look at it, and those that don’t will stop going. The more momentum the movement has, the more difficult it becomes to stop. (I think) reddit was in real danger of falling off that conspiracy cliff and becoming a full-fledged conspiracy clearing house. The positive reinforcement can be extremely strong, in this type of situation. It turned out, however, not to go down that path.

What saved reddit? (Has it even been saved?) Well, there were certainly external factors (like the election, etc.) that made the topic diminish in importance in the political reddits. I think there’s more to the story, though.

Reddit is pretty well-separated and easily customizable. If you only go to programming.reddit, you won’t see much politics. Same thing with lolcats. Or bacon (’nuff said). Many of these well-separated communities thrived. So while the front-page of reddit (and the political sections of reddit) had high conspiracy content, these other communities were largely left alone. Reddit also made it extremely simple to customize (many people are presumably like me and use reddit with all the political stuff turned off). This lets reddit build multiple overlapping communities.

In the end, these sub-communities grew faster than the conspiracy theorists and eventually drowned it off the front page. Over time, less and less 9/11 conspiracy theories have appeared on the front page of reddit (or even politics.reddit). To this day, though, because of this early movement, reddit is one of the most conspiracy-friendly places on the “main” internet. People who go to reddit are at least tolerant of conspiracy theories because those that aren’t haven’t survived this long.

digg vs reddit

Another really interesting application of this kind of thought is how much the layout of a site effects the eventual people who use it. digg is much more visual than reddit. Early on, when the only social site was digg, it was an extremely technical site. This has changed alot as its popularity grew. Since digg is so visual, it lends itself much more to pictures and videos. Consequently, digg has become much more open to general internet hilarity. LOLcats and funny videos are par for the course on digg. As this popularity grew, it became difficult to stop, until the highly technical articles could no longer compete with funny pictures of cats for front page space. The highly technical things moved to other places. Like programming.reddit. Since that community moved on, highly technical articles are almost never seen on the digg front page anymore.

The second half of this argument centers on why programming reddit become one of the best places for technical information (and give digg users a place to flee towards). Well, reddit, unlike digg, has (or at least had) a lot of information crammed into a small space. There was no room for pretty graphics. One of my personal rules of thumb is that junkies love information density (the quintessential example is the Drudge Report). Reddit is one of the most dense sites in terms of information. Given the type of people who enjoy high density, and the digg refugees, made programming.reddit ripe to become the hub of highly technical information.

As these sites travel downhill, they leave voids that others fill on their way. Over time, they each settle into niches. Any time any of these sites are “on the move”, pretty much everyone who was there “before” sees it as the site’s “death”. This is another interesting phenomenon in its own right. I’ve often wondered where these sites are headed as they become more popular. Are they converging? To what? I think I’ve figured out the answer to this question. As a site becomes more and more popular, its user base becomes indistinguishable from 4chan.

trackback

2 Responses to “Algorithms in Real Life: Gradient Descent in Social Media (3/3)”

  1. December 8th, 2008 at 10:59 am

    Aaron says:

    so where is 4chan headed?

  2. December 9th, 2008 at 11:45 am

    Chalks says:

    Hell.

Leave a Reply


new server, if you see badness, please email me!

Need a new job?