IRC is awesome. However, channels can easily start getting drowned in noise as more members join. The following is a brief (personal) history of #xkcd and some of the unique solutions we've come up with to combat the problem of noise. Not all of them have been tested, and not all of them work, but there is very little online documentation to this, I'm sure, all-too-common problem.
My name is Glench. Two years ago, I decided to pop into #xkcd, an IRC channel dedicated to xkcd, the comic, and assorted bits of geekery. The first time I dropped by, I quickly thanked xkcd, a pretty standard entry message by people new to the channel, and spent the next few weeks finding my way around the channel, seeing what was acceptable, what kind of subjects usually came up for discussion, etc. I became a regular part of the community. When I joined, there were about 25-30 total members in the channel.
It was wonderful. The channel was a constant source of entertainment and intelligent discussion. I met quite a few cool people, as well, making some good friends along the way. I even got to meet a few of them in person and plan on meeting more. It is widely regarded by those who experienced it as the Golden Age of #xkcd.** And this is no simple nostalgia, either. These are very intelligent people comparing their current experience of the channel with their past experience. I can't say their judgement is objective, but it certainly has been noted a number of times from independent sources.
From these 25-30 members, the channel slowly grew. From what I remember, the channel stabilized around 50 members for a while, then started creeping up towards 100. Once the channel reached about 120 members, I'd say, the rate started to increase and it became increasingly difficult to keep track of new members.
Fast forward to the present, after two major jumps in users: one from the channel quote database being stumble[d]upon[d], and the other from xkcd's mentioning signal in his blog post (linked above)** I glossed over the part where I incidentally attracted a ton of new members to the channel. This quote made its way to stumbleupon, where many people saw the channel name, and, already being fans of the comic, joined. By and large, this influx destroyed #xkcd's signal-to-noise ratio. Whoops.. As of this moment, there are 318 nicks in #xkcd. I, and many of the other old channel regulars are ops, and the channel, frankly, sucks. It has its moments, but due to the large number of people, anything intelligent usually gets drowned in noise.
So the question now becomes: how do we fix our signal-to-noise ratio, or, if this is not possible, extract as much value from the channel as possible.
xkcd, the channel founder, wrote up a short summary of the problem and some of its solutions here. In this document, I will cover these items and expand upon them with the different techniques we've thought of and/or implemented, and how well they worked.
xkcd's solution is to use a robot to weed out the noise of a channel. It is a great idea, and has worked well according to those who spend a lot of time in #xkcd-signal, the channel the bot resides in. However, instead of being #xkcd with all original content, it is now its own community, distinct from #xkcd** #xkcd-signal's user count is up to 218, most likely beyond the general "suck threshold" for channels. zigdon, one of the community leaders of signal, says that it has managed to become a good channel despite its many users. It would be interesting to see how #xkcd would fare under robot9000's cruel reign.. The problem with this bot is threefold. First, the assumption is that repeated phrases are necessarily noise. Maybe this is true in the technical sense, but not in practice. Second, unsaid things are not necessarily signal. Third, this was an experimental technique only, more of a "just to see what would happen" kind of thing. Because of these things, robot9000 just isn't the solution to #xkcd's noise problem.
So what makes a channel enjoyable? xkcd suggested that "A common factor in my favorite hangouts seemed to be a focus on original and unpredictable content on each line." So we need to have this unpredictable and original content, but also to make sure that this type of behavior is reinforced in some way. Here are some of the ways we've thought of to do this.
Strict entry requirements: This is the secret club/sorority approach. You can vet every new person before they're allowed to speak. This sucks. It reminds me of Feynman's comment on resigning the National Academy of Sciences — he said that he saw no point in belonging to an organization that spent most of its time deciding who to let in. The problems are apparent during sorority rush week on college campuses. Not only is the question of who does the vetting (and how) difficult, but the drama reaches horrifying levels as bitter counter-cliques rise up and do battle.xkcd
For the most part, xkcd addressed everything wrong with this strategy. having some sort of invite-only. It is not only too much work, but it's also highly arbitrary. Many new members who might be desireable to have in the community will become discouraged at the requirements to get in, as well, and the effect will be an overall reduction in cool people. Letting only certain people into a channel just doesn't work.
As there are too many problems with this exact method of invitation only, we thought of a new method. In this model, the channel will be muted (+m) for x hours and unmuted for another x hours, cycling in between the two modes. The unmuted time will allow everyone, especially new members, to talk and prove themselves as high sources of signal. They could then be given a voice (+v) so they could talk all the time, thereby increasing the overall quality of the channel.
Unfortunately, this method doesn't address the problem of arbitration or the discouragement of new potential members. However, it does give a good chance for everyone to prove their worth to the channel, and with tweaked timing (biasing more muted time or less), a balance could be found. Due to practical limitations on who goes about deciding who is cool or not and the ire that would undoubtedly rise as a result of this method, it was never tested, and probably never will be. There is an strong possibility, as xkcd notes, that "it would quickly destroy any communitiy it was tried in." Most likely this method would only be tried as a last resort.
Running peer-moderation: When it's possible, this is a good approach. It's used to great effect on comment threads, with Slashdot pioneering the whole thing and sites like reddit stripping it down to an effective core. But it doesn't work very well for live time-dependent things like IRC channels.xkcd
For instant and relatively anonymous communications, this approach doesn't work well. As egfdwkbhxk linked in the comments, Utu is trying to eradicate the problem of noise by forcing everyone to be tied to a central identity, keeping track of everyone's reputation, and having retribution for unacceptable behavior.
While there is no peer-moderation for specific comments that have been said on IRC, there are ways of tracking a user's reputation called karma. The #xkcd infobot called Bucket used to keep track of every user's karma, which could be modded by any user with the syntax [username]++ or [username]--. In theory, an accumulation of bad karma could lead to punishment, such as Robot9000's method of taking away voice in a moderated channel.
One problem with this is that the karma system was tied to a user's nick, which can be changed at any time. To be thorough, the user's hostmask could be used, which would largely solve multiple identities.
Another problem is that when Bucket's karma function was turned on, there would be large "karma wars" where one user would downmod another user, and that user would, in turn, downmod the user that downmodded him, resulting in useless noise. This could be solved by having the bot only take karma by private message and only so many votes in a certain period of time.
Inevitably, though, the karma ranking would become more important than the actual content of the channel, with people afraid to take risks in fear of their karma being affected. Currently, I can think of no solution to this problem, making this method a possible, but unlikely practical way of dealing with noise.
Finally, a recently proposed method was to just have the known producers of signal talk more. As detailed in the next section, many people now split their time between #xkcd and many side channels due to the overwhelming amount of people in #xkcd. The theory is that having these people present in the channel often will give good examples of the type of content we wish to promote, as well as discourage noise by natural selection. People will follow the more interesting conversations and let the less interesting ones die out, and those users who can't produce high signal will be discouraged from talking.
It is unclear whether the above technique will actually do the things it says it will, especially with the large amount of people and new members joining #xkcd every day, but it would be interesting to try.
Splinter communities: This has happened on most IRC channels I’ve been on -- small invite-only side channels sprout up with particular focuses. Often, the older core members of the community go off to create their own high-signal channel, which is generally kept quiet. But this is limited -- it lacks the open mixing of the internet that often makes online communities work.xkcd
The process described above seems to be the natural progression for large channels, which would suggest that, while it may not be the best way, it is a desireable and feasible way to find signal. The only downside, as xkcd notes, is that new and varied members won't be able to interact. These interactions can often lead to some of the best moments online, but there is no way to stop members from leaving. Indeed, it is hard to replace those members that used to provide interesting content. While this is the natural state of affairs, it is not desireable because it does not help #xkcd's signal-to-noise ratio.
I know from personal experience that this has happened in #xkcd. I myself spend quite a bit of time in splinter channels, and I know other people who do the same. I do not know of any solution to this, if this is seen as a problem.
Moderators: This is the approach IRC channels and forums usually take. You designate a few ‘good’ people who can deal with noise as it happens, by muting, kicking, banning, or editing content as need be. There are a couple problems here -- the circle of moderators has to grow with the community. It eventually becomes fairly large, with complicated dynamics of its own, and the process of choosing moderators leads to sorority/NSF-esque drama and general obnoxiousness. I don’t like the elitism that inevitably develops, and prefer more egalitarian systems.xkcd
I left this one for last because I and many other people are ops, so of course we have the most input in how this one works out.
For the current #xkcd, there are around 15 ops, some because of annoying spammers, some because xkcd was drunk, and some to deal with the recent noise. We have kicked, muted, and, though not much, banned offenders. It is a testament to the kind of community that xkcd attracts, however, that the populace is generally understanding of these things.
So far, we haven't had too much of a problem with the sorority mentality xkcd mentioned, but it has been asked of me a few times to grant op or hop status, and I'm sure other ops and xkcd have gotten similar requests. Generally, it does not come up.
One of our first united actions as ops was to create a set of channel rules to curb the growing trend of noisy behavior. Before, we had no set rules, so users could legitimately claim ignorance. Now these is no excuse. Our main goal besides trying to emphasize signal was to be as reasonable as possible** I am probably the worst offender of this. I do a lot of joke kicks and mute the channel if I think it's getting too noisy. Sometimes this breeds resentment, but I haven't had too many overt complaints. I have, however, made mistakes, such as muting the channel for too long, and for this, I apologize.. I think all of us have had the experience of being unfairly picked on by an op of another channel, and we know how much that sucks. I personally think we're pretty good at not being elitist, but then again our op status invariably sets us apart.
Another interesting approach we've tried is called Story Time. For this, one user who has a story to tell gets voice, and the channel is muted to allow him or her to tell their story uninterrupted. Story Time pauses the usually overwhelming flow of conversation for a minute or two so that we can all hear a potentially cool story and generate discussion. We've found that when used in moderation Story Time is pretty successful.
There is still a noise problem in #xkcd. I'm not sure anything will ever be able to eradicate it completely with such a big channel, but it is worth experiementing with different techniques nonetheless. Some of the methods we've thought of have helped a little, so we thought it would be worth sharing what we've learned so far. This is, of course, still an ongoing process, and we're still learning. As such, we'd appreciate any feedback.
You can discuss this on #xkcd on foonetic.