7
$\begingroup$

Today, the site turned 30 days old. Congratulations!

Okay, enough celebrating. 30 days means that the Area 51 stats for the site are visible, and the results are in (in the form of our number/good number):

  • Questions per day: 2.4/15 (Needs work)
  • Percent answered: 86%/90% (Okay)
  • Users: On pace at 90 days for: 72/150 users with 200+ rep, 3/10 users with 2000+ rep, 0/5 users with 3000+ rep (Needs work)
  • Answer ratio: 1.8/2.5 (Needs work)
  • Visits/day: 98/1,500 (Needs work)

As you can see, we have a lot of areas to improve in (Although the user stats are, as you can check for yourself, inaccurate. Multiply the rep of everyone here by 3 (disregarding any association bonuses) and we're on track for 10 2000+ rep users and 7 3000+ rep users, and I'm assuming we'll come close to the 150 200+ rep users, too).

Think about how we can improve the site, both as a community and individually. Let's set a goal to hit the "Excellent" bar 60 days from now!

|
$\endgroup$
  • $\begingroup$ Where do "good numbers" come from? $\endgroup$ – Alexandre Eremenko Dec 2 '14 at 4:03
  • 1
    $\begingroup$ @AlexandreEremenko That's my poor phrasing, but it's the recommended number for a top-tier site, although as Robert Cartano said, that's pretty unreasonable. $\endgroup$ – HDE 226868 Dec 2 '14 at 23:08
  • $\begingroup$ Recommended by whom? $\endgroup$ – Alexandre Eremenko Dec 3 '14 at 15:28
  • 1
    $\begingroup$ Recommended by the Area 51 overlords. . . $\endgroup$ – HDE 226868 Dec 3 '14 at 16:05
13
$\begingroup$

I think it's worth putting these statistics in perspective now, to avoid setting unrealistic or inefficient goals. These statistics are very easy to misinterpret. I've been tracking a few sites over the past 2 years or so. After looking at these stats almost every day, I've realized that they're pretty limited in what they actually tell you about a site. They are useful over long-term periods for tracking community growth. They are almost completely useless as an objective measure of the health of this site. Community manager Tim Post has written several meta posts ([1] [2]) about the usefulness (or lack thereof) of the statistics, and Robert Cartaino wrote a blog post about it; my post here is similar in viewpoint though maybe not in the details. Sorry for the somewhat long answer, but you can skip to the last paragraph if you just want the summary.

The biggest flaw of the statistics is that they say nothing about the average quality of the posts on the site (which is hard to measure). If a very advanced question comes in, it will count just as much towards our questions per day as a question you could answer after a couple minutes of searching. If it goes unanswered, that hurts our % answered stat, and if it does manage to get a single sufficient answer it still counts against our answer ratio. Trying to optimize the stats too much leads to posting the wrong kinds of content, lowering the quality of the site. The statistics also can't tell things like how many users are still active, or how consistently and clearly the policies are enforced on the site. In addition, different topics have different levels to aspire to; for example, language sites are usually very near to 100% answered, but among the science sites our 88% is quite good.

It's worth looking at the stats for other beta sites at http://codemanteau.com/stacker/, which has a bit more than a year's worth of data, and at stats of graduated sites at the time of their graduation on Area 51. Looking at some sites there will give more reasonable expectations than just looking at our page alone. The levels at Area 51 were just the team's first guesses at what good stats would look like, but unfortunately, even with more data they haven't made any major changes since. Most beta sites aren't "Excellent" in all categories even a year into beta, and many graduate with multiple "Okay" stats (and a few even with "Needs work" in traffic or questions per day).

Individually, the categories are:

Questions per day: This is actually a very misleading statistic. With our community the size it is right now, 15 questions per day would just mean a lot of them go unanswered. Right now, we can probably handle somewhere in the range of 5-10 questions per day, and that would be enough to keep relatively active users. But 15 questions per day would put us in the top 40 sites or so network-wide, and we don't have the user base to handle that either from an answering or a moderation standpoint. I'd also be worried that if we're getting 15 questions per day, they may not be very good. If we can keep quality high and aren't getting overwhelmed, more incoming questions is a good thing, but we're still at the stage where our primary focus should be elsewhere. If it drops below 1-2 questions per day, we may need to work on countermeasures, but if we're at the same level at 90 days that we are now, that's not something to worry about.

Percent Answered: More answered questions is certainly a good thing, all other things being equal. But it's easy to take that too far, and ask a bunch of easy questions, with the intent of boosting this statistic, or to assume that this statistic means that questions that are easily answered are preferable. Some of the reason for this statistic being low is because hard questions almost always end up with lower answer rates, and that's not a bad thing. There are also problems associated with a lack of unanswered questions (for one, it makes it harder for new users to find ways they can contribute). With all that said, this is typically one of the better stats to look at.

Users: This is the only statistic which is mostly without issues, and it's also the only statistic SE views as "required" for graduation (specifically, 10 2k and 5 3k users is considered a minimum requirement to graduate, as well as 90 days in beta). Since the calculation is unusual and not documented anywhere that I know, I'll just note that in the first 90 days, for each of the three rep levels, the number it displays is calculated as $[N (2-\frac d{90})]$ where $N$ is the true number of users at that level, $d$ is the number of days we've been in beta, and $[\cdot]$ denotes rounding to the nearest integer. Starting at 90 days, our actual numbers will be shown.

Answer Ratio: Technical sites tend to have lower answer ratios, while sites which encourage subjective answers tend to be higher. The highest are sites like Code Golf and Worldbuilding, while Chemistry and Biology are both in the bottom 10 network-wide, even though both are doing quite well. The only science sites above 2.5 are Philosophy and Mathematics Educators. History is below 2.5 as well. And note that according to this stat, a question which gets a single, great answer is still considered bad. If our focus is on objectively answerable questions, we'll naturally be somewhat lower on this stat than more subjective sites. So long as we aren't leaving a bunch of questions incompletely answered, it's reasonable to mostly ignore this stat.

Traffic: Traffic is a lot like the % Answered. There's nothing bad about getting more traffic, all other things equal. But it's actually not that hard to get a lot of traffic if you don't care about anything else. Just posting basic questions for very common searches (that are already answered elsewhere in dozens of places) is one way to get lots of traffic. Another is aiming for the network hot questions list with easy, subjective questions that will generate lots of answers. If that's the kind of community we're aiming for, which covers famous stories at the same level as popular science presentations, then doing this is fine, but I'm hoping this site will be somewhat more advanced than that, and lowering the level to boost traffic only hurts us in the end. Watching traffic numbers is certainly fine, but if we keep the quality up they'll naturally grow to the "Excellent" level eventually, and we shouldn't deliberately try to speed this up by other methods.

It's actually not incredibly hard to get all these stats to be Excellent if that's the only thing you care about. Puzzling seems to have inadvertently done just that. But they are having major issues as a community, in part because of the style of Q&A they've moved to. I'd rather we not fall in the same pitfalls here.

Anyway, the short version is, the stats are fine to look at once in a while, and they are useful for some things, but they are far from the final story of the quality of this site. They aren't a report card for this site or anything like that. The statistics are only useful insofar as we aren't trying to deliberately increase them. Our goals shouldn't be to improve the stats, but to improve this community. The stats will follow after that.

|
$\endgroup$
  • 4
    $\begingroup$ I know thanks are not appropriate comments, but I learned much from this. Thanks for this research and perspective. I am enlightened. $\endgroup$ – J. W. Perry Nov 30 '14 at 8:01
  • 3
    $\begingroup$ I don't normally accept answers to meta questions, but this is thorough. . . Wow, nice job. $\endgroup$ – HDE 226868 Dec 2 '14 at 23:10

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .