Relative quality of Conservapedia vs. Wikipedia

From AJS.COM

Jump to: navigation, search

The Relative quality of Conservapedia vs. Wikipedia is a survey of two encyclopedia sites. The relative quality of encyclopedias can be measured in many ways, and over time this article will seek to apply a number of techniques.

Contents

Background

Conservapedia is a MediaWiki-based Web site that seeks to build an encyclopedia that does not contain the alleged "liberal bias"[1] of Wikipedia. The measure of their success over time will clearly be the relative quality control and lack of bias (conservative, liberal or of any other sort) that Conservapedia is able to maintain. In an effort measure that, this article applies a number of metrics.

What does the random survey measure?

The random survey is a measure of the extent to which the average article on a MediaWiki-based encyclopedia is connected to other information which can verify and validate its contents. It doesn't demonstrate that the information is valid, only that it can be validated. This is an important distinction, and it should be noted that most of Conservapedia's issues with Wikipedia relate to questions of bias, not verifiability. However, it is very difficult to generate an objective test for bias, so we use verifiability as a benchmark for quality in general and leave the question of bias to the readers. Overall, this survey has demonstrated that, although Conservapedia is lagging Wikipedia in terms of the average article, that gap has narrowed substantially as Conservapedia's content has become more substantial.

However, the following concerns should be kept in mind when using this article as a source for any reasoning about either site:

First, both sites are rapidly expanding. This means that many articles will be works in progress, and to some extent less complete average articles may be an indicator of the rate of progress.

Also, average article quality may not relate in obvious ways to the quality of articles which are within the "mainstream" of the site. That is, you might never see some of these articles unless you searched for them randomly.

Related to the above, a random search favors no article over another, which is not always fair for purposes of comparison between encyclopedias. For example, the Wikipedia George W. Bush article refers to the Third rail (metaphor) article when discussing the reluctance of the U.S. Congress to deal with Social Security reform. In a strictly random comparison, the short, and under-cited third rail article would have just as much impact on the measured quality of the encyclopedia as the comprehensively cited article about the forty-third President of the United States. This imbalance is difficult to adjust for without employing a complete site map for every survey, and even then it is questionable how much information could be gathered about relative article importance without subjective input.

As it stands, this survey conveys a useful view of the relative qualities, and especially the relative pace of quality change over time for the two sites, but should not be seen as comprehensive or definitive in any way.

A note on sectional citations

A new trend on Wikipedia seems to be the citation only of body sections, not the introductory section (which, presumably is drawn from cited statements in the body). For example, see the Kate Bush article on the day that it was the site's featured article. Since this seems to meet the criteria of the survey, I will be assigning the bonus +2 points to articles which cite all sub-sections, even if they do not cite the intro section. That said, the bonus +2 points will never be assigned to articles that have only one section. I can imagine cases where that would be unreasonable (e.g. a very substantial article that was well cited, but only a single section). This has never come up, but if it does, a revision to the rules may be required to adapt to the way both sites use citations.

Benjamin Franklin

Benjamin Franklin is an article on both Wikipedia and Conservapedia. Conservapedia calls out the article on their Examples of Bias in Wikipedia while on Wikipedia, the Benjamin Franklin article is rated "B-Class" on their assessment scale, so it seemed to make sense to compare the two articles.

2007-03-05

Conservapedia version:

  • The lead paragraph ends with a highly opinionated sentence: "Benjamin Franklin was truly a great American."
  • Starting at the second paragraph, and continuing on to the end of the article, the subject matter is Franklin's religious views.
  • His contributions, inventions and all other details of his life are relegated to a three-sentence introductory paragraph.
  • 4 references are cited.

Wikipedia version:

  • The lead paragraph covers his background as "author, politician, printer, scientist, philosopher, publisher, inventor, civic activist, and diplomat."
  • Sections 1.12 ("Virtue, religion and personal beliefs"), 1.13 ("Virtue"), and to a lesser extent, 1.14 ("Death and afterwards") touch on his religious beliefs, but do not take on the defensive tone of the Conservapedia. Instead, there is a chronological progression to Franklin's life and views that is correlated with the political views toward religion of the time.
  • Many areas of Franklin's life and legacy are covered, including his inventions, musical interests, political undertakings, etc.
  • There are 15 primary sources listed and 13 footnotes along with a collection of external links for further research.

What's more, the claim from the Examples of Bias in Wikipedia appears to be incorrect. It claims that "Wikipedia distorts the youthful acceptance of deism by Benjamin Franklin by never acknowledging that he later abandoned it." In contrast, Wikipedia states, "Walter Isaacson argues[2] that Franklin became uncomfortable with an unenhanced version of deism and comes up with his own conception of the Creator. Franklin outlined his concept of deity in 1728, in his 'Articles of Belief and Acts of Religion.'" There is also an extensive quote from Franklin that clarifies his views on Christianity in particular, and places them in a historical context with respect to the "Dissenters in England."

Overall, Conservapedia appears to throw many slings and arrows at Wikipedia, without first establishing its own baseline for quality. This comparison is only one of the many that anyone can perform on their own to confirm the general lack of quality standards held by the site.

2008 Presidential race

I'm going to be writing up a new section about both sites' coverage of the U.S. 2008 Presidential Race. However, for now have a look at these two versions of the John McCain article on Conservapedia:

Just one section, "electability," tells the tale:

"John McCain was a member of the Keating Five, a scandal relating to the Savings and Loan Crisis. The Senate Ethics Committee criticized him for "questionable conduct." His Democratic opponent, or the media, could publicize this."
"John McCain has cancer (melanoma) and has undergone multiple operations for it.[1] Cancer was an issue that hurt the presidential candidacy of Paul Tsongas in 1992."
"John McCain will turn 71 years old before the first primary, making him one of the oldest candidates to seek the presidency. Several issues relating to McCain's advanced aged and health have been discussed in the media.[2]"
"Among evangelicals, who form a multi-million vote bloc who were essential to electing President George W. Bush, John McCain is weak on trust. James Dobson, a leading evangelical, has already declared that he "would not vote for John McCain under any circumstances."[3] McCain has attempted to make amends with Dobson and the late evangelical leader Jerry Falwell, including speaking at Falwell's Liberty University in an attempt to patch his relations with evangelicals.
"McCain is the only presidential candidate in either party to have served in the military."

And from 2008 (current, now):

"He has been called "the Democrats' worst nightmare".[14]"
"Of the major contenders, only McCain has a background that includes military service. Colonel Bud Day, the United States' most highly decorated officer, and the most decorated since General Douglas MacArthur, said: "Having stood side-by-side with John McCain on the battlefield, I know that he has the character and will to lead this great country. John McCain is the Commander-In-Chief our military needs during this generational struggle against global terrorism.""
"McCain has the reputation as an independent-minded maverick[15] and favorite of the media.[16] David Limbaugh said, "McCain is not only not conservative enough; he has also built a reputation as a maverick by stabbing his party in the back -- not in furtherance of conservative principles but by betraying them. McCain delights in sticking it to his colleagues while winning accolades from the mainstream liberal media." [17]"
"John McCain was a member of the Keating Five, a scandal relating to the Savings and Loan Crisis. The Senate Ethics Committee criticized him for "questionable conduct.""
"John McCain has cancer (melanoma) and has undergone multiple operations for it.[18] Cancer was an issue that hurt the presidential candidacy of Paul Tsongas in 1992. However, it was not even mentioned in the 2004 presidential campaign, despite the fact that John Kerry was treated for prostate cancer in early 2003."
"John McCain is 71 years old, making him one of the oldest candidates to seek the presidency. Several issues relating to McCain's advanced aged and health have been discussed in the media.[19] "

Rather an interesting change in listed facts and weighting of same....

The Random Survey

2007-03-02: 1/54 vs 27/54

A survey using the following metrics was conducted based on use of the "random page" link in the sidebar of each site (a MediaWiki feature):

  • 1 point - Any external references (e.g. External links section or References section) to actual sources (e.g. textual information that isn't autobiographical)
  • 1 point - Specific references for any statements made
  • 2 points - Specific references for at least each body section
  • 1 point - Context links in intro (e.g. foo is a [[bar] [[baz]] of the [[biz]]) -- fail if context is not linked, but later terms are (e.g. foo is a bar baz of the biz that followed [[bam]])
  • 1 point - Categorizational context (e.g. Categories links at bottom)
  • -1 point - Tags indicating factual concerns (these were only found on Wikipedia)

Conservapedia

Wikipedia

2007-03-29: 4/48 vs 31/48

Another survey using the same metrics was conducted based on use of the "random page" link in the sidebar of each site (a MediaWiki feature):

  • 1 point - Any external references (e.g. External links section or References section) to actual sources (e.g. textual information that isn't autobiographical)
  • 1 point - Specific references for any statements made
  • 2 points - Specific references for at least each body section
  • 1 point - Context links in intro (e.g. foo is a [[bar] [[baz]] of the [[biz]]) -- fail if context is not linked, but later terms are (e.g. foo is a bar baz of the biz that followed [[bam]])
  • 1 point - Categorizational context (e.g. Categories links at bottom)
  • -1 point - Tags indicating factual concerns (these were only found on Wikipedia)

Conservapedia

Wikipedia

2007-05-02: 21/48 vs 24/48

Another survey using the same metrics was conducted based on use of the "random page" link in the sidebar of each site (a MediaWiki feature):

  • 1 point - Any external references (e.g. External links section or References section) to actual sources (e.g. textual information that isn't autobiographical)
  • 1 point - Specific references for any statements made
  • 2 points - Specific references for at least each body section
  • 1 point - Context links in intro (e.g. foo is a [[bar] [[baz]] of the [[biz]]) -- fail if context is not linked, but later terms are (e.g. foo is a bar baz of the biz that followed [[bam]])
  • 1 point - Categorizational context (e.g. Categories links at bottom, not including maintenance links such as "Stubs" or "Articles needing references")
  • -1 point - Tags indicating factual concerns

Conservapedia

Wikipedia

Commentary

By far, this is Wikipedia's worst and Conservapedia's best showing. It will be interesting to see if this trend continues in future surveys. 21 total points for Conservapedia vs. Wikipedia's 24 is certainly a narrow margin.

2007-06-04: 12/48 vs 22/48

The same survey was conducted based on use of the "random page" link in the sidebar of each site (a MediaWiki feature):

  • 1 point - Any external references (e.g. External links section or References section) to actual sources (e.g. textual information that isn't autobiographical)
  • 1 point - Specific references for any statements made
  • 2 points - Specific references for at least each body section
  • 1 point - Context links in intro (e.g. foo is a [[bar] [[baz]] of the [[biz]]) -- fail if context is not linked, but later terms are (e.g. foo is a bar baz of the biz that followed [[bam]])
  • 1 point - Categorizational context (e.g. Categories links at bottom, not including maintenance links such as "Stubs" or "Articles needing references")
  • -1 point - Tags indicating factual concerns

Conservapedia

Wikipedia

Commentary

This month, the margin is certainly wider, suggesting that last month's numbers may not have been as significant as the appeared. Wikipedia does make a fairly weak showing, however. Many of the articles barely met their criteria for the scores they got (California Proposition 14 is an excellent example), and compared to early surveys, these results are not strongly encouraging.

2007-07-17: 17/48 vs 20/48

This month's random article survey and comparison.

Conservapedia

Wikipedia

Commentary

Conservapedia scores 17 vs Wikipedia's 20, so it's close this month, but this appears to be a result of the ongoing dip in Wikipedia's score moreso than any improvement in Conservapedia.

Retrieved from "http://www.ajs.com/ajswiki/Relative_quality_of_Conservapedia_vs._Wikipedia"
Personal tools