Several other websites that provide links to movie reviews have weighed the quantity vs. quality issue and come out in favor of quantity. These sites typically include links to as many reviews as there are available on the net. And lately, with every Joe Schmo posting a movie review both before and after movie releases, there are quite a few reviews for each movie (we're talking 100's of reviews for the more popular titles). True, some of these Joe Schmos do have quality sites with useful reviews and information. But the quality of many is inconsistent at best. In addition, there is such a thing as too much information, and statistically, once we include a certain number of reviews in our calculations, adding additional reviews will not change the overall METASCORE much in one direction or another.
Thus we felt that selecting a modest number of higher-quality publications would make the pages of the site easier to read and would maximize the quality of our information.
Choosing which publications and critics to track in each genre is a trickier proposition, and it involves a careful research and study. For games and music, we work to identify publications that (1) are well-regarded in the industry and are known for quality reviews; (2) actually seem to produce quality reviews (or, if not, are so influential in the industry that they have to be included); and (3) have published a good quantity of reviews. Although we typically limited our search to domestic publications, we do include a good number of UK publications in our music calculations, since many of the artists reviewed are from Britain. We also include reviews from top publications from around the world in our games section.
We took these same characteristics into effect when selecting the publications to include in our movies and TV calculations. For movies, however, we also made a point of selecting publications from every region of the country, to ensure that local tastes would be taken into account. True, a very small number of movie publications included in our site probably might not have made the cut if it weren't for their geographic location; however, most of the publications included are of a high quality, and those that are not are weighted accordingly in the formula.