Try counting the number of stars in the sky. Now try collating the number of opinions on how star ratings should be used by reviewers at the Fringe. The two tasks are quite similar. Never-ending.
At one end of the spectrum some argue that star ratings are so flawed, that they should not be used. That they are just one person’s view. That a single rating cannot reflect all elements of the performance. That they favour local Scottish acts, male acts, established comedians, first time comedians, white people, posh people, connected people, stand-up comedians…
At the other end of the spectrum, people argue that the current star ratings model is ‘the worst kind of ratings system, except for all the others’. That it is the most understood method of rating, and arguably the most wanted, both by readers and performers.
Others take the view that ratings should be used but that star ratings should not be given. So ‘5 star’ gets replaced by ‘must see’, ‘1 star’ by ‘a forgettable performance’.
Last year we at the EFC used the five star rating system, but with half stars to allow greater flexibility.
We will continue to use a five star rating system this year.
We believe the advantages of using the star rating system outweigh the disadvantages.
We believe the flaws of the star rating system are understood, and as such we can address some of those concerns.
- You are Scottish and will favour Scottish acts over mine! Absolutely not – our team is international, and we view each show on its own merits.
- I am gay and that will cost me a star at least! Absolutely not – our team is diverse, and to repeat the above, we view each show on its own merits. We also have editorial guidelines our reviewers have to abide by, and our comms include a footer in English and Gaelic to assure people who contact us that we have a diversity policy in place and treat people equally whatever their background.
- You just give out 5 star reviews to get your name on posters – Absolutely not! We have strong criteria and judge shows objectively against them. And in our experience, giving a 1 star review tends to get us more exposure than a 5 star!
- You only gave a 5 star review because you got a free ticket! – Absolutely not! The EFC has a policy of buying tickets and paying the same price as any other audience member and does not accept ‘reviewer/ free tickets’, or payment in any kind from a performer we review.
So how can we improve the star rating system?
It seemed to us that there are two areas of criticism of the star rating system that we could improve upon this year. Firstly, that the criteria for the ratings should be more detailed and transparent. Secondly, the criticism that star ratings are only a snapshot and cannot cover all aspects of the overall performance. On both points, we believe our new rating system below will provide depth of review and transparency as to how we arrived at any given rating.
What will change this year?
We will continue to use the five star rating system but will not give half stars. So ‘3 stars’ this year but not ‘3.5 stars’. This brings us into line with the majority of other Fringe review sites and is more readily understood by performers and readers.
Secondly, we will publish our assessment criteria below. This will keep our reviews honest and also give performers transparent guidelines.
The big change
This year we will break down our review into 5 areas – content, audience engagement, staging, originality and artistic performance. These criteria are designed to cover all types of show from musical theatre to absurdist comedy.
We did not select ‘value for money’ as a criteria given that our guidelines are designed to be universal; able to cover all types of show from free fringe productions and street performers, to higher-priced musicals and theatre productions.
The guidelines have also been written in a way that we can judge a type of show ‘for what it is‘. Therefore a street theatre company should be able to achieve five stars for staging as much as a well-known comedian performing at the McEwan Hall, providing of course they both meet the 5 star criteria.
The ratings across the five categories will then be added together and divided by 5. A rating of 4.5 will be rounded to 5 stars, a rating of 4.4 rounded to 4 stars.
Therefore, we are still rating a performance based on 5 stars, but also providing a breakdown on specific aspects of the performance, all of which contribute to the overall rating.
As an example, ABC productions score :
Content 4, Audience Engagement 3, Staging 5, Originality 2, Artistic Performance 3. Total = 17 (3.4) Star Rating 3
As such, it is clear from the detailed rating in the worked example above that the staging was fantastic, but that the show lacked originality. Something an overall single rating of ‘3’ might not be able to convey.
Our conclusion
We believe this ‘extended’ star ratings system addresses some of the identified flaws in the existing 5 star model and gives performers and readers transparency and greater confidence in the rating received. In our tests of re-rating shows from last year, 62% of shows kept their original rating with only 9% moving 2 ratings or more. There was also a 12% drop in shows given 5 star, and an 11% rise in 3 star rated shows, therefore no evidence of ‘rating inflation’ but equally more of a bell curve effect with fewer shows receiving a 1 or 5 star.
We will keep a close eye on the stats this summer and the performance of the new model, but are excited to able to use it for the first time this year and to be able to give performers and readers more detailed feedback
What do you think?
We know how hard people have worked on their shows in the weeks and months leading up to the Fringe and we want our review methodology to be robust, fair and detailed.
We want to know that you, the Fringe performers, have confidence in our reviews. We would therefore be really interested in hearing your thoughts on our new ratings format. If you have a view, please get in touch via our contact page!
Our Star Ratings Criteria in detail
Content
- 1 star! The show is confused as to its message. There is no coherent theme, genre or story to the performance. The show’s content lacks any depth or vision.
- The show’s content does have a message, but it is so well hidden that it rarely emerges. It is apparent what the performer the message is trying to put across, but for the majority of the show’s content, they failed to do so.
- The content delivers in an obvious ‘does what it said on the can’ way. The content does not surprise, and is not complex, but on the whole delivers on what is promised.
- The show’s content provides few surprises but is refreshing and has an entwined story throughout that builds to deliver a solid message at the end of the show.
- Five Stars! The show’s content is engaging, challenging, at times surprising, and multi-layered. The content causes the audience to reflect, long after the show has finished.
Audience Engagement
- 1 star! Complete audience disconnect. Multiple walk-outs and/ or audience confusion as to theme or nature of performance, audience members visibly bored, angry or disengaged. Several leave early. Total lack of connection between performer and audience.
- Minority of audience engaged to some level but majority look dis-interested. One or two audience members walk out or leave early, and as the curtain closes, majority leave without wanting to meet the performers. A forgettable performance that audience members will quickly erase from their memories. Not a show to recommend to others.
- A good audience connection during the show. A couple of the audience members clock check but in general the audience were engaged with the show’s content. The audience leaves the performance satisfied, but would not necessarily recommend the show to others afterwards.
- Great connection between audience and performer(s). The show’s content was well understood and it engaged members of the audience enough to interact with the performer after the show, or talk about the show’s content on social media or forums. The audience leaves the show thinking that the time watching the performance has been well-invested, and would recommend the show to others.
- Five Stars! The audience are engaged from beginning to end. Electrical connection between the performer and audience which leaves them spell-bound, and leaving the majority of the audience wanting to engage with the performer after the show. The content has motivated the audience to actively become fans of the performer. Previous audiences create a buzz about the show, and it recommended by the vast majority of those who see the show.
Staging
- 1 star! The staging is confused/ severely lacking and/ or amateurish. It appears that little thought has gone into staging or its execution. The overall experience of the staging severely impacts the audience’s appreciation and enjoyment of the performance.
- There is more than one issue with the staging that detract from the impact of the performance.
- The staging is good but does not add significantly in any way to the overall impact of the performance. There is an issue with one of lighting, use of space, set design, props and costume, stage business or sound that could have been improved.
- The set-design, lighting, music, sound and/or use of space are excellent and in some ways elevate the performance though not in an exceptional way.
- Five Stars! The staging is a masterpiece of stagecraft. The performance space is not only used to full effect, but in a way that significantly adds to the impact of the show. Where the show has technical support such as lighting, costume design or sound effects, these are all delivered flawlessly and add greatly to the enjoyment and impact of the performance.
Originality
- 1 star! The show not only lacks any originality, it comes across as hackneyed and a cliché of superior productions. A copy and paste at best of past productions.
- This show either represents a bad attempt to re-stage a previous production with minor tweeks to claim the show is ‘original’, or is a new show which lacks imagination and/or innovation.
- The show clearly states that it is a repeat performance from a previous year and for the majority part it is, but also includes new original sections that bring fresh perspective. For a new show, the performance has some original thinking, but for the most part it is offering standard fare and lacks ambition.
- For performers bringing past productions for a re-run; for the most part the show brings new ideas that keeps the performance innovative, current and relevant. For new productions, the show is innovative and gives fresh thinking to old themes.
- Five Stars! A totally original and innovative production that not only challenges but inspires the audience to think about the performance’s theme in entirely new ways. Revolutionary and will be remembered as a ground-breaking production!
Artistic Performance
- 1 star! The artistic performance is amateurish, ill-prepared, error-prone, and/or lacking in confidence which tarnishes the overall impact and enjoyment of the show for the audience.
- The performer(s) has made some effort, but overall it comes across as a lacklustre performance. The errors and technical faults are evident at times and have the potential to detract from the audience’s enjoyment of the show.
- Good performances from the majority of the cast of the show. On a few occasions there is a lack of confidence or a lack of technical ability but for the most part this does not impact the audience’s overall appreciation of the performance.
- Great casting and solid performances, especially those playing key roles or taking centre stage.
- Five Stars! Exceptional casting and performances from everyone on stage. Timings, empathy, delivery, artistic skill are to the highest level. Stand-out and faultless performances from all involved. Award-worthy!