Does a team's reputation help them place?

[Rahul3]

Say a team has a great season and then returns to a competition the following year with a new routine. Knowing that in the past season they had great performances and have the reputation of a "power house" team, do you think it would cause you to like their new routine more than if a brand new team did the exact same performance?


I'm not directing this towards any team. I was just thinking about whether the people on BTF who judge have ever accidentally let this sway their opinion on a performance or if they have thought about it. I know that those judges (saleem, govind, etc.) know so much about the North American bhangra scene, but is it more fair to let someone judge who has no idea of a team's past performances or reputation?

 

[Saleem]

A team's performance at the event is all that matters and is what is judged. I think the only element that comes into play from previous performances is if we've seen the same performance again and again and it's stale. A team's reputation or any subjective slant like that is not allowed.


As a side note, there are a few reasons why the "playing from home" people judging off videos will disagree:


1. Videos inherently show a different performance. I have watched performances online that I have seen in person and video is a leveler: teams with sub-par energy will look ok, teams with amazing grace will get lost on video, etc. Video will NOT give you the same experience.


2. There may be aspects you dont know about. Late registration? -10. Over time? -10.


3. Performances are judged based on the rubric. We as judging panels do everything we can so that we think the rubric scoring accurately reflects who we think is 1st 2nd and 3rd but sometimes teams will come with an "omg so good" performance that is bleeding points on the traditionality or synchronization parts of the rubric.

 

[yraparla]

it goes both ways. SGPD has not placed/placed lower in the past when compared to previous routines ("you were better last year")

at the same time, I think the mystique of some teams have helped them placed when expectations are set. Imagine that you're known to be not a traditional team and you bring out some traditional flavor...the impact is much more and the threshold for what constitutes a 'good traditional segment' is lower compared to a new team. Conversely if you are known to be super clean and you take a risk, etc etc. Subjective judging inherently enters the human factor of expectations so yes a teams reputation affects the scoring

 

[Saleem]

Swi I can't argue with human nature but the job of a judge is to try to counteract known psychological cognitive 'biases' to be objective. An example is to not forget the teams who went 1st and 2nd. I will compare scoring sheets as the show progresses to make sure I am being consistent and equitable.

 

[sapg21]

1. Videos inherently show a different performance. I have watched performances online that I have seen in person and video is a leveler: teams with sub-par energy will look ok, teams with amazing grace will get lost on video, etc. Video will NOT give you the same experience.

completely agree here...i think people who watch videos and judge outcomes of competitions (i.e. HOLY CRAP HOW DID THIS TEAM NOT PLACE?!) should be wary of the "wideo effect"

 

[Stylez1]

+100

it goes both ways. SGPD has not placed/placed lower in the past when compared to previous routines ("you were better last year")

 

[kinnell]

Being a judge is extremely difficult. There are many subconscious things that judges will battle in efforts to be as objective as possible.

My definition of the judge:
A judge is someone who has experience to recognize aspects/components of a dance routine and then objectively allocate points to a given grading rubric to evaluate the performance presented in front of them live without taking into account personal biases. A judge is not someone who writes comments to teams on how to improve. In fact, unless the judge writes suggestions/comments after the end of the performance, the latter will conflict with the former as your thoughts/eyes may be diverted.

Saleem is absolutely right that the judge should only take into account the performance seen. However, unless it is on the rubric, a judge cannot/should not reduct points from a performance because they have seen it before at other competitions or youtube videos. One can argue that unless a competition specifies to the judges that they can deduct points for creativity (ie: creativity is in their rubric) if teams do not bring in newer/different concepts to their competition, teams would be able to get away with bringing the same routine to the same competition year after year.

As Swi pointed out with examples, there are negative and positive consequences of a judge taking into account previous routines. But honestly, I think the greatest threat to a judge's effort to be objective as possible is them knowing that an entire team will storm the judging room following the competition if they do not place/win/get-best-dressed and threaten their life.

Does it happen? Yeah. But probably not as often as you may think.

 

[MPElive]

Several times, I have seen "Powerhouse Teams" not winning/placing at a competition. Here are few instances: Surrey India Arts (SBB 4 - No placing), PPA (SBB4 - No placing), AVA Boys (Boston 2007 - No Placing), Kohinoor Folk Arts (Masti 3 - No Placing).

We, as judges have always followed the Rubric the committee has provided us. All I know from my experience is that Most Folk/Traditional Team of the night does not always win the competition. But if they followed a rubric, they might place or even win it all. Personally, I try not to watch videos if I know I am judging a team at an upcoming competition.

 

[yraparla]

So tangentially to the topic, something has been bugging me for a while now:

Why do many comps drop the highest and lowest score?! The initial mindset was to offset biases for/against teams...but this normalizes scores (leading to drama as the score variance is arbitrarily reduced) and it diminishes judges who judge the teams relatively (i.e. what I dub the faizan effect). If identical judges have identical scores for teams but judge B is just offset by 20 pts, he runs the risk of being discounted even though his point differential is the same (he just has a different base level for what an average performance should be allotted in points).

I personally think if theres a need to drop scores to avoid bias, then you should pick better judges or a more homogeneous judging panel that fits your style of comp. Like saleem said, each judge should set a baseline in first two teams and score the rest of teams adjusted to that baseline

 

[MPElive]

So tangentially to the topic, something has been bugging me for a while now:

Why do many comps drop the highest and lowest score?! The initial mindset was to offset biases for/against teams...but this normalizes scores (leading to drama as the score variance is arbitrarily reduced) and it diminishes judges who judge the teams relatively (i.e. what I dub the faizan effect). If identical judges have identical scores for teams but judge B is just offset by 20 pts, he runs the risk of being discounted even though his point differential is the same (he just has a different base level for what an average performance should be allotted in points).

I personally think if theres a need to drop scores to avoid bias, then you should pick better judges or a more homogeneous judging panel that fits your style of comp. Like saleem said, each judge should set a baseline in first two teams and score the rest of teams adjusted to that baseline

Totally agreed. I am not a fan of dropping the highest and lowest score at all. There's only one competition that I judged who dropped the highest/lowest score but didn't make any difference in placings.

 

[rajkaran]

at the same time, I think the mystique of some teams have helped them placed when expectations are set. Imagine that you're known to be not a traditional team and you bring out some traditional flavor...the impact is much more and the threshold for what constitutes a 'good traditional segment' is lower compared to a new team. Conversely if you are known to be super clean and you take a risk, etc etc. Subjective judging inherently enters the human factor of expectations so yes a teams reputation affects the scoring

Swi, I agree with you that a teams reputation (unfortunately) does affect judging somehow. However, I think one can also say that if a team with a reputation for being non-traditional pulls out something traditional, people (namely the people who are blinded by the non-traditional reputation of the team...and unfortunately they might be judges) will still say the team hasn't done anything traditional or acknowledged traditionality. I speak from experience.

 

[Saleem]

A competition I judged recently suggested to drop highest and lowest scores. I'm glad they listened to the judges' (unanimous) feedback to not do that. It will often remove one judge from having any influence, and is not a good filter.

I think an interesting approach (that I've raised and we've discussed before) is that each judge takes his/her scores (lets say from 71 to 95) and normalizes that from 0 to 100. then all judges' scores are added and you should have a bigger difference, less issues of a "tie" or "off by 1 point". It should make differences more marked.

Kinnell, to respond to your point, yes I would reflect it in the Creativity section.

 

[yraparla]

I think an interesting approach (that I've raised and we've discussed before) is that each judge takes his/her scores (lets say from 71 to 95) and normalizes that from 0 to 100. then all judges' scores are added and you should have a bigger difference, less issues of a "tie" or "off by 1 point". It should make differences more marked.

This came up after elite 8 judging, doesn't work because the scale varies from judge to judge. So whereas a judge might place teams all 1 pt from each other, the scaling factor would make the difference now 8, 10, 12 pts instead of 1! In this case the judge's opinion should not be counted so heavily since clearly the judge did not feel the teams distinguished themselves from one another.

Because judges score arbitrarily by points, I tend to think you can't adjust the actual pt differentials lest you diminish a judges intent. I'm also wary of arbitrary subjective placing of teams after point tally, but that's a different story

 

[Kartik]

A team's reputation definitely has helped them place in the past, and it is important to realize that this cannot only be evaluated from a judge's perspective. There are many factors that go into a team's performance, and noting that "we do not take into account a team's reputation" etc. simply means that the directly the reputation is not why they are placing. However, this does not include many many many indirect ways in which a team is assisted by their reputation. I'll give an example:


This past weekend at Boston Bhangra DJ Pup Dawg was MCing as always, and came out before Bhangra Empire's performance. In his longest intro of the night (by far) he went on a tirade about how Bhangra Empire has solely taken bhangra to a national level and made the audience cheer multiple times: "put your hands up for this bhangra being at the white house" (everyone cheers) a little while later "everyone make some noise for bhangra being taken to a national level" (everyone cheers) a little while later "now put your hands together for bhangra empire" (everyone cheers. sidenote: I think he actually called them boston empire by accident haha).


At a competition like Boston, where crowd support, standing ovations, and pure entertainment are valued, making sure that the entire crowd recognizes Bhangra Empire as the team that everyone has been waiting for, is a great example of how reputation does influence how the dance is received. If instead, NYPD's routine had been hyped and over-hyped and triple-hyped, I bet you they would have had a marginally better crowd response than what they got because of their tough spot in the lineup.


There are many examples like this, and we can go on and on forever about the specifics. My main point is that for judges to say "no we do not take into account reputation" is a simple response to an extremely complex question that requires a lot more depth. I doubt Rahul3 was expecting you guys to respond with "yea totally, that's like number 3 in my list of top 5 things I look for". We, as teams, always appreciate unbiased judging, but specifically ignoring reputation on a direct level in no way ensures that it does not play a role.