• Bhangra discussion is still going strong. Join us in our Facebook group!

    New user registration has been closed (as it was entirely spam). We encourage you to post in our Facebook group, even if it's a followup to an existing thread. BTF will continue to be archived and hosted here - Saleem

Judging Bias/Rubrics

siddyp

Tough times never last, but tough people do.
Messages
1,270
I disagree that simply because a judge goes beyond the categories, he is deemed to throw a competition and hence should be responsible for any perceived backlash as to why another team should have gotten first.

Perhaps that judge disagreed with the already biased nature of the Judging rubric, and thought of factors outside of the rubric that should be considered. There should be some flexibility in this sense, but without it, we are left with just a biased rubric for the judge to evaluate.
No. Absolutely not. We've seen in the past when judges do this. It does not end well.

There has to be a playing field which all teams are judged equally too. That is the rubric. No team should have to predict what a judge is going to do. And that is why we have a rubric. So all teams know what they're being judged by, what is going to be valued at this competition, etc. It allows complete transparency. No team should have to pay registration fees, drive/fly out, get hotels, etc. ($$$) and spend the time/headaches/stress to practice, build the set, and do all a team does to prepare for a comp, to then be judged in a way that they had no idea about.

As you said, the rubric is "biased" which yes it is. But at least the competing teams know what that bias will be. They know which sections/categories etc will be weighed heavier than others. They'll know what the wording is that the judges are using to judge their performance. There should be no more bias or "flexibility" to take into hand.

Judges are folks that are deemed qualified or knowledgable enough to be able to watch a competition and rank teams according to said rubric. If they disagree with something on the rubric, that should be brought up long before the competition. If rubric does get adjusted, it needs to be made known to teams. All in all, no judge should be put on a pedestal to go against or "beyond" the rubric. That is not fair to anyone.
 

sahab

Well-Known Member
Messages
169
No. Absolutely not. We've seen in the past when judges do this. It does not end well.

There has to be a playing field which all teams are judged equally too. That is the rubric. No team should have to predict what a judge is going to do. And that is why we have a rubric. So all teams know what they're being judged by, what is going to be valued at this competition, etc. It allows complete transparency. No team should have to pay registration fees, drive/fly out, get hotels, etc. ($$$) and spend the time/headaches/stress to practice, build the set, and do all a team does to prepare for a comp, to then be judged in a way that they had no idea about.

As you said, the rubric is "biased" which yes it is. But at least the competing teams know what that bias will be. They know which sections/categories etc will be weighed heavier than others. They'll know what the wording is that the judges are using to judge their performance. There should be no more bias or "flexibility" to take into hand.

Judges are folks that are deemed qualified or knowledgable enough to be able to watch a competition and rank teams according to said rubric. If they disagree with something on the rubric, that should be brought up long before the competition. If rubric does get adjusted, it needs to be made known to teams. All in all, no judge should be put on a pedestal to go against or "beyond" the rubric. That is not fair to anyone.
Agree 100%.

Sid to add to your last point, the judge should also be qualified to explain their thought process when ranking teams to the rubric in a concise and informative manner. An added bonus is the judge can then explain to that team where they can improve to increase their ranking to that rubric.
 

yomamajama

Active Member
Messages
230
Then why not just let the people vote? No one is coming to these competitions, the marketing aspect for NA comps is ridiculously terrible, and who cares what the expectations are of teams and there “preparation”?

If you are interested in going to a competition, its reasonably foreseeable that anything can happen. We have been having it this dumb way for a while now, and it has gone no where.

There is always going to be bias/conflict/etc., it’s fundamental human nature. You resolve it by assessing the opinions of people, not some judge.
 

Saleem

Administrator
Staff member
Messages
1,928
Then why not just let the people vote? No one is coming to these competitions, the marketing aspect for NA comps is ridiculously terrible, and who cares what the expectations are of teams and there “preparation”?

If you are interested in going to a competition, its reasonably foreseeable that anything can happen. We have been having it this dumb way for a while now, and it has gone no where.

There is always going to be bias/conflict/etc., it’s fundamental human nature. You resolve it by assessing the opinions of people, not some judge.
Chiming in here. If you allow anyone to vote electronically (online), you're inviting spam as people will send out the link and get everyone they know to vote. In a venue of capacity 700 you might have 5,000 votes. Then it's just a social media popularity contest and no one even needs to show up and dance.

If you instead allow only people in the venue to vote, you heavily bias local teams who are able to get more supporters to show up.

I'm not sure how the question of "these hand-selected judges have bias or afilliations to teams!" is solved by saying "allow anyone with a ticket to the show despite their affiliations to teams/schools vote"
 

yomamajama

Active Member
Messages
230
If you instead allow only people in the venue to vote, you heavily bias local teams who are able to get more supporters to show up."
What is more economically viable, a packed, sold out audience? Or a half-to full empty audience because of your perceived little judging morals?

Sometimes you people overthink, and starting to make things nonsensical. If my idea is going to bring more money to competitions, should at least try it.
 

Saleem

Administrator
Staff member
Messages
1,928
Any competition can feel free to do in-venue voting. Do it by applause volume if you want, it would be fun.

But if you're looking to solve the thread topic of "Judging Bias/Rubrics"-- your solution doesn't solve anything.
 

yomamajama

Active Member
Messages
230
Any competition can feel free to do in-venue voting. Do it by applause volume if you want, it would be fun.

But if you're looking to solve the thread topic of "Judging Bias/Rubrics"-- your solution doesn't solve anything.
Have you tried it? How do you know it’s not going to work?

Simply saying something without any evidence to rebut only shows the fallacy and insanity that is the Bhangra circuit. Doing the same thing over and over again, expecting a different result.
 

yomamajama

Active Member
Messages
230
I’m not trying to attack you Saleem, I, and many others, are just simply frustrated at the process and how slow the development is.

We are smart people, it’s time we start thinking outside the box.
 

Saleem

Administrator
Staff member
Messages
1,928
I know youre not attacking, it's all good. Competitions can try out new stuff. I think (as I think I wrote too many years ago now lol) that the solution is
  • well thought-out rubrics so that we reward and incentivize the behavior we want
  • adherence to that rubric
  • transparency of judges selection, which hopefully leads to respected judges
  • transparency of judging outcomes
It's hard, and unfeasible for beginner competitions, which is why we have this problem again and again. Maybe competitions should try the audience applause measurement because it'll be fun and interesting. I think it'd be interesting to do that in conjunction with an exhibition like Pao Bhangra, so that every team is still getting paid and maybe one team gets an extra award as Audience Favorite.
 

srazdan

Active Member
Messages
11
Actually, we are doing audience voting for the match-up winners at Pao Bhangra! Last year was the first time we tried it (both the matchups concept as well as the audience polling concept) and we intend to continue it into this year.

We did an online voting system, so that the win was more quantifiable and the audience seemed to love it! We gave the match-up winners a small trophy to bring in a little bit of competitiveness to an overall fun event.

I think this system works for Pao, where the end goal is to perform for an audience and have fun on stage. The same might not hold true for a competition where teams want to know if they are and get rewarded for being better than other teams.

(This year's matchups should be interesting. FCB vs AEG and GT vs Hurricane. Tune in on March 23rd! We should have a livestream up for the event)
 

Saleem

Administrator
Staff member
Messages
1,928
Actually, we are doing audience voting for the match-up winners at Pao Bhangra! Last year was the first time we tried it (both the matchups concept as well as the audience polling concept) and we intend to continue it into this year.

We did an online voting system, so that the win was more quantifiable and the audience seemed to love it! We gave the match-up winners a small trophy to bring in a little bit of competitiveness to an overall fun event.

I think this system works for Pao, where the end goal is to perform for an audience and have fun on stage. The same might not hold true for a competition where teams want to know if they are and get rewarded for being better than other teams.

(This year's matchups should be interesting. FCB vs AEG and GT vs Hurricane. Tune in on March 23rd! We should have a livestream up for the event)
Awesome!
 

sriharifez

Member
Messages
53
there is no perfect rubric or perfect system - every comp's board decides what they value and what they wanna see on their stage. in fact rubric is one of the last chances a competition has to really influence the vibe of a competition before it gets into the hands of the judges.

folk competitions should have a rubric that reflects that central theme of being traditional.
modern competitions should not have a rubric identical to folk comps - that would be counterintuitive.
the audience/public should never be trusted in any circumstance except an exhibition style event like Pao where that's the whole point and its perfect. else you get insane home field advantage and out-of-towners never stand a chance.

and you just have to respect the rubric - sometimes the team you want to win won't win at the competition they're at because of a lack of fit. (then again, bruin last year is maybe an interesting counterexample.....lol)

p.s. a fun thought experiment, in the past (non-dance) competitions have played with the idea of rubrics without numbers, so its solely based off of relative rank between teams for each category. there's more than one way to do it i guess....
 

yomamajama

Active Member
Messages
230
the audience/public should never be trusted in any circumstance except an exhibition style event like Pao where that's the whole point and its perfect. else you get insane home field advantage and out-of-towners never stand a chance.
Why can’t the audience/public be trusted? This is rather conclusory and you don’t establish any facts to support this baseless contention. I mean really? Good luck trying to market your competition to folks if your gonna say that.

People can surely be trusted, if you play to their likes. An example can be seen in the American judicial system, whereby we allow parties to have a right to present their case at trial to 12 people, not simply just a judge. Why is this? Because society realizes that people have social understandings of things that cannot be transcribed and/or that cannot be decided by a judge who may have unknown biases. Bhangra is a dance that many people in the Punjabi/ large Asian subcontinent community have grown up. I mean ask your uncles, aunts, etc. and they will give you a better understanding of Bhangra than a college team performing at some competition.
 
Top