Judging Bias/Rubrics

siddyp

Tough times never last, but tough people do.
Messages
1,249
I disagree that simply because a judge goes beyond the categories, he is deemed to throw a competition and hence should be responsible for any perceived backlash as to why another team should have gotten first.

Perhaps that judge disagreed with the already biased nature of the Judging rubric, and thought of factors outside of the rubric that should be considered. There should be some flexibility in this sense, but without it, we are left with just a biased rubric for the judge to evaluate.
No. Absolutely not. We've seen in the past when judges do this. It does not end well.

There has to be a playing field which all teams are judged equally too. That is the rubric. No team should have to predict what a judge is going to do. And that is why we have a rubric. So all teams know what they're being judged by, what is going to be valued at this competition, etc. It allows complete transparency. No team should have to pay registration fees, drive/fly out, get hotels, etc. ($$$) and spend the time/headaches/stress to practice, build the set, and do all a team does to prepare for a comp, to then be judged in a way that they had no idea about.

As you said, the rubric is "biased" which yes it is. But at least the competing teams know what that bias will be. They know which sections/categories etc will be weighed heavier than others. They'll know what the wording is that the judges are using to judge their performance. There should be no more bias or "flexibility" to take into hand.

Judges are folks that are deemed qualified or knowledgable enough to be able to watch a competition and rank teams according to said rubric. If they disagree with something on the rubric, that should be brought up long before the competition. If rubric does get adjusted, it needs to be made known to teams. All in all, no judge should be put on a pedestal to go against or "beyond" the rubric. That is not fair to anyone.
 

sahab

Well-Known Member
Staff member
Messages
161
No. Absolutely not. We've seen in the past when judges do this. It does not end well.

There has to be a playing field which all teams are judged equally too. That is the rubric. No team should have to predict what a judge is going to do. And that is why we have a rubric. So all teams know what they're being judged by, what is going to be valued at this competition, etc. It allows complete transparency. No team should have to pay registration fees, drive/fly out, get hotels, etc. ($$$) and spend the time/headaches/stress to practice, build the set, and do all a team does to prepare for a comp, to then be judged in a way that they had no idea about.

As you said, the rubric is "biased" which yes it is. But at least the competing teams know what that bias will be. They know which sections/categories etc will be weighed heavier than others. They'll know what the wording is that the judges are using to judge their performance. There should be no more bias or "flexibility" to take into hand.

Judges are folks that are deemed qualified or knowledgable enough to be able to watch a competition and rank teams according to said rubric. If they disagree with something on the rubric, that should be brought up long before the competition. If rubric does get adjusted, it needs to be made known to teams. All in all, no judge should be put on a pedestal to go against or "beyond" the rubric. That is not fair to anyone.
Agree 100%.

Sid to add to your last point, the judge should also be qualified to explain their thought process when ranking teams to the rubric in a concise and informative manner. An added bonus is the judge can then explain to that team where they can improve to increase their ranking to that rubric.
 

yomamajama

Active Member
Messages
158
Then why not just let the people vote? No one is coming to these competitions, the marketing aspect for NA comps is ridiculously terrible, and who cares what the expectations are of teams and there “preparation”?

If you are interested in going to a competition, its reasonably foreseeable that anything can happen. We have been having it this dumb way for a while now, and it has gone no where.

There is always going to be bias/conflict/etc., it’s fundamental human nature. You resolve it by assessing the opinions of people, not some judge.
 

Saleem

Well-Known Member
Staff member
Messages
1,916
Then why not just let the people vote? No one is coming to these competitions, the marketing aspect for NA comps is ridiculously terrible, and who cares what the expectations are of teams and there “preparation”?

If you are interested in going to a competition, its reasonably foreseeable that anything can happen. We have been having it this dumb way for a while now, and it has gone no where.

There is always going to be bias/conflict/etc., it’s fundamental human nature. You resolve it by assessing the opinions of people, not some judge.
Chiming in here. If you allow anyone to vote electronically (online), you're inviting spam as people will send out the link and get everyone they know to vote. In a venue of capacity 700 you might have 5,000 votes. Then it's just a social media popularity contest and no one even needs to show up and dance.

If you instead allow only people in the venue to vote, you heavily bias local teams who are able to get more supporters to show up.

I'm not sure how the question of "these hand-selected judges have bias or afilliations to teams!" is solved by saying "allow anyone with a ticket to the show despite their affiliations to teams/schools vote"
 

yomamajama

Active Member
Messages
158
If you instead allow only people in the venue to vote, you heavily bias local teams who are able to get more supporters to show up."
What is more economically viable, a packed, sold out audience? Or a half-to full empty audience because of your perceived little judging morals?

Sometimes you people overthink, and starting to make things nonsensical. If my idea is going to bring more money to competitions, should at least try it.
 

Saleem

Well-Known Member
Staff member
Messages
1,916
Any competition can feel free to do in-venue voting. Do it by applause volume if you want, it would be fun.

But if you're looking to solve the thread topic of "Judging Bias/Rubrics"-- your solution doesn't solve anything.
 

yomamajama

Active Member
Messages
158
Any competition can feel free to do in-venue voting. Do it by applause volume if you want, it would be fun.

But if you're looking to solve the thread topic of "Judging Bias/Rubrics"-- your solution doesn't solve anything.
Have you tried it? How do you know it’s not going to work?

Simply saying something without any evidence to rebut only shows the fallacy and insanity that is the Bhangra circuit. Doing the same thing over and over again, expecting a different result.
 

yomamajama

Active Member
Messages
158
I’m not trying to attack you Saleem, I, and many others, are just simply frustrated at the process and how slow the development is.

We are smart people, it’s time we start thinking outside the box.
 

Saleem

Well-Known Member
Staff member
Messages
1,916
I know youre not attacking, it's all good. Competitions can try out new stuff. I think (as I think I wrote too many years ago now lol) that the solution is
  • well thought-out rubrics so that we reward and incentivize the behavior we want
  • adherence to that rubric
  • transparency of judges selection, which hopefully leads to respected judges
  • transparency of judging outcomes
It's hard, and unfeasible for beginner competitions, which is why we have this problem again and again. Maybe competitions should try the audience applause measurement because it'll be fun and interesting. I think it'd be interesting to do that in conjunction with an exhibition like Pao Bhangra, so that every team is still getting paid and maybe one team gets an extra award as Audience Favorite.
 

srazdan

Member
Messages
9
Actually, we are doing audience voting for the match-up winners at Pao Bhangra! Last year was the first time we tried it (both the matchups concept as well as the audience polling concept) and we intend to continue it into this year.

We did an online voting system, so that the win was more quantifiable and the audience seemed to love it! We gave the match-up winners a small trophy to bring in a little bit of competitiveness to an overall fun event.

I think this system works for Pao, where the end goal is to perform for an audience and have fun on stage. The same might not hold true for a competition where teams want to know if they are and get rewarded for being better than other teams.

(This year's matchups should be interesting. FCB vs AEG and GT vs Hurricane. Tune in on March 23rd! We should have a livestream up for the event)
 

Saleem

Well-Known Member
Staff member
Messages
1,916
Actually, we are doing audience voting for the match-up winners at Pao Bhangra! Last year was the first time we tried it (both the matchups concept as well as the audience polling concept) and we intend to continue it into this year.

We did an online voting system, so that the win was more quantifiable and the audience seemed to love it! We gave the match-up winners a small trophy to bring in a little bit of competitiveness to an overall fun event.

I think this system works for Pao, where the end goal is to perform for an audience and have fun on stage. The same might not hold true for a competition where teams want to know if they are and get rewarded for being better than other teams.

(This year's matchups should be interesting. FCB vs AEG and GT vs Hurricane. Tune in on March 23rd! We should have a livestream up for the event)
Awesome!
 
Top