Elite 8 Bhangra Invitational - Judging Package

[faizan]

Saleem, Kinnell, Swi, Congrats on making this thread incomprehensible to everyone.

I was a harsh judge, always have been. Ask my team what comments and numbers I give them during our run throughs. (we judge each other). No big deal, I think all my scores were dropped anyway.

 

[Saleem]

Swi I like how you bring up how different ranges (judges who give 80 to 100 versus another who gives 70 to 80 and is marginalized). Here's the suggestion:

Each score gives their sheets in with whatever grading they gave. Then their scores are normalized 0 to 100. So if Judge A gave scores between 70 and 82, the team that got a 70 gets 0 and 82 gets 100 and everything in between is linearly interpolated.

This gives:
- Each judge equal say regardless of judging style and range
- Large differences between teams (not 1pt 2pt) so it's less likely to tie

I was a harsh judge, always have been. Ask my team what comments and numbers I give them during our run throughs. (we judge each other). No big deal, I think all my scores were dropped anyway.

Faizan dont think that anyone's putting you down; it's just a different judging style. However it does highlight a flaw in the "rotating panel" of judges method.

 

[Billy Bob]

Exactly, they need to normalize the scores either based on what Saleem said. Or better yet normalize them against the judges scores because they were consistenlty there.

 

[kinnell]

I can say with 90% confidence that saleem and swi are my new favorite BTF people!

Swi I like how you bring up how different ranges (judges who give 80 to 100 versus another who gives 70 to 80 and is marginalized). Here's the suggestion:

Each score gives their sheets in with whatever grading they gave. Then their scores are normalized 0 to 100. So if Judge A gave scores between 70 and 82, the team that got a 70 gets 0 and 82 gets 100 and everything in between is linearly interpolated.

This gives:
- Each judge equal say regardless of judging style and range
- Large differences between teams (not 1pt 2pt) so it's less likely to tie

I am going to try that later today and see what the results say. Nonetheless, we need something to compare that to and see whether normalizing would have affected the results. see the next post.

 

[siddyp]

We're brown, too many of us are too good at math for this shit to keep happening

hahaha +1

 

[Saleem]

I can say with 90% confidence that saleem and swi are my new favorite BTF people!

Only 90%? Thats not statistically significant and you know it hahaha I'm here til Tuesday, try the veal!

</nerd>

 

[kinnell]

Data: http://spreadsheets.google.com/pub?key=tmUnF56SYz-5N56K01QvDOQ&output=html

The inherent flaw in the system are the team judges. In my opinion, there are three reasons why:
1) They do not watch all the performances.
2) Their qualifications to judge are never questioned.
3) While we do want to overlook it, there are rivalries between teams. In addition, teams may not like each other's styles. Essentially, we have a variation of game theory present because these teams are competing against each other.

Let's compare how the teams ranked (both raw and cut) to how the actual judges (both individual and combined).

Attached.

It will be interesting to see how things would have played out if we normalize the scores.

EDIT: I made a mistake in the Totals. I fixed it now.

 

[nb0913]

bhangra used to be the one of the few things i had to get away from school....

not so much anymore haha.

one other way to get rid of partial bias from the team judges wouldve been to make the 2 team judge scores only 25% of the final average, or some percentage, rather than making all 4 of the judges having an equal 25%. just to get rid of a little bit of the impact that any bias would come from teams juding each other/the fact that each member only watches 4 teams, not all of them.

just my thoughts

 

[yraparla]

I am going to try that later today and see what the results say. Nonetheless, we need something to compare that to and see whether normalizing would have affected the results. see the next post.

exactly, have to try different methods and see what results are. And please, no one think we're trying to figure out the system that allows any one team to place or anything like that. This is all objective of the teams, they are simply reference points for analyzing the effects we create (i.e. if we see drastic changes in scores or placings then we have a reference point to examine what happened).

 

[Vick]

What the hell is going on in this thread? Feels like senior year stats courses...

 

[yraparla]

Would it be worth dropping highest and lowest score in EACH category for each team instead of dropping highest lowest judge?

Would this marginalize differences though? like reduce disparity making each point matter that much more...

 

[Saleem]

Right swi it would make each point count more... which is I think a bad thing b/c points are given almost as a split second decision [how much time is there really between acts when you dont have an intro video, just MCs], plus there was no opportunity to go back and review the scores.

 

[nashtybangraw]

if you avearge out the scores given by the judges the placings would look something like the following
sgpd
vcu nachdi jawani

 

[ban3rg]

For any team that was disgruntled about judging...please remember that everyone is guaranteed to place at Pao Bhangra 9

http://bhangrateamsforum.com/discuss/index.php/topic,8548.0.html

 

[Kanvar]

kinnell cool ur jets with the spreadsheets.

 

[ranjit singh]

Nachdi Jawani should have won. That is all.

 

[Vick]

Nachdi Jawani should have won. That is all.

dude shut up...

 

[ranjit singh]

Nachdi Jawani should have won. That is all.

dude shut up...

dude. you're cool.

 

[RS.JATT]

YOU KNOW WHAT REALLY GRINDS MY GEARS? a comp that lets team judge other teams :S

 

[mafzal]

The rotating panel of judges seems like a big problem to me.

What determined who judged who?