ELITE 8 FINAL SCORES/STANDINGS

[nb0913]

i was looking thru the score sheets and saw that pashi gave empire 6 out of 5 points for props....im confused but maybe im reading something wrong...

Not because its Empire or anything, but care to tell us any of the other 559 marks that confused you/you could have read wrong?

Not because its annoying or anything, but care to tell us why you feel the need to be an arrogant ass to someone who asks a simple question? Or to 90% of the people on this forum for that matter?

The judging sheets literally have 560 different spaces for evaluation. Pashi made a mistake. Now on to the next one and the next one and the next one? Pashi's credibility as a judge will not change, neither will the scorings or overall 'placings'.

As far as you considering me being an arrogant ass to 90% of people on this forum, I thank you for giving me your input on the way I conduct myself on the forum and will try to be more conscientious of the way I approach others.

Or maybe Pashi found the Amplifier car gimmick you did so creative that it deserved some extra credit, which no other team received anywhere in the other 559 marks, so she was confronting that mistake/inconsistency/outlier, take it how you may. She clearly didn't read anything wrong as it blatantly says 6/5 on that category. No where does she say anything about the placings or the judges' credibility, so I don't even know where you're getting that comment.

Congratulations on repeating as an Elite 8 selection, but being on a successful team shouldn't give you a free pass to be a d-bag whenever you're bored.

 

[Remy]

everybody in this thread needs to calm their ass........LOL

Seriously - competition doesn't even matter ,we all love bhangra..enough said.

 

[Mojo]

APD definitely deserved to be higher up on that list. Empire was better than PANJ, but Elite8 has no love for their style so I'm not surprised they ranked so low.


But SGPD definitely deserved the top spot. In my mind, VCU was a close second followed by APD, not NJ.

I think APD and Empire fall short on Vardiyan, Tradional Aspects and grace, probably hurting their overall scores.

Yeah, the Elite8 rubric puts a lot of emphasis on traditional aspects, so I do see where the scores come from. I for one LOVE APD's vardiyan (those vests are insane), but I can see the judges being turned off by it not being very traditional. I thought both teams deserved to be ranked better based on the overall performance and how entertaining it was. I'm not a judge, so I won't meddle with their reasoning, but I enjoyed those two much more than their rankings suggest.


Just my opinion. I though all of the teams did a good job that night and ultimately, I thought the right people won.

 

[Laddi]

What I thought:

1. SGPD
2. NJ
3. PCBCA
4. VCU
5. APD
6. Empire
7. PANJ
8. Virsa


8)

 

[faizan]

1. SGPD
2. PCBCA
3. VCU
4. NJ
5. Empire
6. APD
7. PANJ
8. Virsa


Everyone is on edge on this forum, calm the mothership, calmm. jundiiiii.

 

[ahmad]

look point is....everyone agreees SGPD was 1st and seeing that its winner takes all judges chose the right winner....

But PCBCA was my favorite of the night

 

[Sue Sylvester]

The Elite 8 judging process, once again, is logically unsound. I cannot comment on whether this is what caused the discrepancy and what seems to be disagreement between the calculated overall placings and personal opinions of the BTF community, but EMDO and other bhangra competitions should really have more mathematically savvy individuals look over the process.

The problem lies in the fact that in attempt to remove bias, the highest and lowest scores are dropped. Judges have their own grading scales, so a harsher judge who scores everyone low and a nicer judge who gives everyone high marks and dots their letter "i"'s with hearts and smiley faces, will unintentionally be removed from contributing to the final marks. The scores need to be normalized (in addition to standard deviations) in order for such a method to be used correctly. It may have simply been better to just keep all the scores.

It needs to be said, because I feel it is too easily forgotten: Thank you, EMDO, for making these files public for all to see, comment on and provide critique on. More and more competitions should make their judging process and results transparent not only to the teams that competed, but to the bhangra community.

 

[harjap_singh]

i think pcbca is so low because they went first. If they went later in the show, i have no doubt in my mind they would be in the top 4. The judges did not know how good the teams would be, and probably assumed the pcbca was just the first in line of more amazingness to come.

completely agree. oh and apd shoudlnt be that hi. apd was invited because they won alot of competitions, but honestly the only reason why is cuz they just did the same routine over and over and over again. not an elite team (my opinion)

 

[faysal]

...they won a lot of comps bro, that kinda automatically puts them in the elite category as its the same reason why the rest of the teams are selected. Whether they are as elite as their competitors is the whole point of the competition no? I understand where you're coming from but your argument didn't make any sense that's all lol

 

[Basim]

The Elite 8 judging process, once again, is logically unsound. I cannot comment on whether this is what caused the discrepancy and what seems to be disagreement between the calculated overall placings and personal opinions of the BTF community, but EMDO and other bhangra competitions should really have more mathematically savvy individuals look over the process.

The problem lies in the fact that in attempt to remove bias, the highest and lowest scores are dropped. Judges have their own grading scales, so a harsher judge who scores everyone low and a nicer judge who gives everyone high marks and dots their letter "i"'s with hearts and smiley faces, will unintentionally be removed from contributing to the final marks. The scores need to be normalized (in addition to standard deviations) in order for such a method to be used correctly. It may have simply been better to just keep all the scores.

It needs to be said, because I feel it is too easily forgotten: Thank you, EMDO, for making these files public for all to see, comment on and provide critique on. More and more competitions should make their judging process and results transparent not only to the teams that competed, but to the bhangra community.

Definitely agree with your points. I understand the attempt to remove bias, but there needs to be a better way of going about doing so.

Kinda funny that you mentioned "a nicer judge who gives everyone high marks and dots their letter "i"'s with hearts and smiley faces" because when you look at the Team Scores Excel file, you'll notice the "Max Dropped" column has almost the every score from this very judge. Nothing against Jasmine as a person, but her "niceness" leads to the entire competition primarily being judged by 4 individuals instead of 5.

Overall, EMDO does a miraculous job in terms of organizing/execution and really pushes the envelope whenever they host the Elite 8 competition. Their efforts should really encourage other competitons to step up their game. I agree the transparency when releasing judging results is unparellel and only helps future teams get an idea of the entire judging process.

~ Basim

 

[faizan]

Despite all the bullshit (and i think there was more last year), Elite 8 is still the most important, and best competition in north america. I am sure they will learn EVEN MORE from this year, and be EVEN BETTER next year. Don't let btf comments deter you guys, we all just like bitching, you're competition is still fundamentally sound.

 

[Vic]

The judges had the freedom to deliberate after the scores were added up. So if there was more then one placing, I'm sure they would sit down and discuss all the teams performances and get it right.

 

[yraparla]

'bias' is really an asset, not a detriment! A traditional judge judges traditionally...that's how it should be. If you don't like that don't add him/her to your judging panel, OR don't invite certain-style teams into a losing scenario. Either way the real way to be fair is to have a balanced panel with different judging preferences. Then the group as a unit is balanced/fair as long as every score from every judge is counted.


Also about elite 8 scores, if you adjust for difference in means there was no 'nice' judge. All the judges had std devs from 9.0-10.2, jasmine just had a higher average but the variance of her scores was on par with everyone else so teams. I attached a chart so you can see it visually. I took the median of the judges averages (I think it was pashi's at 72) and just centered all the other judges scores around that. It's pretty cool to see.

I don't know what an overall solution for scoring is. Scaling is not a solution because it is vulnerable to outliers. i.e. if you have one team that is really bad, they will mess up the scaling factors. I guess you could calculate a std dev for each judge and then adjust scores on that so the variance is comparable. But it wouldn't do much for such a small dataset

 

[faizan]

Whew, I new a guju with a chart is what this thread needed, sanity is restored. thanks swi.

 

[yraparla]

Whew, I new a guju with a chart is what this thread needed, sanity is restored. thanks swi.

haha faizannn
1) not gujju. think masala dosa
2) it's a cool chart! Just pointing out that jasmine was as critical as any other judge

 

[faizan]

South Indian? Even better, haha. jokes my guju faithful, jokes.

 

[Sue Sylvester]

What would I do without you, Swi!

You bring up two really good points that has made me re-evaluate how to approach this.

The first that every judge brings a different perspective to the overall score. We all know that some of these judges are more traditional and others more modern. Removing a score from one judge is analogous to have judges evaluate different aspects of the performance (formations, choreography, execution, vardiyan) and then simply removing one judge's score for that category. It wouldn't make sense. While this is not the case, each judge is evaluating a performance and allocating points based on their interpretation of dancing/bhangra acquired from years of experience.

The second point is the notion that scaling will be affected by teams who performed exceptionally well compared to the rest or essentially gave people an excuse to use the restroom. The point of removing intended bias is to eliminate the possibility that a judge had preconceived notions about a team and let that negatively/positively affect the way they allocated the points. However, it is necessary to differentiate between an undeserved biased score and a deserved outlier. The difference is that the former will appear only on one judge's score sheet while the latter should appear on most score sheets.

Let me ponder for a bit...

 

[Sue Sylvester]

The point of judging, unlike getting grades in school work (with the exception of most engineering classes), is not about the absolute score, rather the relativity of teams to each other. It doesn't matter if Team A got a 97 average, it's how Team A did compared to the rest of the competition.

With that said, it's imperative that we start somewhere and then attempt to work backwards to remove inconsistencies and analyze and interpret the data.

Let's call Method 1.0, the "Elite 8" Method. This is simply a sum of all the values with an "attempt" to remove bias by removing high/low scores.

Rankings: SGPD > VCU > NJ > PANJ > PCBCA > APD > BE > V:OT

Let's call Method 2.0, the "Absolute Sum" Method. This is simply a sum of all the values without attempting to remove bias.

Rankings: SGPD > VCU > NJ > PANJ > PCBCA > BE > APD > V:OT

Let's try something new...

Method 3.0: "Basic Z-Score"
Benefits: Analysis of scores relative to each other, scaled to individual judging scales
Drawbacks: Effect of excellent performances/really-poor performances on determining means/std to scale the scores (Brought up by Swi)

A z-score is essentially how far a certain score is from the mean in terms of standard deviations, which is the average distance each point is from the mean. It is simply calculated by taking each score and subtracting the judge's average and dividing it by that judge's standard deviation.

In my opinion, the sum of the z-scores received by each team would give a better indicator of placings than summing absolute values. There is an obvious drawback (If you see more drawbacks, feel free to point them out.) and I'll try to compensate for it in later methods.

But from this method,
Rankings: SGPD > VCU > NJ > PANJ > PCBCA > BE > APD > V:OT

Graph of comparison attached.

As I said, the methods can be improved logically and I hope to do that. So far, the only changes we notice is the placings of APD & BE, but more changes may make themselves apparent as further techniques are used to remove outliers...etc. For example, SGPD was placed first by every judge and by some judges, by far (almost 2 z-scores ahead of their averages by some). If we solidify SGPD's place as 1st and remove their scores, we might find more accurate placings for the rest of the teams and remove the outlier of an exceptional performance.

 

[Gurbir23]

Oh man stats midterm tommorw, who knew I could procrastinate on BTF and study at the same time...GO BTF!

 

[Danny]

I think this is being blown up a little too much now. These rankings mean nothing and shouldn't mean much to anyone either. Elite is a "[one]Winner-takes-all" competition and big ups to SGPD on it.


Any team worth their salt would agree with me on this one: it doesn't matter if Elite called me today and told us we got 2nd place or 2,002nd place...at the end of the day, it wasn't the top spot at a competition where only one winner is crowned. Not knocking on your theories and formulas and all that good stuff, keep at it for all i care, but the winner has been picked and there was little to no objection then nor is there any now.