Elite 8 Bhangra Invitational - Judging Package

[Vick]

SGPD got robbed in my opinion by the team judging. That one score of 70 took them down to 3rd. If they had gotten even a few more points they would've won. I realize that case could be made for every team that if a few more points were given here and there they could've won, but this specific score really jumped out at me, as the 70 was well below any of the other scores given. Obviously this kind of thing was expected for some team or another. Also arguable that Pashi's score of 98.5 was a little much as well, but theres always the never ending discussion of judging; it'll never be perfect. So not gonna bitch about that here. It was experimental, it had its ups and downs, just like everything else does in bhangra and anything else.

From just watching the videos that I've seen (VCU, SGPD, NJ, AEG), I personally thought NJ should've won, with SGPD in a closeee second. VCU was definitely the cleanest of the teams, but I felt like SGPD and NJ both hit much harder on that stage. Bringing out the alumni was a creative idea, and definitely got the crowd going, but NJ and SGPD going crazy with 16 dancers topped that in my opinion. NJ's super speed sapps segment and creativity throughout put them at first in my mind.

Congrats to VCU for winning it all, and to all the teams for competing. I wish I had made the decision to roll up there haha, regretting it like crazy now

What about the 65 by Mark? I know it was dropped, but what routine was he watching?

 

[sulmoney]

for your consideration...

 

[oJaLa]

But the props/gimmicks were outta 5 not 10 im pretty sure

a 1-10 scale is a default scale. and props were outta 4

 

[MPElive]

I give big props to Emdo Entertainment for releasing all the judging scores to the public. I think all the competitions should provide scoring sheets to the teams if not release to the general public.

Faizan, damn brother.. you mark teams harddddddd. LOL

 

[Dholaholic]

i may be wrong but on sgpd's scoresheet on the excel document it says that jasmines (lbc) and marks (nj) scores were dropped. Jasmine gave SGPD 92 but the real highest score was 98.5 given by Pashi so shouldnt Pashi's score be dropped, not Jasmines?

 

[nb0913]

SGPD got robbed in my opinion by the team judging. That one score of 70 took them down to 3rd. If they had gotten even a few more points they would've won. I realize that case could be made for every team that if a few more points were given here and there they could've won, but this specific score really jumped out at me, as the 70 was well below any of the other scores given. Obviously this kind of thing was expected for some team or another. Also arguable that Pashi's score of 98.5 was a little much as well, but theres always the never ending discussion of judging; it'll never be perfect. So not gonna bitch about that here. It was experimental, it had its ups and downs, just like everything else does in bhangra and anything else.

From just watching the videos that I've seen (VCU, SGPD, NJ, AEG), I personally thought NJ should've won, with SGPD in a closeee second. VCU was definitely the cleanest of the teams, but I felt like SGPD and NJ both hit much harder on that stage. Bringing out the alumni was a creative idea, and definitely got the crowd going, but NJ and SGPD going crazy with 16 dancers topped that in my opinion. NJ's super speed sapps segment and creativity throughout put them at first in my mind.

Congrats to VCU for winning it all, and to all the teams for competing. I wish I had made the decision to roll up there haha, regretting it like crazy now

What about the 65 by Mark? I know it was dropped, but what routine was he watching?

NJ gave SGPD the lowest score, and vice versa. I just kind of expected that..

 

[nb0913]

i may be wrong but on sgpd's scoresheet on the excel document it says that jasmines (lbc) and marks (nj) scores were dropped. Jasmine gave SGPD 92 but the real highest score was 98.5 given by Pashi so shouldnt Pashi's score be dropped, not Jasmines?

only the team scores were dropped, pashi and navdeeps scores were kept regardless of rank

 

[Don]

haha damn benny is a generous grader

 

[zagreus]

haha damn benny is a generous grader

and faizan is hard to impress. haha.

 

[Meistro]

I really hope this becomes a trend in judging. However, I would make the grading scale to be more objective because there was clearly a discrepancy in some of the scores. Either some of you were being assholes on purpose, didn't care, or wanted to grade easy. There has to be a balance and all judges have to judge the same.

 

[Naveen Singh Khokhar]

ya'll should add a part at the end where the judges can choose their top three in order after watchin all the teams to decide who they think should have won. i can see it hard to be consistent with giving points because everything is based off of how you judge the first team. also not having the same judges for every team could skew scores also cuz if one or two judges are willin to give higher scores while another two generally give lower scores the teams that get only hard judges get kind of screwed. Maybe if the judges discussed the performance after they watched it among themselves the scores would be closer rather than some scores being much higher/lower than other scores.

**or you could have a home team perform first (or a control team) that's not competing so the judges can base the performances off of that performance by designating that team with specific points for each category. For example, giving that team 50% of the points in all categories so the judges can judge the competing teams based on that first performance

 

[Harmeet]

US West got murked! haha ouch.

 

[kinnell]

Thanks so much for this, EMDO! This allows for a lot of insight and gives us data to work with to see how the experimental Elite 8 judging system worked out and see whether it worked.

Preface to this: I am providing mathematical insight and making conclusions from all the numbers Emdo has so generously given us. I will explain what I am doing. However, no one is forcing you to read it. Why not apply the stuff we learned in high school and college to something we all love to do? Also, my analysis may take up a few posts. Remember, this is to provide insight and promote discussion, not cause stupid flaming wars.

First thing is first. You cannot take figure out who is generous grader and who is a harsh grader from sulmoney's chart. Remember that a team judge ONLY watched four performances! It is obvious that the actual judges will be the most balanced and fair judges because they have watched all the performances and probably not take part in the game theory. If Person A gave four low scores, is it fair to ascertain that he/she is a harsh grader if they watched some of the less awesome performances? Hell no. sulmoney's chart is way too simplified and may be a bit misleading. Due to the low number of judge scores, sulmoney's chart may be indicative of the actual graders but it is definitely not the best way to determine this.

Here is a better way to do it. We want to see which judges were nice and which judges were harsh. This can only be done by taking the score they gave for a certain team and comparing it to the scores of the other judges. We find the average of all the scores. The standard deviation is essentially the average difference that each score is from the average. The Z-score is how many standard deviations the given score is from the average. We can use this as a way to see how far Person A's score was from the average relative to the other scores that they gave. The average Z-score will show how the judge graded against his fellow team judges.

5 minutes later. Attached are the results, which closely resemble the "average" method used by sulmoney. The only main discrepancy is that Vic was a good amount more generous than Mike which was not indicated before. As sulmoney pointed out, this is for your consideration! More to analysis to come soon!

 

[Sandeep Sahota]

I don't know if it has already been mentioned but how was it decided which four teams judge each squad?

 

[ENVY BANG-RUH]

Seeing this, Faizan reminds me of simon cowell. lol

 

[sulmoney]

oh kinnell, you just took it to the level i was too lazy to do so haha. we can run all the statistical tests and what not that we want, but the bottom line is that this judging system is inherently flawed. having different judges for different teams, with each judge having their own range of scores that they are giving out leads to serious discrepancies. the most ideal situation would have been to have pashi and navdeep, and then ask each team to bring their own person who was a designated judge (alumni, coach, choreographer, ie someone not performing). then everyone can have their own scale that does not affect the cumulative, which gets affected by throwing out the high and low scores. then, for each judge, they would have their teams ranked 1-8, and if you add the ranks up across the judges, and take the lowest score, you have your winner, hopefully eliminating variance in scoring. as it stands now, it is a broken system.

 

[walkertexasjr]

I was just flipping through some of the sheets - did anyone else catch the 9 out of 5 AEG got for their stunts/gimmicks? I guess someone really loved their three-man tower ;P

best part is, it was factored into that judges final score - read carefully! Imagine if that was on VCU's sheet - we'd have an uprising

 

[Saleem]

Kinnell I commend you on bringing stats and quant into this.. It's helpful to have these z scores (arent they really t scores?). So from this the most 'harsh' judge was Faizan, with the lowest z-score. I think an interesting takeaway is that [given the low DoF] there is a 30% chance that such an extreme score was completely by chance, and that Faizan was a completely average-harshness judge*

* - b/c i know only kinnell is interested in where i got this # lol: http://stattrek.com/Tables/t.aspx 2-tailed t-test with n= 6 therefore dof = 5.

 

[yraparla]

there has to be a viable mathematical system from this setup, like sulman said, the ideal is that each team has a judge for all the performances (or maybe 7/8 performances, I dunno). But I just feel like every judging system has the same issues as this, just with different degrees of obviousness.

What about doing the std dev based on each judges score and taking the relative points and then std dev each teams score based on that and adjusting for that.

Essentially we have vertical and horizontal discrepancy: if a judge has a larger range, he will have a greater impact on teams if we just use adjusted scores by judge. Conversely if we just adjust per team a judge with a higher mean will have undue impact, this is pretty intuitive and is usually accounted for by dropping highest and lowest.

What if we did both, just adjust ranges to be in terms of std dev, drop highest lowest, redo the std dev and alot points based on that. Will it be a bitch? yes, can we set up an excel sheet to do it ahead of time, absolutely.

Judges will always have this issue of variance and offset....judges talking will help consistency but is that really all that great? Isn't part of the point that each judge brings a unique perspective? We should find ways to create comparisons in the math and let the judges do their own thing. I'm not saying they shouldn't figure out consistent judging guidelines or how to interpret a rubric, I'm sayin there's no way that would get all judges in an 80-100 range....you'll always have a faizan judge, one that is not necessarily harsh but just using different scale compared to other judges, it's best for the circuit if we can figure something out that can mathematically and statistically adjust everything somewhat fairly


edit: I have 2 midterms today and 3 more in next two but then I have spring break. I'll sit down with the spreadsheet at some point and actually create the formulas and macros, cause it's bullshit to keep having arguments. We're brown, too many of us are too good at math for this shit to keep happening

 

[Badwal]

Anyone else lost just like I am? All this does is bring back Data Management and Advanced Function nightmares back...