$$$$$$$ COMPREHENSIVE JUDGING SYSTEM $$$$$$$$

[Sue Sylvester]

Not to offend you or Faizan or anyone else, but this is 37 pages of the same scores just organized differently into charts and graphs and matchups etc.

Personally, I found some benefit in:
- Being able to see the teams ranked by overall points and seeing how many points they were from the winner
- How each judge ranked each team by overall points
- To see which categories each team excelled in and have teams ranked in categories (to see which team, according to the judges, had the best opening, or the best formations, or the best synchronization)
- How upper level and lower level judges compared to each other in terms of points for each team
- How each judge scored each category relative to the other judges in their rubric (whether more favorably or less favorably)
- Who each judge thought was better in each of the five match-ups and by what degree (there were some disagreements for sure)
- How match-ups were affected by lower level judging vs. upper level judging
- How close each match-up was and how each match-up fared against each other

Sorry, you didn't see the benefit of this and felt the need to falsely belittle it. Personally, I would have loved this type of insight last year in our match-up against BK and to see how each judge scored us, how we did in each category relative to the rest of the teams and what we could have improved on according to what judges.

WOW! Didn't even notice that. Something is fishy there. VanCity and AG are two of the better teams in the circuit and definitely this competition. No way they were 7th and 8th.

Remember that it's always wise to watch the performances before judging it based on pre-conceived notions. VanCity and AG had excellent performances, but it's also important to remember my earlier point that the judge in question had a specific rubric and specific criteria. The judge could have had appreciated various other aspects of those teams' performances (hypothetically speaking), but been unable to transcribe that opinion onto the rubric because those criteria were not present. Just food for thought - everything may not be as black and yellow as you may think it is or as you know what it is. Everything I do, I do it big.

 

[aneeshsahni]

It's cool to have all of these different ways to look @ the scores.


Something that stood out though was the disagreement between judges of the same level (lower or upper) on scores (see page 17)


WSA: Danny scored 10th overall, and Karmjot scored 3rd
UVA: Arjun scored 4th; Avneet scored 9th
VanCity: Danny scored 2nd; Karmjot scored 8th


Those are some big discrepancies....either judges need to be more on the same page or the rubric needs to offer more specific guidance.


For next year, assuming you're going to continue to have diff judges score different elements, I think you should make sure that the questions for upper/lower judges are the same if they're judging the same thing. For example, see differences in questions about ending on judges scoresheets. Were those intentional differences?

 

[jattkharku]

SUE, pehli gal, muh sambahleh aapni, aakard naal boldi/likhdi. Mein chajj naal gal karda si.

You said

, but the judging packet wasn't meant to find the cure to cancer or teach you how to dougie.

The original poster said this.

This in my mind should be the new standard in judging for the circuit.

I was responding to this and just pointing out that this was nothing but a regular rubric. I could appreciate being able to see differentials in the categories but this still doesn't fix the judging, which is what the original poster seemed to insinuate this might do.

Remember that it's always wise to watch the performances before judging it based on pre-conceived notions. VanCity and AG had excellent performances, but it's also important to remember my earlier point that the judge in question had a specific rubric and specific criteria.

I saw most of the performances. Not 7th or 8th, especially in this comp!

You claim that this packet means nothing, but I find it amusing that you would have never come your conclusion about Cornell without this packet making that information so freaking easy to access.

Someone else pointed it out. Then I checked the first score sheet only. I am totally with releasing score sheets for everyone to see and glad wbbc did it. But breaking the same thing down into over 30 pages and trying to sell it as some CJS analysis that the whole circuit should use is deceiving.

The original post just seemed like it was saying, "we got it right". My response was just a disagreement. I am done with this thread.


p.s.

Everything I do, I do it big.

Tenu Punjabi'ch kehnde, Hankaari.

 

[J-Skeet]

\

The original post just seemed like it was saying, "we got it right". My response was just a disagreement. I am done with this thread.


p.s.

Everything I do, I do it big.

Tenu Punjabi'ch kehnde, Hankaari.

 

[hardeep_singh]

SUE, pehli gal, muh sambahleh aapni, aakard naal boldi/likhdi. Mein chajj naal gal karda si.

You said

, but the judging packet wasn't meant to find the cure to cancer or teach you how to dougie.

The original poster said this.

This in my mind should be the new standard in judging for the circuit.

I was responding to this and just pointing out that this was nothing but a regular rubric. I could appreciate being able to see differentials in the categories but this still doesn't fix the judging, which is what the original poster seemed to insinuate this might do.

Remember that it's always wise to watch the performances before judging it based on pre-conceived notions. VanCity and AG had excellent performances, but it's also important to remember my earlier point that the judge in question had a specific rubric and specific criteria.

I saw most of the performances. Not 7th or 8th, especially in this comp!

You claim that this packet means nothing, but I find it amusing that you would have never come your conclusion about Cornell without this packet making that information so freaking easy to access.

Someone else pointed it out. Then I checked the first score sheet only. I am totally with releasing score sheets for everyone to see and glad wbbc did it. But breaking the same thing down into over 30 pages and trying to sell it as some CJS analysis that the whole circuit should use is deceiving.

The original post just seemed like it was saying, "we got it right". My response was just a disagreement. I am done with this thread.


p.s.

Everything I do, I do it big.

Tenu Punjabi'ch kehnde, Hankaari.

lol this is cute, an anonymous person is butthurt about placings...ANONYMOUSLY!!! maybe i should give you some karma too, cuz obviously you're such a brave soul to post on btf anonymously for the good of bhangra! i apologize on behalf of faizan, he should have been aware of the fact that the common jatt "kharku" isn't educated enough to understand the value of statistical analysis. sorry brah, i only understand punjabi when it's written in gurmukhi.

 

[faizan]

ohr tenu Angrezi'ch kehnde, TROLL.

 

[Aka$h]

Didn't go through everything but it does seem off that one of the judges score is +2 Standard Deviations away while all others are within ~1 St Dev, and guess what team it was for...

 

[siddyp]

After reading through the whole packet and looking at the scores, here's what's to take away:

it shows WHY Cornell won. It shows WHY every other team got the placing that they got. Thank you very much for doing this as it takes away all debate on this forum as to WHY a team won or a team didn't.

Does this mean judging will be perfect? no. It probably will never be. Because even at Elite 8 2011, when to me it was without question that SGPD destroyed everyone, people still came on this forum and thought PCBCA should have won.

And don't get me wrong, this is OKAY. Everyone should have different views and opinions otherwise we'd never innovate and teams would be all the same.

But this packet, and any competition that releases scores is being transparent and showing WHY a team won and another team didn't.

What I think can be improved is the display of scores, scores by category, ranks by category and anything with a BIG spreadsheet where one had to scroll back and forth and keep matching up columns and rows so they could compare teams.

Attached is a picture of what we did at MLIB, DL the full packet here: http://bhangrateamsforum.com/discuss/main-bhangra-discussion/my-life-is-bhangra-judging-workbook/

The way this was displayed IMO was easier to do team comparisons and how they ranked in comparison to each other in each category.

But you guys took it to a whole new level which I applaud.

Great work

 

[RsKderpu]

Jattkharku, just stop. You're making yourself look like an idiot by attacking someone who is just trying to be helpful and clear up some questions/thoughts......Sala jattkharku da lagda.

Now, let's just get one thing clear; this thread is to show how teams were scored and what the judges believed was a clean, well executed performance.

With that being said, no matter how perfect the judge is, he or she will always be question for minor detail. Keep in mind they make their decision based off of what they see in front of them. There's no replay button or something that can assist their judgment, so cut the judges some slack and appreciate what is being provided.

Faizan and others, thank you for providing all this information. Im sure many appreciate it.

 

[amancheema]

 

[Sue Sylvester]

SUE, pehli gal, muh sambahleh aapni, aakard naal boldi/likhdi. Mein chajj naal gal karda si.

p.s.

Everything I do, I do it big.

Tenu Punjabi'ch kehnde, Hankaari.

jattkartku, su karey che? SUE karey che!

But you're right. For you, I'll refrain from throwing Wiz Khalifa lyrics into my replies. I can be your best friend and you be my homie.

The original poster said this.

This in my mind should be the new standard in judging for the circuit.

I was responding to this and just pointing out that this was nothing but a regular rubric. I could appreciate being able to see differentials in the categories but this still doesn't fix the judging, which is what the original poster seemed to insinuate this might do.

It's nice to see people come out and admit that they're wrong and that they misunderstood the original post. Good on you! Faizan clearly indicates that there are multiple parts to his system - qualification of judges, rubric, video chat sessions, feedback, analysis..etc. No one insinuates that judging has been fixed, simply that we can get better results, better feedback and overall, a better experience for teams, judges, competition organizers and the circuit if more competitions can incorporate more of these elements. The analysis is merely a component of the whole picture.

 

[SGupta7]

After reading through the whole packet and looking at the scores, here's what's to take away:

it shows WHY Cornell won. It shows WHY every other team got the placing that they got. Thank you very much for doing this as it takes away all debate on this forum as to WHY a team won or a team didn't.

Does this mean judging will be perfect? no. It probably will never be. Because even at Elite 8 2011, when to me it was without question that SGPD destroyed everyone, people still came on this forum and thought PCBCA should have won.

And don't get me wrong, this is OKAY. Everyone should have different views and opinions otherwise we'd never innovate and teams would be all the same.

But this packet, and any competition that releases scores is being transparent and showing WHY a team won and another team didn't.

What I think can be improved is the display of scores, scores by category, ranks by category and anything with a BIG spreadsheet where one had to scroll back and forth and keep matching up columns and rows so they could compare teams.

Attached is a picture of what we did at MLIB, DL the full packet here: http://bhangrateamsforum.com/discuss/main-bhangra-discussion/my-life-is-bhangra-judging-workbook/

The way this was displayed IMO was easier to do team comparisons and how they ranked in comparison to each other in each category.

But you guys took it to a whole new level which I applaud.

Great work

Sid, thanks for bringing some wisdom to the thread and for actually understanding the use of the packet.


On a sidenote, big shout out to the judges who made it a point to answer any questions about the packet. I was lucky enough to catch Kinnell as he was leaving the hotel on Sunday morning and he took some time to explain some of the point deductions and how CMU bhangra could avoid issues in the future, which I really appreciated. Regardless of what this packet may or may not do, the judges and organizers at WBBC were extremely helpful in explaining the scores.


In my opinion, this packet is should be used to supplement the post-performance captain's meeting and the actual handwritten scoresheets that were provided by each judge.

 

[Navi Sandhu]

I really like the idea of normalizing the scores to set the "average" for the competition. Great idea.

 

[faizan]

BUMP!!!

It's competition season again and I would like to just leave this here for other organizers to review. If you choose to adopt the CJS, please contact me.

This year WBBC will again be employing CJS and will be adding INSTANT REPLAY for judge's deliberation. i.e., judges will get to see the sets of the top two teams over before they make their decision.

 

[sammyk]

BUMP!!!

It's competition season again and I would like to just leave this here for other organizers to review. If you choose to adopt the CJS, please contact me.

This year WBBC will again be employing CJS and will be adding INSTANT REPLAY for judge's deliberation. i.e., judges will get to see the sets of the top two teams over before they make their decision.

Instant replay would be great for those really close calls. You guys just keep adding cool features to the competition. It is really exciting to see. Great job!

 

[Danny]

This year WBBC will again be employing CJS and will be adding INSTANT REPLAY for judge's deliberation. i.e., judges will get to see the sets of the top two teams over before they make their decision.

Curious to see how this will go. Faizan, I hope you really think this through brother lol.

Honest feedback: Last years judging deliberation wasn't bad but it sure as heck wasn't great either. First, get a room with a damn door for the judges to deliberate. The whole having it in the lobby thing was a bit sketch lol and it was also pretty rushed IMHO. Someone was always snapping their fingers saying "Hey mickey singh is done singing we can't hold off any longer we need to hurry this up!"

Now to have instant replay in that experience would've thrown a serious wrench in everything. I'm trying to picture last years heated yet civil debate between judges then implementing this. I visualize it as:

Judge: "Hey that one guy on that one team messed up that one song where they did that one thing!"

Now someone has to whip out a laptop..............scrub through film footage to find that part.............still looking for it..........(more time wasted).............AH FOUND IT! ..............now what?

Have a team of 4-5 guys JUST for replays, give each on a laptop and a notepad for timestamps, do a practice run with your judges, idk you guys will figure it out. [SIZE=small]Just make sure its done right boys. I love WBBC because they are open to feedback and use constructive criticism very well making them an overnight Tier 1 competition. [/SIZE][SIZE=small]I'm confident you guys will iron out the details and make it work somehow.[/SIZE]

[SIZE=small]Who knows, after WBBC 2015, Instant Replay might become a staple in Bhangra [/SIZE]

 

[faizan]

Danny,

thanks for your feedback...

First, absolutely agreed, we need a PRIVATE ROOM for deliberations. Choosing a secluded area in the lobby did not go well, at all! lol. Rest assured this year we will be deliberating in a private room, with ZERO access for non-judges or non-board members. YOU RIGHT, YOU RIGHT.

Second, as for the instant replay: no pauses, no forwards, no rewinds. Just watch the set all the way, and then watch the other set all the way. We aren't there to nit pick mistakes and mishaps, just watch it and refresh yourself. I may even mandate silence during both viewings. Things happen fast in sets, if judges talk, it could distract.