Judging at Competitions

[mafzal]

Neil that's an awesome idea. It's like instant replay sort of. I think judges should be able to see 1-2 minutes replayed from a team's performance. This allows them to refresh their memories, and make the right decisions, AND its time sensitive, so that the deliberation ends up being like 15-20 minutes (normal time).

I think it's a good idea

 

[jvirk]

i agree nikhil

 

[yraparla]

INSTANT REPLAY
the instant replay is kind of difficult to handle because which 1-2 minutes are you going to see? Also most competitions' cameras are not going to be ready to replay so you'll need separate cameras with which to record, adding to cost and operations as it's just one more thing that the organizers need to worry about.

It's not a bad idea or even infeasible, it's just something that really really needs to be fleshed out in terms of implementation and planning. Really needs to be made clear what the purpose of the replay is and how it fits in with judging.

For instance, each judge during the performance gets to 'mark' ten seconds of time they want to review if they wrote something down and missed something, or want to review a 'traditional' step or w/e. then with 5 judges that's about a minute of review for something SPECIFIC. If you leave it general that's a lot of power in the hands of the guy cutting up the video or fastforwarding the tape.

TEAM INTERACTION WITH JUDGES
I know that one thing I wanted to do for the BIB08 website was to set up a forum where teams could interact with each other and the judges. That is, teams could air any questions/concerns/comments on the forum, in a public place where everyone could see what was being asked (Our teams didnt need it, but a lot of teams need help with general Vardiyan assembly, so this would be a good place to ask for help/who would be available day of, etc). This could also have been a good area for teams to ask questions of judges, etc. as most of our judges were pretty young and I assume could handle logging in a couple times before the competition. Its not a perfect solution, but I think we just need to increase moderated communication rather than come up with a silver bullet

I couldn't get the forum up in time because it wasn't the highest priority, but I talked to a few team captains about it and they seemed pretty interested in it. Again, its not a stand-alone solution, but a facilitator for understanding between teams and judges.

Truthfully, I'm not entirely convinced myself, because increasing interaction before a competition only increases chance for bias if a judge doesnt like the questions/tone/whatever. Thoughts?

 

[manmeet]

are u serious dude? u still bugging about the Boston thing? First of all empire had more stunts and an introduction, second they had crazy formations, and thirdly they had a bigger variety of moves. So quit tripping about what happened and move on with your life.

 

[Don]

First of all empire had more stunts

ok i'll take the rest of ur arguments as being valid but come on really??? i'm not saying that empire should not have won but just because a team has more stunts doesn't mean they should win. bhangra is not about who can do more stunts

 

[manmeet]

true stunts arent everything but they do count. and vcu had not many of them and formations. im not saying that they're not a good team. im just saying that empire was better

 

[Manjot]

yeh like swi said instant replay would be hard...maybe the judges should do like after the 1st team performs...they put them in the middle...2nd team performs and they should compare it to the 1st team that perform and put them on top or bottom of it...3rd team you compare it 1st and 2nd...etc..

with points if the first team does like good stunts or w/e the category is..and the judges give 10/10 to the 1st team in that category and then the 2nd team does even better stunts but they will still get 10/10 right? so maybe the judges should always leave room for the next team to do better rather then having ties and 1 team coming out in front with 1 point because they did better 1 category but both team got max points but werent as good as in another category.. i hope im making sense lol

 

[mafzal]

with points if the first team does like good stunts or w/e the category is..and the judges give 10/10 to the 1st team in that category and then the 2nd team does even better stunts but they will still get 10/10 right? so maybe the judges should always leave room for the next team to do better rather then having ties and 1 team coming out in front with 1 point because they did better 1 category but both team got max points but werent as good as in another category.. i hope im making sense lol

No, no no all wrong

somebody help him please

 

[Manjot]

shutup mariam! lol

 

[mafzal]

yeh like swi said instant replay would be hard...maybe the judges should do like after the 1st team performs...they put them in the middle...2nd team performs and they should compare it to the 1st team that perform and put them on top or bottom of it...3rd team you compare it 1st and 2nd...etc..

with points if the first team does like good stunts or w/e the category is..and the judges give 10/10 to the 1st team in that category and then the 2nd team does even better stunts but they will still get 10/10 right? so maybe the judges should always leave room for the next team to do better rather then having ties and 1 team coming out in front with 1 point because they did better 1 category but both team got max points but werent as good as in another category.. i hope im making sense lol

This what he means if it helps anyone. I understand him now:

F0lk you: Team A and team B
F0lk you: Team A go first and do good formations which the judges like so the team gets 10/10 right
F0lk you: right?

F0lk you: Team B goes next and does even better formations which the judges like even better BUT the team is only gonna get 10/10
F0lk you: doesnt that make sense
F0lk you: even though 2nd teams' formations were better then 1st

1 x: should Both get 10/10 or, shud judges go back and make A only 9.5 ?

F0lk you: they should give like 8s
F0lk you: until they see somethin better
F0lk you: give it 9
F0lk you: doesnt matter if u dont get perfect
F0lk you: if u get first with 8s
F0lk you: ur better then rest of the teams
F0lk you: if u start givin 10s then wudnt there be more ties
F0lk you: ok and
F0lk you: after watchin both performances
F0lk you: judges think

F0lk you: Team A shud get 8...team B 10...but both get 10 cuz they already did it SO...Team A's stunts were better so Team A get 10 in stunts but Team A gets 9...in total Team A has 18 points and Team B has 19 points BUTTT if they would have given 10/10 in formations in the first place then it would have been 20 for A and 19 for B cuz they could only get 10 max
F0lk you: give lower scores first

F0lk you: if team A gets 8s and others teams dont compare to it
F0lk you: then other teams
F0lk you: shud get lower

So in the end even if highest score for, say, formations was 7/10... it still HIGHEST and therefore wins. Doesn't matter if you score 10/10, because everything in comparison and your total score doesn't matter. Your total score stacked against other total scores is what matters.

You're welcome, Manjot

 

[Manjot]

It just took me sometime to explain it to you but in the end I WIN! ;D hahah jk
thnx Mariam

 

[mafzal]

Note: His system virtually eliminates ties, too

 

[yraparla]

Bad idea:

the point of rubrics is to eliminate comparison subjectivity. It's to judge each performance individually for its own merits, and then to compare them at the end. By comparing in between performances, you bring back all the issues of placing: A team that throws a sick formation may be hindered by another team throwing an equally difficult and impressive formation.

The issue is that rubrics/points diffuse the holistic view of the performance, while granting judges more leeway leads to subjectivity and human tendencies. It's best I feel to have a systematic and structured filter process that picks the top 3 and either a 1 or 2 wildcards (I realize this would mean half the teams or more at many competitions). THEN you go back and subjectively look the teams in the framework of the rubric/placings.

That is, we see that so and so team is 4, but after viewing ALL the performances we feel they really had the best energy of the night, and adding some points would put them over the top. Judges vote based on whether the RUBRIC should put them over the top and a super majority wins.

-----------------------------------------------------------------------------------------
Complicated? Put simply. At the end, the judges review the top half and vote on whether they followed the results match the rubric well or not. If they can find justification for changing placings, i.e. we probably ranked them lower in formations then some of the teams that got points, then they can take a group vote to change placing where supermajority (i.e. >=60%) decides.

Not a perfect system, but just an example of a better balance then doing this in between performances

 

[yraparla]

Also, I'm a huge backer of the organizers setting up someone knowledgeable with bhangra, but associated with the event, with the judges. Their role would be to keep pushing the judges to stick to the rubric and ensure that the process is fair to the teams so later it doesn't again become a judges vs. teams thing, the organization can back that the judges did what they were supposed to do and not let the judges take the flak.

This whole thread was started because people on teams felt that judges didn't follow the rubric. THE RUBRIC IS A PROMISE to the teams, and as much as people stressed they weren't attacking BBC, that doesn't change the fact that it's BBCs responsibility to make sure the rubric is used and followed (Same for any organization running an competition, and not that BBC didn't do this, it's just an example where something was questioned and could have been avoided)

 

[Boston Bhangra]

People will always question judgements at almost any competition where there is vast amount of talented teams. That is fine, and that is what forums are made for, so that people can talk through it. We go through our rubric with teams and the judges, and try to do our best to make sure that all of them understand it. We do have a BB member with the judges at all times to make sure there are no biases occurring to ensure that the best results are made. Our judging scoresheet indicates that we have 100 points allocated to the various categories for points, but also an A-F ranking of what the impact of a given team was on a judge. Points are just one way of denominating a winner, and at the end of the day, we want to make sure that all the judges come to a consensus on who they feel were the best teams. Points are just one way to come to that decision. Multiple teams were all in the race for receiving the top 3 spots. We were not going to designate a tie again this year, so we made it known to the judges that only 3 places would be awarded. Based on points, and discussing what all the judges thought were fair, the three top teams were awarded. It's unfortunate that we can't award multiple teams, multiple awards, but that is what a ranking system is made for. We thought we had fantastic performances at our competition, and that makes judging at our competition very difficult. Hard decisions have to be made, and unfortunately all the teams can't rank. If any teams have questions or concerns about what took place that day, you can very easily contact our organization. Just e-mail us @ info@bostonbhangra.com. We thought the judges did a good job, and we stand by their results.

 

[shareef85]

regardless of trying to make judging better in the future, I believe no team nor any single person on a team should publicly portray their personal views on judging or placing. We do this for fun and for spreading the culture NOT for first place, regardless of the circumstances. Just my opinion. Empire went last and blew the crowd away If you were in there you would have felt it. Also even tho you say your trying not to take anything away from Empire, what do you think you've done?
just my opinion

 

[Danny]

regardless of trying to make judging better in the future, I believe no team nor any single person on a team should publicly portray their personal views on judging or placing. We do this for fun and for spreading the culture NOT for first place, regardless of the circumstances. Just my opinion. Empire went last and blew the crowd away If you were in there you would have felt it. Also even tho you say your trying not to take anything away from Empire, what do you think you've done?
just my opinion

Ouch. He's right. Couldn't agree more with Osman.

 

[Danny]

whatever happened to having good, plain fun?

 

[Surmeet1]

are u serious dude? u still bugging about the Boston thing? First of all empire had more stunts and an introduction, second they had crazy formations, and thirdly they had a bigger variety of moves. So quit tripping about what happened and move on with your life.

i dont think they had a bigger variety of moves. but yes, lots of stunts, and some formations. but not a bigger variety of moves.


regardless, good job empire, esp meistro and ashveer

 

[Saab]

If anything I think the some of the biggest things to be gleaned are that the rubrics are ostentatious in their ability to attract teams attentions with categories that are defined but more-or-less undeveloped(no structure). If we are, per say, going to nitpick with the "traditional" category we see that the category could be as far-flung as doing bhangra to actual individual incorporated segments in the dance. I haven't had time to look at BOB rubric, but I think if we had written language on what should be in a bhangra dance(maybe a checklist of say...ten points) and the other ten to be given at the discretion of the judges, then that can better mediate how dances are judged and what judges are looking for. Also subject of comparison comes up within all judging rubrics. I for one agree with Swi and think each dance should be judged individually then compared at the end. While traditional elements could be checked off, as with prop segments, scores could be given to formations that should be subject to a weight distribution, something I think that has already been discuss. At the end when the judges distribute their tallies they should engage in discussion as to placing teams based on non-traditional aspects(formations, stunts, "wow"-factor) and then give them an appropriate total. Personally, I think that the Traditional category would be better off divided into two equally important entities:Segments and Choreo. The Segment part would get a checklisted amount whereas the choreo amount would fall under the weight system, thereby saying that if you have this in your dance, your good, but if you want to separate yourself, please show some individuality or flair in your routine. Other than that, I like the idea of videotaping(If your going to videotape your own show, you might as well let the judges use some of the film) and I really think that above all else, competitions and judges together should make the rubric as to provide an accurate document that well represents the feelings of the audience and the tradition of bhangra.