*** NEW Venture- Judging Database ***

[yraparla]

Hi Guys, this is long. But please read!!!

There have been a few proposals floating throughout the years about how to standardize and raise the level of competition organization. I have my own plan that I've shared with many people, and while I'm not ready to share that plan publicly, but I AM ready to implement the first part of that plan. A database of judges!

The goal of this database is to provide a comprehensive profile of judges so that competitions have the ability to create a unique lineup of judges that matches their spirit and rubric. The first component is quantity, giving competitions a chance to avoid over-worked judges (and correspondingly providing teams with fresh insight). The second component is providing them a profile so that competitions can pick judges who match their style of competition and their objectives. It's not just traditional vs. modern anymore...the sliding scale for judges is much more complex. Our goal is to describe and present the judge in a way that a competition and teams can reasonably understand how they will judge and not be surprised by results.


What I'm looking for is feedback, from both competition organizers and team captains. Ultimately the goal is to reduce the irritation with placings. The very nature of ranking teams means disputes will always occur, our goal is to facilitate judging lineups where the results is at least respected when disagreed with. There's a difference, and I'm hoping we can bring the circuit back to a more constructively-competitive atmosphere. If we can't respect the judges, we won't respect the placings, it's that simple.


Currently here is the overall plan:

• The database will mostly include everyone who applies to be on it. The threshold is legitimate Bhangra experience. Although we may eventually screen by quality, initially the goal is to make the profiles and let organizers make the decision on if a judge is qualified or not. This is the quantity aspect I mentioned.

• We will however feature certain judges of higher quality and experience. Similar to a review magazine that has an "editor's choice" label. These are the judges we subjectively think will do a good job and fill the role well. While previous judging experience is preferred, it won't be necessary as we want newer judges to also get opportunity. So based on the discussions we have with them, their profile, and their experience, we will try to feature current and future judging stars.

• All applicants will be required to provide a small description of their experiences in bhangra, why they're qualified to judge, and what their view of bhangra is. This is the initial application and filter.

• All applicants will also be required to complete a profile analysis. The main component of this will be watching a video and completing a rubric. The rubric will give insight into the ability of the judge to critique, analyze, comment, and absorb a performance. While judging a video is nothing like judging live, it is the easiest most practical way to build the initial profile.

• When a judge finishes at a comp, we will feature their rubrics/comments if available (and more competitions are doing it these days). This is to further create a representative profile of them for teams and organizers to look at.

• The database will also track the judging issues that come up at competitions. This includes verified comments a judge may have made (in context of course), as well as competitions who are not transparent with their process. This is to deter similar actions in the future as well as catalog the problems so future teams know what they're getting into (i.e. don't complain, it's your risk)

The goal is to hold judges and organizers accountable publicly. It also gives teams an opportunity to see what their judges are like well-ahead of the judging meeting. This means teams should know what to expect, and that reduces confusion and frustration.


Some questions to consider:

• How do we do the video-rubric test? Should it be the same video for all judges, should we do multiple videos? Should we time it so that they are forced to do it in real-time? Should we do multiple related videos so there's a comparison?

• Should everyone get access to the database, or just competition organizers and captains or something. The information should be available, but I'm not sure if I want it all publicly available, I'd prefer a libraryish system where you can request for information (if you're not already authorized as an organizer). This is to balance exposure vs. privacy. Still transparent and available, but just a barrier so that only people who have legitimate use for the info get it. (would also limit people from seeing rubrics and trying to leverage that in their own profiles). Just a thought.

• What about organizer-only content, to limit the exposure of judges but still give plenty of info to organizers. Over-exposed judges can cause headaches for organizers as well, since teams often times complain in advance without merit (Bruin has seen this I believe, I experienced some of it with Sin City). The goal is balance. The public has all necessary info, and we can give organizers just a bit more, such as a video of the judges explaining their rubric (to get a feel for their interpersonal skills).

I'm ready to go through with this. What I'm looking from the community is feedback. Instead of bitching about judges constantly, please take 5 min to read the idea and let me know what you'd like to see. What would you do differently, or add to the process? I'm always open to feedback, and even if we can't do something right away, we can always move to implement ideas in the future. We have to start somewhere first, and eventually we can clean up these issues. The database is for teams and organizers, so please help me accomplish the goals!


Thanks,
Swi

 

[Kartik]

This is a great idea Swi - glad to see some proactive steps towards dealing with one of the biggest concerns of the past year. I think having judges openly discuss their viewpoints would be a crucial aspect of the database - something that competition organizers and teams could both benefit from. This would greatly help improve some of the communication, or lack thereof, between captains and judges.

I think if this is implemented well it could go a long way in justifying competitions - if a competition selects judges outside of the database it won't be a credible move. Also, some of the shifts in judging standards will be addressed since judges will be expected to hold true to their public comments regarding their paradigm.

Definitely a step in the right direction - I'm very interested to see how this pans out. Would be very willing to give constructive feedback as the process continues.

 

[vmural2]

Great idea Swi. A few questions I had after reading your post:

If full transparency is the overall goal, then where does the issue of privacy play in? In other words, if the point of this database is to bring accountability to judging, then what would the downside to having it being publicly accessible? Judging information is what a lot of teams look into before applying to competition. If I were a team captain, or even just a dancer, knowing what judges look for in a routine, or seeing how they critique certain teams, is information that I would find very valuable.

I'm also curious as to how this information would be presented. Would there be another section opened on BTF solely devoted to judging? Or would it only be made available via requests? The latter may be a lot more work for the moderator in terms of the sheer number of requests, and having to validate all of them.

 

[yraparla]

Great idea Swi. A few questions I had after reading your post:

If full transparency is the overall goal, then where does the issue of privacy play in? In other words, if the point of this database is to bring accountability to judging, then what would the downside to having it being publicly accessible? Judging information is what a lot of teams look into before applying to competition. If I were a team captain, or even just a dancer, knowing what judges look for in a routine, or seeing how they critique certain teams, is information that I would find very valuable.

I'm also curious as to how this information would be presented. Would there be another section opened on BTF solely devoted to judging? Or would it only be made available via requests? The latter may be a lot more work for the moderator in terms of the sheer number of requests, and having to validate all of them.

Great questions,

The issue of privacy is not really privacy, in the sense of sensitive personal material. What I really mean is that suppose a judge posts a scathing review of a team...we want to avoid awkward situations that could escalate. It should still be available to organizers and to the team captains. But I'm not sure EVERYONE needs to see it, I think the team should also be able to have some influence on what goes public. We will never hide anything, good or bad, but in terms of public access without a legit request, I don't know that full access is necessary.


It's not a critical component of the product, but it is something I wanted to put out there. I know some teams want full transparency, some prefer their scoresheets and comments private. Want to make sure we can fairly accommodate both.

It will be a website, regardless if we do it by request or not. I have to still work out the details with saleem, but it will not be a direct part of BTF. I think it needs to be featured here, as it really is for the community. But I envision it as an independent operation, with a partnership with BTF (i.e. getting a link, and possibly a btf url like the CTL, we'll see).

 

[Govind]

Excellent idea. I especially like the fact you will feature some new, upcoming judges...I would like to see more new people join panels. I do think that it is important however to have a good mix of old/new b/c there is a lot of insight to be passed down regarding the judging process.

I am bringing a new judge on board, who will be making his debut with me VERY soon. We are very lucky to have him in our midst! ...announcement should be made soon.

 

[vmural2]

@Swi
That makes sense. I can see how that sort of thing might be an issue, and in a sense, I guess it does fall into the category of privacy. It seems like what you are trying to provide is a resource for competition organizers ONLY. This is not necessarily a bad thing. It addresses the concerns you had about teams and judges being put in a public spotlight when score sheets are released. Although a few more questions do arise from this:

If ultimately you are providing a resource for organizers to use, one question I have, is how much control do you have over whether organizers use this resource wisely or not? If judges profiles are going to be private to comp coords only, if the coords themselves are a bunch of screwballs (as we have seen with comps before) and cannot pick proper judges to fit their competition needs, will this knowledge be released to the public if the judging does not occur as originally described by the comp?

How much access do the teams themselves have to the database? If access is restricted to competition organizers only, could there be a way where DB access is given to teams by the comp organizer upon a team's acceptance into the comp? (after which access is denied to the organizers and teams after the comp is over)

One more question that arises from the scenario you mentioned earlier about if a judge were to give a team a particularly bad review (which does happen on occasion); if this harsh review were made accessible to a comp organizer, and that same team that received the harsh review applies to the competition being organized, would that affect that teams acceptance into the competition?

 

[yraparla]

@Swi
That makes sense. I can see how that sort of thing might be an issue, and in a sense, I guess it does fall into the category of privacy. It seems like what you are trying to provide is a resource for competition organizers ONLY. This is not necessarily a bad thing. It addresses the concerns you had about teams and judges being put in a public spotlight when score sheets are released. Although a few more questions do arise from this:

If ultimately you are providing a resource for organizers to use, one question I have, is how much control do you have over whether organizers use this resource wisely or not? If judges profiles are going to be private to comp coords only, if the coords themselves are a bunch of screwballs (as we have seen with comps before) and cannot pick proper judges to fit their competition needs, will this knowledge be released to the public if the judging does not occur as originally described by the comp?

How much access do the teams themselves have to the database? If access is restricted to competition organizers only, could there be a way where DB access is given to teams by the comp organizer upon a team's acceptance into the comp? (after which access is denied to the organizers and teams after the comp is over)

One more question that arises from the scenario you mentioned earlier about if a judge were to give a team a particularly bad review (which does happen on occasion); if this harsh review were made accessible to a comp organizer, and that same team that received the harsh review applies to the competition being organized, would that affect that teams acceptance into the competition?

I never once said it was organizers only! In fact I specifically mention public access...

The best way I can describe it is that the primary function of this is to connect judges and organizers: New judges get an opportunity to have a profile and organizers can see them. Organizers get a singular location to find judges, with all the necessary info.

The secondary purpose is to hold organizers and judges accountable, by making their profiles and comments public. The scale and details of what becomes public are up for discussion, but it's not really important for the primary purpose of increasing the quality of judging. It's just a channel of communication and arbitration AFTER a comp ends that we should utilize.

I think when you look at it that way, your questions will answer themselves.

 

[LuckMC11]

The only issue I see is that with this method you are standardizing the judges (unless I am misunderstanding this part):

All applicants will also be required to complete a profile analysis. The main component of this will be watching a video and completing a rubric. The rubric will give insight into the ability of the judge to critique, analyze, comment, and absorb a performance. While judging a video is nothing like judging live, it is the easiest most practical way to build the initial profile.

With this I feel you are limiting the growth of the circuit if the judges are seen to judge all the same way, or at least based on what the rubric says determines a more "able" judge.

Thoughts?

 

[yraparla]

The only issue I see is that with this method you are standardizing the judges (unless I am misunderstanding this part):

All applicants will also be required to complete a profile analysis. The main component of this will be watching a video and completing a rubric. The rubric will give insight into the ability of the judge to critique, analyze, comment, and absorb a performance. While judging a video is nothing like judging live, it is the easiest most practical way to build the initial profile.

With this I feel you are limiting the growth of the circuit if the judges are seen to judge all the same way, or at least based on what the rubric says determines a more "able" judge.

Thoughts?

I think you are misunderstanding, the rubric is meant to create a profile. The organizers then determine the judges that best fit their competition. The judges that will be featured are based on quality of their material (i.e. ability to give good comments and provide coherent arguments for the points they gave). This will likely be determined by phone conversations or emails (I'm trying to have the database do the legwork for the competition who may not have bhangra people leading it).

But how you fill the rubric is not important, there's no 'right' answer, it's just about seeing how someone judges and what they emphasize, so that competitions and teams can understand better how a judge thinks.

 

[Kaur]

this is a great idea Swi. Let me know if you need any help im almost done the vardiyan update

 

[LuckMC11]

Oh ok I see what you mean--that's better!

Only thing I would see is that competitions could potentially discriminate against judges from what they had in the past to only have what they want--is that a positive thing or negative thing?

 

[yraparla]

Oh ok I see what you mean--that's better!

Only thing I would see is that competitions could potentially discriminate against judges from what they had in the past to only have what they want--is that a positive thing or negative thing?

Neither, it's how it should be. A competition has every right to pick judges they believe will judge along the lines of their competition style, we're actually encouraging that. The point is that teams should also have that information so they can understand what they're getting into.

 

[avtarchana]

Hi Swi !!

Thank you very much for posting this very senstive issue. According to my rich experience I can feel the real inner feelings of the budding artistes esp. when they had performed extremely well and are given low marks due to the undeserved 'Judges' planned by the organisors and the Judges are there just to 'ENJOY' the event.

I had already watched many 'Judges' in North America, who are just enjoying the event.. laughing... having cold drinks/eatables etc.. talking each other.. and not even watching the performance, properly.

Very Very Sad.

On the other hand, I never tried to sip a drink and not even tried blink my eyes the performance so that there should not be any loss from my point of view to any team.

According to me, there is no idea of having 5-6 or more judges in a Bhangra Championship. In a single row, one can not communicate after a performance and the organisors have the tendency to take the sheets after every performance and I really dislike that. According to me, there should be just 2 or 3 capable judges, so that they can discuss the situation after each performance and can be made accountable after the performances or at the time of results.

I know, when I was 'Judge' at the Bhangra Blast, Bruin Bhangra, Bayou Bhangra etc.. some teams came to me asking their faults. Since, there were 5-6 Judges and they gave their marks according to their capability.. I said 'NO' to comment as I have given maximum marks to a particular team and the other Judges might have given less, so their average came down and lost that Championship. I was really very very upset/sad on this part that why the deserving team is not the FIRST ? So, there is lack in the organisors to check the right Judges. I have also watched that during some functions, the organisors are bringing the music teachers who have no sense of dance, and that too of the Bhangra.

I hope that the organisors would try their BEST to bring the BEST JUDGES, not the quantity, but the quality wise, enabling to give much much JUSTICE to the capable teams in the future.

Hope to hear "VALUABLE COMMENTS" from the participants/organisors/judges n wishing you..

ALL THE BEST !!

-Avtar Chana/Chd.

 

[Neha]

Great idea Swi!

I think this database will also solve one of the biggest issues we have in the circuit currently, which is the lack of female judges. This point has been discussed at length in the past but the fact of the matter is that it's still an issue. As a female dancer and former captain of an all-girls team, it was really annoying at times to not have even one female judge at a competition when there clearly were so many experienced dancers out there that were qualified to judge at the time. After having multiple discussions over the years with fellow dancers, competition organizers, other judges, etc. it seemed to boil down to a few main issues:

1. Lack of knowledge of potential female judges - people just didn't know who to reach out to. In a recent conversation with a comp organizer regarding whether or not they had a female judge on their panel - the answer given was "We thought of asking Jasmine but she was too far and it would be really expensive so we just dropped it." There are so many female dancers, coaches, captains, etc. that exist that so many people are friends with in the circuit but for some reason people seem to just forget about them or not realize. Furthermore, when and if your first choice for a female judge is unavailable, shouldn't an effort be made to search for another one who is a plausible option just as you would if you needed more judges in general?

2. Male vs. Female - this debate is old and pathetic but sadly I have heard it so many times from fellow dancers and judges in the past and present. I have had my opinions and input disregarded so many times by people because they will think that the guy sitting next to me, solely for the reason that he is a guy, is a greater authority on Bhangra even though many of us girls have learned the same traditional moves, led teams on our own, and have proved ourselves time and time again as dancers, leaders, and teachers.


3. "It's not a big deal" - again. Another pathetic excuse. I don't understand how it is fair that girls get judged by male dancers and some even go to the extent of telling us or deciding how we should or should not dance, but girls can't judge them? If you have male and female dancers sharing the same stage and are at par with one another, then you need to have that represented within your judging panel as well. It is not to say that all-male judging panels are always bias or can't judge - they definitely can but you want to be able to be judged by a fair and representative panel of dancers in terms of skill, style, experience, and even gender.


I really hope that this database will shed light on the immensely talented male and female potential judges we have in the circuit. I stress the female judging part only because as the circuit progresses in terms of ideas, caliber of performances, etc. we need to start making it a norm for a female to be on a panel of judges and for competition organizers to think twice before they finalize their panel and for team captains to understand and support the importance of this issue.

 

[mafzal]

+500 Neha

There are plenty of well-qualified female judges out there and this database should help bring them to light and get more on the judging circuit by providing their credentials and contact info

Jasmine is great but seriously she's not the only qualified girl out there

 

[partik614]

Any update on this judging database?

 

[yraparla]

Honestly just backed up like crazy with work and other projects. We're also doing some exciting new things with the CTL / updating BTF so this is taking a backseat for now.


What I can do in the meantime is at least build up a list of judges with contact info so that we can get that going for future organizers. I'll set that up sometime next week.