9C.3: Pair Judging (Closed for comments)
Comments about this discussion:
Started
The rules stipulate that pair judging is preferred but gives no actual mention of how judges should be matched up or what to do if the two judges in a pair fundamentally disagree. Given the variation in age, that not everyone may be super proficient in English (for international competitions), that the judges are also competitors and similar factors it seems to be a place a lot of bias can creep in.
My preference would be to scrap the rule so that only solo judges are used (even if it sometimes may make sense to have younger riders shadow a judge to get them familiar with judging). If the rule is not scrapped it seems it could use some specification.
Comment
I do not think we should scrap the pairs judging. If we are going to have a competition openly beckoning freestylers and flatland riders, pairs judging is essential for a fairly judged competition.
I would make pairs judging the standard, and only mention that solo judging is acceptable when there are not enough participants. I would add to the clarification of pairs judging by having the Chief judge, who should be someone knowledgable of the riders and the differences between freestyle and flatland, to help split up the pairs for pairs judging within each group. I would add in the rulebook that it is preferred that flatland riders get paired up with freestyle riders for judging.
Right now, flatland judging is ultimately decided by a group discussion on who the winner is, and this seems to work out fine. And I have almost always done pairs judging when I run x-style competitions, and they have always come to a conclusion together. The best and most unbiased judging sheets I have seen come from pairs judging.
Comment
The issue of having flatland and freestyle riders competing against each other is no different than the issues faced in freestyle riding where you also have multiple styles (Japanese, Euro/America, flatland/hiphop, etc) compete. The strength of the sport is that there's no "gold standard" for how to run a routine and as such you get to see many different styles. The solution is the same as in freestyle, namely to make sure the judges have been exposed to a wide range of unicycling styles and to make sure everyone understands the intricacies of the different styles.
With that said pair judging or solo judging doesn't really matter as long as the judges are drawn from the participant pool. If you have a competition that's 70% freestyle you'll also get 70% freestyle judges – you'll then either have pairs only with freestyle judges or there'll be freestylers that won't judge as much as flatlanders. There's no problem that certain judges will favor certain routines (ultimately that's why there's at least 5 judges). Pairing people up mostly just ensures you won't see the full diversity of opinions on the judging table.
~~
Also, the chief judge needs to be extremely careful in how to make the pairs. Not just pairing up freestylers and flatlanders but also in making sure that each judge in each pair will have an equal voice – e.g. by considering if they're from the same club or one is much older or one is much better ranked or similar. Another issue is that for any competition with riders speaking different languages (ie, unicon or any international competition) it can be challenging to actually get a debate going with your co-judge if both aren't super proficient in English (or share another language) – this is again something that slightly favors the voices of certain judges over certain other judges.
Comment
I think that the difference between flatland and freestyle styles is a lot different than the differences between freestyle styles. Even though the two disciplines are becoming closer now, there is still a large disparity between the types of tricks being displayed.
Because of this disparity, judging in pairs, specifically one freestyler and one flat rider, gives a more comprehensive approach when ranking participants than just a solo judge who has no way of comparing a freestyle trick to a flatland trick by themselves. I don't think pairing up the judges will limit the diversity of opinions, I think it will enhance them and lead to a more informed choice when ranking participants.
Yes, the Chief judge does need to be careful, but there will be issues with both individual judges and pairs judges that will be similar. The differences are negligible, and I think the benefits of pairs judging outweigh potential biases that are going to be impossible to completely control anyhow.
Comment
Danielle,
You specifically have not responded to my points about judges who don't share a language or where one is much better at English than the other. That's something that'll be prevalent in all international competitions and even a bunch of national competitions.
Comment
I already judged solo and paired up and i highly prefer to judge in pairs. It just gives you another view as well. But I would write down how to pair the judges that mostly works out itself. If a pair fundamentally disagree i only see two options. Either you let them both hand in their own ranking or this pair won't hand in a ranking so there's one ranking less.
I never had problems with the language cause we always found a way to comunicate but i can see a problem there. An idea for that would be to make the ability to communicate in english a criteria for judges for international events.
Comment
How is the issue of language solved in other competitions? Could those solutions be applied to the issue at hand?
I guess my question is whether or not the issue of language when pairs judging is any different than issues arising when describing the rules of the event to a solo judge.
Comment
I think that judging should be compulsory for all riders. In that way, it is ensured that the pool of judges corresponds to the pool of riders.
As stated in the other discussion, I would like to enforce this stronger as many people just did not do their judging responsibilities at the last UNICON. Maybe disqualification would stop people from leaving the competition and signing up for events that happen at the same time in the first place.
I think that having more individual judges instead of pairs would also do balancing of different styles. Although I personally like pair judging, it has some problems such as one judge being dominant, language barriers, etc.
Comment
I agree with Marie. I think that all riders competing in X-Style should be required to judge.
I vote we add "Each rider competing in X-Style must be on the Judging Table. Failure to participate in the Judging Table will lead to disqualification from the X-Style competition." under rule 9C.3 Judging Table.
Comment
Do you want to have any age minimums for the judges?
Do you want an 8-year-old judging?
Comment
I think one should be a least a senior to be a judge (15+). That would in practice mean the seniors are asked to judge both the senior and the junior competitions. This would still present a problem if a junior chose to participate in the senior competition.
For international events we can't really expect paired judges to be younger than 18 unless they have the same mother tongue or one is EXTREMELY good at english in my opinion.
Comment
People don't magically approve there english skills a lot when they turn 18 so i don't see any difference in a 17 year old judging or an 18 year old judging. Also there are people way older that still haven't learnt english and never will.
I would suggest to find criterias a judge has to fullfil like a minimum age (i would maybe vote for 15), for international events the ability to communicate in english,...
In my opinion we shouldn't make it compulsory for all riders to judge and rather set a rule of how to nominate judges like in Freestyle.
"Parties (Countries/Clubs) that participate at competitions must nominate judges in relation to the number of Artistic Freestyle participants they send (see table below). After registration finishes, the chief judge will send a request to all parties. The request contains the compiled number of minimum judges per party and a question to nominate the candidates. Judge nominations include the experience of the judges (such as previous competitions, how long he/she has been judging, age group/expert judging or other relevant qualifications such as educations)."
X-Style needs more judges than Freestyle does so we could maybe say something like the number of judges that has to be nominated is the same number as senior riders signed up for the competition.
I already had the problem of two or more different events i signed up that took place at the same time. And i didn't sign up to other events just cause i don't like X-Style judging in fact i really like it. I feel it's wrong to push riders out of the event just because they also participate in other events.
Comment
I think we should be careful about requiring English. In order to judge, you must have a good understanding of the rules. That usually means understanding English because a reminder of the rules comes from the director before the event. But if you have a detailed understanding of the rules, and you are pair judging with another person with whom you share a language of communication, there is no problem. At the end of judging, you only need to submit your ranks. No English needed. Specifically many of our Asian participants at Unicon have less English proficiency than European riders. That are an important part of our competition, and notably they may have a different impression of how hard different tricks are due to differences in style.
There are many things that define a good judge and experience is one of those things. Age is an imperfect, but measurable, proxy for experience so that's an okay requirement in my opinion. Being able to speak and/or understand English well does not make someone a better judge.
Comment
Being able to speak English really doesn't make anyone a better judge but at least it makes it possible to attend the required judging workshop and if there are any major mistakes in ranking or a question about rules it also allows the chief judge to communicate and talk with that judge. I wouldn't feel comfortable being chief judge if i wouldn't be able to communicate with every single judge.
Comment
I think the discussion about English (or another shared language) is only relevant to pair judging. The point of pair judging is that two judges with different viewpoints can discuss the differences and come to an agreement about how to judge. If we keep pair judging but take note of the language barrier we'll have a number of pairs that have similar viewpoints (say, two from the same country), defeating the purpose of pairing up.
With that said I'm in personally in favor of changing the rules so it's more aligned to freestyle and each team need to nominate judges (who would deliver their results individually).
Comment
I think that getting rid of pair judging would make it a lot simpler.
If you keep the same number of judges - there would be double the amount of scores, adequately reflecting what every single judge thinks (rather than a biased "averaged" score from two people of which one might be dominant) and, as I mentioned in the other discussion, maybe we could cross out the lowest and highest scores from the final score?
Comment
I think whether or not pairs or individual judging is used for the competition should be up to the discretion of the X-Style director, as this is a choice that can be made after seeing a list of participants. I do not think we should eliminate pairs judging, as it has worked well in the past and is beneficial to an event that is catering to a wider range of skills/styles. It facilitates better cooperation and more accurate results, in my opinion.
Taking away the highest and lowest score is already apart of scoring X-Style. See Rule 9C.4
-
The highest and the lowest placing points per rider are discarded. All the remaining placing points get summed up for each rider. The 3 riders with the fewest points win and advance to the next round.
Comment
I think this is a very important question and should not be left to the chief judge.
I like pair judging because I learn about how other people value certain tricks regarding difficulty - but it poses some problems (as stated above).
And I think that doubling the judging panel should also have the effect of averaging out. Obviously, the initial scores that are entered will vary more, but you could also increase the number of scores that would not count towards the final result.
Comment
I also still think that all competitors should be judges.
And I would even vote for letting the juniors judge themselves. They might not be able to give names to all the tricks they see (as there might be huge differences between riders) but they will certainly know if they can do the trick or a similar one, if other people they know can do the trick and how long they practiced for it, and thereby get an idea of difficulty.
If we start to introduce external judges - we will need to find criteria, have workshops for it, etc.
The whole idea of X-Style was to have the riders judge themselves by their own criteria - obviously we gave some guidelines in the rulebook.
We had originally observed that intuition of riders of who should be in the lead for the technical scores for freestyle competitions was not reflected by the judges scores. And we wanted to created something as simple as possible where riders would get the chance to judge themselves based on their own intuition.
Comment
I really wouldn't eliminate pair judging as stated it helps to learn how other people value the difficulty of tricks. I already learnt a lot pair judging and it changed my view on tricks as well. As you already said X-Style is also a competition for all kind of different styles and judging in pairs helps to bring them closer together and understand each other.
Comment
I'm happy to have pair judging in the rulebook - but as stated, it makes most sense to have pairs of people riding different styles (and it makes communication a lot more difficult). And I would really want to give the riders the opportunity to judge themselves. That's the main reason why I would vote for individual judging at this point in time.
If there is a way we can solve the communication issue (also for junior riders) - I would say let's go for pair judging.
Maybe it could be encouraged to judge with someone from a different country, but if the language skills are not sufficient, pairs of riders from the same country are allowed?
Comment
I am not in favor of changing the current judging setup. I think ALL riders should also judge and I think pair judging is easier and also more fun. It forces judges to justify their opinions to their partner and allows for more critical thinking.
Comment
I vote we add "Each rider competing in X-Style must be on the Judging Table. Failure to participate in the Judging Table will lead to disqualification from the X-Style competition." under rule 9C.3 Judging Table.
I think Juniors are capable of judging their peers. Either they will be judging each other in a junior expert setting where all riders are under a certain age or they will be competing with older riders who they can be paired up with.
I agree, I do not want to change the current judging system. I think it is working fine right now, and needs more time to be tested as is before anything changes. However, I believe we can make judging mandatory for all riders at this point in time.
Shall I make a proposal to make judging mandatory for all riders?
Comment
Yes.
Comment
I agree
Comment
I strongly disagree. At international events juniors will have a hard time judging and it's mendatory for them to find someone able to speak their language in most cases. Also there never existed a time schedule so you knew when you're competing and when you have to judge. X-Style seems more like a competition that takes away the whole day. You can't just walk away cause you never know when you have to judge next time. And that's a problem if you compete in multiple disciplines. Adding that rule will maybe help to keep some judges but also disqualify other riders that have different events going on as well.
Comment
As a previous organizer many times, there seems be an expectation from riders that they should be able to participate in absolutely every discipline. That's simply not possible at Unicon or ECU due to the number of riders and the length of the event. It would be vastly easier to organize over three weeks, but that would increase the costs and make it so fewer people could attend. Riders need to be aware of these constraints.
Having the riders be the judges in X-Style is inherent to its invention. This may mean that participation in an X-Style competitions takes longer than just your own run. A good event director should publish which riders are judging which groups, with an estimated schedule. It's not very hard to have an accurate schedule for an X-Style event.
Comment
Basically, I like the idea that each rider competing in X-Style must be on the Judging Table. But I think since it's the Chief Judge who finally makes the judging table, he should have the authority to decide if every starter really needs to judge. I would therefore formulate the rule with a little more freedom for the chief judge, in that way: "Every rider competing in X-Style must be available for the judging table. Non-participation at the judging table can lead to disqualification from the X-Style competition."
Maybe we can also add a sentence like: "The Cief Judge should publish before the competition the times when someone has to judge."
Comment
I disagree here – all expecting riders to judge is making riders think twice about competing and it's hard to imagine it leads to a higher quality of judging. I realize it may have been needed to bootstrap the competition but at this point it seems a bit misguided. I think riders just expect the best possible competition and the fairest possible judging – we're not providing them with either in my opinion.
Comment
I strongly agree with Scott here.
The idea of the competition is to have the riders judge themselves. Even though people might have different views of what is more "difficult", it ensures that the final ranking still shows what the riders think about difficulty and who has shown the most difficult tricks.
Comment
While I can agree that it makes it harder for riders to compete in other events, we can't tailor X-style to accommodate every rider and whether or not they are in other events, we should be creating rules based on the one competition, X-Style, itself. Sometimes riders will have to pick and choose, an unfortunate occurrence but a reality when considering we only have one week to fit in events.
I agree that the chief judge should have the authority and freedom to decide if every starter needs to judge. I like Jan's modification.
The idea of X-style is that it is a peer-judged competition, and on this basis, riders should be required to stay and judge their peers. If we are having riders leave part way through for something else, it takes away from the competition.
Comment
This topic needs further discussion in the future, but will be closed for now.