Knowbility’s AIR Judging Form (.xlsx Excel file 29.99KB)
The form used by judges to evaluate the accessibility of entries in AIR competitions.
What criteria do the judges use to evaluate the sites that are submitted for the OpenAIR Competition?
We use a specific set of criteria to evaluate all of the sites submitted during an OpenAIR competition. We have also included details that help a judge or a competitor determine how to measure each item. Current judging forms will be made available during registration.
How are the judging criteria developed?
The original Judging Criteria were developed in 1998 and were based upon the WCAG accessibility guidelines and Section 508 Guidelines for Web Accessibility. During the periods between rallies, the judges frequently discuss and often approve improvements to the judging criteria based upon changes in technologies, our own experiences evaluating web sites, and input from competitors, their clients, and the accessibility community.
How do we choose OpenAIR Judges?
All of our judges volunteers and experts in the area of accessibility. Our core panel of experts, Jim Allan, Jim Thatcher, and Sharron Rush are internationally recognized experts, and published authors in the field of accessibility. This core group, lead by Mike Moore developed the judging criteria and the judges training manual which has been used to train new judges for OpenAIR programs throughout the United States. Most of our remaining judges started as competitors, developed their expertise and went on to become trainers before completing judge training and judging OpenAIR competitions. During their first competition as a judge a new member of the judging team is mentored by an experienced judge to ensure fairness in the judging process.
Couldn’t a judge’s involvement with team members, their employer or an NPO be a conflict of interest?
Our judges recuse themselves from judging any site produced by a team to which they have a personal affiliation or produced for any organization with whom they have a personal affiliation. Thus a judge who works for the same company as a team that is competing will not judge the site that they create and a judge who works closely with non-profit will not judge a site that is created for them.
How do we control the quality of the judging process?
During each competition we designate one judge as a quality control judge. This person reviews the judging forms submitted by the judges for completeness. As the judging progresses through several rounds, if the point totals vary by more than 15 points between judges the Head Judge will review the judging forms. If errors of fact are found (one judge finds that alt text is missing and another does not) these issues are corrected. If differences of objective opinion are found such as aesthetics or appropriateness points then the head judge will mediate the differences among the judges who have reviewed the site.
How many judges review a site?
Each site is reviewed by two judges. The completed judging forms are then reviewed by the quality control judge. If during the judging process a judge has a question about a particular item, he or she will send that question and the appropriate URL on to the other judges for comment.
Can trainers compete as team members?
Many of our trainers compete as team members. We feel that this is a very good thing for everyone who participates. The competitor/trainers have a good understanding of what works and what does not during the rally and bring that knowledge to all of the competitors during the training classes. They also frequently mentor team members who are new to accessible web development and may not have the confidence to compete without more experienced members on their teams.
Can judges compete as team members?
Active judges are not allowed to be the member of any competing development team. If a judge wishes to join a team and compete as developer he or she must wait six months after the last competition that they have judged prior to joining a team. The former judge does not participate in discussions regarding judging for the competition and is removed from the judges’ mailing list.
1.1 The Accessibility Internet Rally (the “Rally”) involves designers (“Entrants”) and not-for-profit organizations (“NPOs”) creating a website (“an Entry”) for each NPO that is accessible to people with disabilities.
1.2 The Rally is organized by Knowbility and supported by Entrants’ entry fees, grants, sponsorship, and contributions from local agencies, businesses, schools and individuals.
1.3 OpenAIR activities are posted; all registered design teams (Entrants) and nonprofit organizations (NPOs) must be represented at the OpenAIR activities as noted on the registration forms.
1.4 Any notices to Entrants, including changes to these Rules, announcements of winners, notices of disqualification, or otherwise, may be posted at the Rally Website and/or sent by email to Entrants.
2.1 The Accessibility Internet Rally is open to participants in three categories:
2.1.1 Community Organizations: to qualify in this category, entities must be either a 501(c)(3) not-for-profit organization or an entity that exists for community benefit such as a church group, an arts organization, or a performance organization, to be decided at the discretion of the Organizer.
2.1.2 Web design team: to participate in this category, a group of up to 8 web and related professionals sign on as a team. Each web design team shall be considered a single Entrant. No person may enter more than once, either individually or as part of any team. If an Entrant uses the name of a corporation, partnership or other legal entity, the Entrant certifies it has the permission of the entity to do so.
2.1.3 Individual web professional: Individuals may sign on to participate and will be matched to other individuals at the discretion of the Organizer to create a Web Design Team.
2.2 All Entrants must be officially registered to participate in the competition. Participants register by completing the appropriate entry form. Upon the Organizer’s acceptance of this form, participants will be sent a confirming email message. The Organizer may rely on its records and, if it chooses, participant’s possession of an email confirmation, as full and complete evidence that participant is successfully registered.
2.3 Organizer reserves the right to limit participants to any number it chooses, using any mechanism it selects, subject to these rules and applicable state and federal laws, including non-discrimination laws.
2.4 By participating in the Rally, participants fully and unconditionally agree to and accept these Rules.
2.5 Organizer can, at any time and without notification of reason, exclude or disqualify any Entrant whom it believes has not complied with these Rules. Organizer can, at any time, exclude or disqualify any Entrant whose participation it reasonably believes would violate state or federal law.
The Design: Conditions
3.1 Entries must be the Entrant’s own original work or appropriately acknowledged open source materials.
3.2 Entrants are required to submit their Entries in accordance with a mechanism determined by the Organizer This mechanism shall include instructions for preparing files for upload, file specifications including file type and file size and location for uploading.
3.3 Entrants will be given access to a collection of online resources (training) to help them build accessible Web sites. Entrants will be expected to read and familiarize themselves with these resources prior to the Rally.
3.4 Entrants must be represented at the mandatory activities associated with the Rally.
Jury and Judging Entries
4.1 Entries will be judged by one or more Judges selected from a Judging Committee, comprised of experts in accessible design, to be determined and invited by the Organizer.
4.2 The Judges will judge each Entry based on the following criteria: (1) accessibility; (2) creativity; (3) usability; (4) aesthetics and (5) extensibility of concept to a larger website. In addition, Judges will accept the judgment of the nonprofit partner to score the category of appropriateness for the intended audience. The NPO partner will determine to what extent the Entry meets the need of the nonprofit constituency.
4.3 In order to encourage more interactive, sophisticated, community driven sites and fuller user experience, the use of third-party modules or pre-existing components is allowed under these conditions:
4.3.1 Third-party modules or pre-existing components may include discussion forums, shopping carts, video (including captions and audio descriptions), audio files and transcripts, Flash movies, blogs, and PDF documents.
4.3.2 Entrants may enhance the accessibility of third-party components or pre-existing modules prior to the OpenAIR Kick-Off.
4.3.3 The third-party component or pre-existing module may not be the interface/template for the entire web site.
4.3.4 The third-party component or pre-existing module will be judged as part of the site. No bonus points will be awarded for the use of such components. However, this does not preclude the judges from awarding bonus points for effective and innovative modifications for accessibility.
4.3.5 In accordance with clauses 4.2 and 4.3 the Judging Committee shall select the best designs in each of the divisions. The decisions of the Judges are final and binding.
5.1 In accordance with clause 4, each winning team will receive an engraved trophy and one badge to the 2016 SXSW Interactive Festival. The Second and Third Place winners will also receive awards.
5.2 The prizes will be distributed to the winners at the Awards Gala.
5.3 The Organizer will notify winning participants by email that their Entries have been selected and they are prize winners.
Disqualification. Termination of Design Contest
6.1 An Entry which does not comply with these Rules will be disqualified.
6.2 If for any reason it becomes impossible for the Organizer to run the Rally as planned, or if a technical or non-technical problem corrupts the administration, security or proper judging of the Rally, then the Organizer reserves the right, in its sole discretion, to terminate, suspend, modify or cancel the Rally.
Release of Responsibility and Liability
7.1 Organizers, sponsors and their respective parent, subsidiary, and affiliate companies, their successors and assigns, and their directors, officers, stockholders and employees shall not be liable for (i) excluding participants or potential participants pursuant to these Rules; or (ii) late, lost, misdirected, or unsuccessful efforts to notify winners of any prize.
7.2 Organizers, sponsors and their respective parent, subsidiary, and affiliate companies, their successors and assigns, and their directors, officers, stockholders and employees shall not be liable for: (i) late, lost, delayed, damaged, misdirected, incomplete, illegible, non-functioning or unintelligible entries; (ii) telephone, electronic, hardware or software program, network, Internet, or computer malfunctions, failures or difficulties of any kind; (iii) failed, incomplete, garbled or delayed computer transmissions; (iv) any condition caused by events beyond their control that may cause the Rally to be disrupted or corrupted; (v) any injuries, losses or damages of any kind arising in connection with or as a result of the prize, or acceptance, possession or use of the prize, or from participation in the Rally; (vi) any printing or typographical errors in any materials associated with the Rally; (vii) or any damage to participant’s computer occasioned by participation in the Rally, or downloading any information necessary to participate in the Rally.
7.3 Each participant in the Rally agrees to and hereby does release and hold harmless Organizers, sponsors and their respective parent, subsidiary, and affiliate companies, their successors and assigns, and their directors, officers, stockholders and employees from and against any losses, damages, rights, claims and actions of any kind in connection with the Rally or Judging, or resulting from the acceptance, possession or use of any prize, including without limitation, personal injury, death, property damage, lost business or claims based on publicity rights, defamation or invasion of privacy.
Intellectual Property; Privacy
8.1 Each participant, by virtue of his or her participation in the Competition grants to Knowbility, Inc., the absolute and irrevocable right, free of charge, to use and/or publish the entirety or any part or parts of his or her submitted Entry, including posting it, along with the name(s) of the person(s) who created it, on the Knowbility web site, and to pass Entrant’s name and contact information to others for the purpose of contacting Entrant to discuss web design services.
8.2 Each Entrant warrants that the Entry is Entrant’s own original work and that the Entry, the assignment and transfer thereof and the eventual publication and use thereof does not infringe on any right of any third party, including intellectual property rights and rights of privacy.
9.1 Any matter relating to the Rally or its implementation shall be decided by these Rules and applicable state and federal laws. Any matter that cannot be so decided shall be decided by the Organizer in consultation with the Judging Committee. The Judges’ decisions shall be final.