BOCA Manual for the judges

From be2491b093b1f0ca430bede679ecbb670041e483 Mon Sep 17 00:00:00 2001 From: cassio Date: Tue, 2 Jul 2013 09:46:45 +0400 Subject: restructuring of boca's git --- doc/html/manualjudge.en.html | 203 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 203 insertions(+) create mode 100644 doc/html/manualjudge.en.html (limited to 'doc/html/manualjudge.en.html') diff --git a/doc/html/manualjudge.en.html b/doc/html/manualjudge.en.html new file mode 100644 index 0000000..a7f8a0e --- /dev/null +++ b/doc/html/manualjudge.en.html @@ -0,0 +1,203 @@ + + + + +BOCA Manual for the judges + + +BOCA Online Contest Administrator +

BOCA Reference Manual for judges - +version October/2011 (BOCA 1.4.1+) +

+ +

Copyright (c) 2006-2011 Cassio P. de Campos (cassio@ime.usp.br).
+Permission is granted to copy, distribute and/or modify this document under +the terms of the GNU Free Documentation License, Version 1.2 or any later +version published by the Free Software Foundation; with no Invariant Sections, +no Front-Cover Texts, and no Back-Cover Texts. A copy of the license may be +found in http://www.gnu.org/licenses/

+ +

BOCA is a software created to control a contest with the ACM International Collegiate +Programming Contest rules. It has been developed in PHP and the interaction between judges +and the system is done through a web browser. In the following we describe +the set of available features for a judge in the system.

+ +

It is assumed here that the judge has already logged in the system with their +username and password in a windows of their browser. Tthe URL +to access BOCA will depend on the setup of your server. Usually this URL ends with +/boca/, where a simple authentication form will show up. + +

+After a successful login, the web page contains the judge identification in the upper-left +corner. In the upper-right corner, the clock of the contest is shown, indicating if it has +started or not, if it has already ended, stopped, or in progress, in which case the number of +minutes to go are displayed. Below that, there is a set of options in a vertical menu, namely +Runs, Score, Clarifications, History, As Team, Options e Logout. +There is still an extra option named Chief which only appear to the chief judge, who +has to be appointed in the admin's interface (with in the Site options). +

+ +

Runs

In this area, the judge might visualize the runs that are still to be judged and shall be +taken care of. In order to process a run, the judge has to click in its number (eventually a +message might pop up here indicating the run was already taken; this happens if another judge +got the run before you). After clicking on the run to judge, a new set of data appear about it: + +

Site: site number of the run.
Number: run number.
Time: minutes from the start of the competition until this submission.
Problem X: X is the name of problem. There are links +to download the input and output files (click on the names), or simply to visualize them in the browser (click on view). +These links are mostly used when the autojudge is turned off or is not running properly for a given problem/language. +Instead, if the autojudge feature is working correctly, then the judge can directly to go the links in the bottom part of +the page in order to see the expected output together with the output generated by the team. +
Language Y: Y is the language chosen by the team. The scripts that are used to +compile and execute the submission with this language are available through links. +When using the autojudge, note that is not necessary to use this links, because the results are +already shown in the bottom of the page. +
Source code: here it is possible to view or download the submitted file. An important +task is to check if the name and extension of this file are correct with respect to the chosen problem, language, +and the specification in the booklet of problem descriptions. +
Answer: the judge has to choose, among the options available in the system (which were configured by the admin), +the correct answer to be sent to the team. +
Autojudging answer: here it is presented the suggestion from the autojudge about this submission. +Usually the autojudge properly identify the answer that has to be sent to the team. However, it may +fail to do so. For example, the autojudge might indicate a wrong answer, because the +diff procedure identified the output and expected output to be different, while the only mistake +happened in the punctuation/accent of a letter (this is usually a case of presentation error, but +may vary according to the contest). Other issues might happen, and the role of the judge is to check everything +for eventual mistakes. +
Autojudged by: indicates the computer that has acts as autojudge. Usually this is not +much relevant for judging a submission. +
Standard output: links for downloading and visualizing the output generated by the team's code are +available. This is the output generated by the team, which is to be compared with the expected output of +the given problem. It might also be possible to see some error, in case it happened. +
Standard error: links for downloading and visualizing the standard error output generated by the team's code. +When a error happens (including compilation or runtime errors, but not restricted to them), this is the most probable file to identify it. +The judge must always check the content of this file. In the end of it, the judge also finds the output of the diff command +that has compared the team's output and the expected one (to facilitate the visual inspection). +

+ +

By pressing the button named Judge, the judge submits their veredict about the run, and then +has no access to the this run anymore. It is also possible to give up judging the run by clicking +on Cancel, which will send the run back to the pool (and it will be eventually judged by another judge). +

+ +

In the tab Runs, the judge can see all the submissions that are yet to be judged. The colors +represent their status, meaning that they are already been judged by others, or that they are waiting +you to judge them (red color). You must act with respect to those in red, as only you can judge them (or +send them back to the pool). +

+ +

Chief

+Besides acting as a normal judge, there is a designated judge with access to the tab Chief. This tab +is used for the chief judge to resolve disputes regarding submissions that received different answers by different +judges (they are shown in red). The prodecure is equivalent to that of a normal judge, but the decision made in the tab of the chief is +final. The idea is to have the chief judge acting as a tie-breaker between the distinct answers that were assigned to +a run. Hence, if the chief judge is acting as a normal judge (and they may do so), it is recommended to use +the tab Runs instead, leaving the tab Chief only for resolving issues. (In the very special case where there +is a single judge in the competition, then this judge has to be designated as chief and shall use the tab Chief +to judge the runs, otherwise they will never receive the judgement of another judge and thus will not be sent to teams.) +In this same tab, the chief judge is able to ask the autojudge to be re-executed for some submissions (selected by +the boxes besides each of them) or to completely re-open the submission for judging again, which implies in a new round +of autojudging and judging by actual judges. The team does not become aware of this rejudging unless the new final +result for the run has to be changed (with respect to the first round of judgement). +

+ +

Score

In this tab the judge can see the scoreboard of the competition. It has to be noted that the scoreboard available +for the judges is complete and not subject to the freezing of the final part of the contest. Hence, the judge is +expected to keep the scoreboard in secret. In case the scoreboard is consolidated among different sites, the final part +of the other sites is not shown (*this fact is for technical reasons, to be discussed later*). +

+ +

Clarifications

This tab allows the judge to answer clarifications submitted by team regarding +a specific problem or any general aspect of the contest. +In order to reply to a clarification, the judge must first click on the clarification number. +The box available in the bottom part is meant for creating a new clarification, as judges are allowed to do +so. The new clarification will only be meaningful if the judge that gets it to answer selected the reply to all +option (explained in the next paragraph). This is useful for the judges to send a general information to the teams. +

+ +

After clicking on the clarification number, the judge sees information about it, such as +site number, clarification number, time in minutes from the start of the competition, and the +problem to which the clarification regards. There are two text boxes: one with the question (on top) +and one to be filled with the reply. Finally, there is a selection box to indicate if the reply +should be sent to all the teams or just the the team that posted the question. +

+ +

The system also allows the judge to select the No response button. In general +this is used for questions that are already stated in the booklet of problem descriptions, or +that has already been replied, or even that should not be answered at all (for example, about +the timelimit of a problem, or about some sensitive information of the input/output). +

+ +

History

This tab shows the history of Clarifications and +Runs that were processed by the judge so far. +

+ +

Options

This tab shows the information of the judge, such as Username, +User full name , User description. It is possible +for the judge to update their password, although this is not necessary neither +recommended. Instead it is recommended that the admin who created the users already +specifies secret and safe password for all users. +

+
+ +

Logout

Button to log out from the judge interface.

+ +

Important hints for judges

While judging a submission, it is necessary to be very careful. Even if it is possible +to alter the judgement of a run afterwards, this has to be avoided as much as possible, because +the team might suffer an undesired situation. +

Every time a judge is about to answer a problem for its first time (that is, the problem has not +been submitted before, or has not received yet any YESes), when possible it is interesting to have other judges +also checking the submission, which should be analyzed with even greater care. Sometimes this is the +moment when an unfortunate issue with the autojudge or with inputs or outputs is discovered +(obviously nobody wants it to happen, but it might). +

In order to reply to clarifications, it is important that the judge has read and understood +the question and the problem to which it concerns. When possible, it is interesting to have the opinion +of the person or group that created the problem description, its input and output. After that, +it is necessary to think if such reply has to be sent only +to this user or to all teams in the contest. If it is a relevant issue and the contest is also offered in +other sites, then the judge shall find a way to contact other sites about the clarification. +Some examples of clarifications that are not disclosured (that is, are replied with a no response) +are +What is the timelimit for this problem?, +Given this input, which is the correct output?, +questions that are already explained in the problem description, +etc. +In case the judge is not certain about it, they must contact the chief judge of the site or the +chief judge of the contest for help. +

+ +

About BOCA and this document

BOCA System and this document have been created by Cassio Polpo de Campos and can be found at +http://www.ime.usp.br/~cassio/boca/.

+ +

+ + + +

-- cgit v1.2.3