diff options
Diffstat (limited to 'doc/html/manualjudge.en.html')
| -rw-r--r-- | doc/html/manualjudge.en.html | 203 |
1 files changed, 203 insertions, 0 deletions
diff --git a/doc/html/manualjudge.en.html b/doc/html/manualjudge.en.html new file mode 100644 index 0000000..a7f8a0e --- /dev/null +++ b/doc/html/manualjudge.en.html @@ -0,0 +1,203 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html><head> + +<meta http-equiv="content-type" content="text/html;charset=iso-8859-1"> +<title>BOCA Manual for the judges</title> +</head> +<body bgcolor="white" link="blue" vlink="blue"> +<font size="5"><b>BOCA Online Contest Administrator</b></font> +<p><b>BOCA Reference Manual for judges<font size="5"> - </font> +version October/2011 (BOCA 1.4.1+)</b> +</p> + +<p><font size="1">Copyright (c) 2006-2011 Cassio P. de Campos (cassio@ime.usp.br).<br> +Permission is granted to copy, distribute and/or modify this document under +the terms of the GNU Free Documentation License, Version 1.2 or any later +version published by the Free Software Foundation; with no Invariant Sections, +no Front-Cover Texts, and no Back-Cover Texts. A copy of the license may be +found in <a href="http://www.gnu.org/licenses/">http://www.gnu.org/licenses/</a></font></p> + +<p>BOCA is a software created to control a contest with the <i>ACM International Collegiate +Programming Contest</i> rules. It has been developed in PHP and the interaction between judges +and the system is done through a <i>web browser</i>. In the following we describe +the set of available features for a judge in the system.</p> + +<p>It is assumed here that the judge has already logged in the system with their +<i>username</i> and <i>password</i> in a windows of their browser. Tthe <i>URL</i> +to access BOCA will depend on the setup of your server. Usually this URL ends with +<i>/boca/</i>, where a simple authentication form will show up. + +<p> +After a successful login, the web page contains the judge identification in the upper-left +corner. In the upper-right corner, the clock of the contest is shown, indicating if it has +started or not, if it has already ended, stopped, or in progress, in which case the number of +minutes to go are displayed. Below that, there is a set of options in a vertical menu, namely +<i>Runs, Score, Clarifications, History, As Team, Options e Logout.</i> +There is still an extra option named <i>Chief</i> which only appear to the chief judge, who +has to be appointed in the admin's interface (with in the <i>Site</i> options). +</p> + +<p><font size="4"><b>Runs</b></font></p> +<p> In this area, the judge might visualize the runs that are still to be judged and shall be +taken care of. In order to process a run, the judge has to click in its number (eventually a +message might pop up here indicating the run was already taken; this happens if another judge +got the run before you). After clicking on the run to judge, a new set of data appear about it: + +<ol> +<li><b>Site</b>: site number of the run.</li> +<li><b>Number</b>: run number.</li> +<li><b>Time</b>: minutes from the start of the competition until this submission.</li> +<li><b>Problem</b> <i>X</i>: <i>X</i> is the name of problem. There are links +to download the input and output files (click on the names), or simply to visualize them in the browser (click on view). +These links are mostly used when the <i>autojudge</i> is turned off or is not running properly for a given problem/language. +Instead, if the <i>autojudge</i> feature is working correctly, then the judge can directly to go the links in the bottom part of +the page in order to see the expected output together with the output generated by the team. +</li> +<li><b>Language</b> <i>Y</i>: <i>Y</i> is the language chosen by the team. The scripts that are used to +compile and execute the submission with this language are available through links. +When using the <i>autojudge</i>, note that is not necessary to use this links, because the results are +already shown in the bottom of the page. +</li> +<li><b>Source code</b>: here it is possible to <i>view</i> or download the submitted file. An important +task is to check if the name and extension of this file are correct with respect to the chosen problem, language, +and the specification in the booklet of problem descriptions. +</li> +<li><b>Answer</b>: the judge has to choose, among the options available in the system (which were configured by the admin), +the correct answer to be sent to the team. +</li> +<li><b>Autojudging answer</b>: here it is presented the suggestion from the <i>autojudge</i> about this submission. +Usually the <i>autojudge</i> properly identify the answer that has to be sent to the team. However, it may +fail to do so. For example, the <i>autojudge</i> might indicate a <b>wrong answer</b>, because the +<i>diff</i> procedure identified the output and expected output to be different, while the only mistake +happened in the punctuation/accent of a letter (this is usually a case of <b>presentation error</b>, but +may vary according to the contest). Other issues might happen, and the role of the judge is to check everything +for eventual mistakes. +</li> +<li><b>Autojudged by</b>: indicates the computer that has acts as <i>autojudge</i>. Usually this is not +much relevant for judging a submission. +</li> +<li><b>Standard output</b>: links for downloading and visualizing the output generated by the team's code are +available. This is the output generated by the team, which is to be compared with the expected output of +the given problem. It might also be possible to see some error, in case it happened. +</li> +<li><b>Standard error</b>: links for downloading and visualizing the standard error output generated by the team's code. +When a error happens (including compilation or runtime errors, but not restricted to them), this is the most probable file to identify it. +The judge must always check the content of this file. In the end of it, the judge also finds the output of the <i>diff</i> command +that has compared the team's output and the expected one (to facilitate the visual inspection). +</li> +</ol> + +<p>By pressing the button named <b>Judge</b>, the judge submits their veredict about the <i>run</i>, and then +has no access to the this <i>run</i> anymore. It is also possible to give up judging the run by clicking +on <b>Cancel</b>, which will send the run back to the pool (and it will be eventually judged by another judge). +</p> + +<p>In the tab <i>Runs</i>, the judge can see all the submissions that are yet to be judged. The colors +represent their status, meaning that they are already been judged by others, or that they are waiting +you to judge them (red color). You must act with respect to those in red, as only you can judge them (or +send them back to the pool). +</p> + +<p></p> +<p><font size="4"><b>Chief</b></font></p> +<p> +Besides acting as a normal judge, there is a designated judge with access to the tab <i>Chief</i>. This tab +is used for the chief judge to resolve disputes regarding submissions that received different answers by different +judges (they are shown in red). The prodecure is equivalent to that of a normal judge, but the decision made in the tab of the chief is +final. The idea is to have the chief judge acting as a tie-breaker between the distinct answers that were assigned to +a <i>run</i>. Hence, if the chief judge is acting as a normal judge (and they may do so), it is recommended to use +the tab <i>Runs</i> instead, leaving the tab <i>Chief</i> only for resolving issues. (In the very special case where there +is a single judge in the competition, then this judge has to be designated as chief and shall use the tab <i>Chief</i> +to judge the runs, otherwise they will never receive the judgement of another judge and thus will not be sent to teams.) +In this same tab, the chief judge is able to ask the <i>autojudge</i> to be re-executed for some submissions (selected by +the boxes besides each of them) or to completely re-open the submission for judging again, which implies in a new round +of autojudging and judging by actual judges. The <b>team does not become aware</b> of this rejudging unless the new final +result for the run has to be changed (with respect to the first round of judgement). +</p> +<p></p> + +<p></p> +<p><font size="4"><b>Score</b></font></p> +<p>In this tab the judge can see the scoreboard of the competition. It has to be noted that the scoreboard available +for the judges is complete and not subject to the freezing of the final part of the contest. Hence, the judge is +expected to keep the scoreboard in secret. In case the scoreboard is consolidated among different sites, the final part +of the other sites is not shown (*this fact is for technical reasons, to be discussed later*). +</p> + +<p><font size="4"><b>Clarifications</b></font></p> +<p>This tab allows the judge to answer <i>clarifications</i> submitted by team regarding +a specific problem or any general aspect of the contest. +In order to reply to a <i>clarification</i>, the judge must first <b>click on the clarification number</b>. +The box available in the bottom part is meant for creating a new clarification, as judges are allowed to do +so. The new clarification will only be meaningful if the judge that gets it to answer selected the reply to all +option (explained in the next paragraph). This is useful for the judges to send a general information to the teams. +</p> + +<p>After clicking on the clarification number, the judge sees information about it, such as +site number, clarification number, time in minutes from the start of the competition, and the +problem to which the clarification regards. There are two text boxes: one with the question (on top) +and one to be filled with the reply. Finally, there is a selection box to indicate if the reply +should be sent to all the teams or just the the team that posted the question. +</p> + +<p>The system also allows the judge to select the <u><i>No response</i></u> button. In general +this is used for questions that are already stated in the booklet of problem descriptions, or +that has already been replied, or even that should not be answered at all (for example, about +the timelimit of a problem, or about some sensitive information of the input/output). +</p> + +<p><font size="4"><b>History</b></font></p> +<p>This tab shows the history of <i>Clarifications</i> and +<i>Runs</i> that were processed by the judge so far. +</p> + +<p><font size="4"><b>Options</b></font></p> +<p>This tab shows the information of the judge, such as <i>Username, +User</i> <i>full name </i>,<i> User description</i>. It is possible +for the judge to update their password, although this is not necessary neither +recommended. Instead it is recommended that the admin who created the users already +specifies secret and safe password for all users. +</p> +<br> + +</p> +<p><font size="4"><i><b>Logout</b></i></font></p> +<p>Button to log out from the judge interface.</p> + +<p><font size="4"><b>Important hints for judges</b></font></p> +<p> While judging a submission, it is necessary to be very careful. Even if it is possible +to alter the judgement of a run afterwards, this has to be avoided as much as possible, because +the team might suffer an undesired situation. +</p> +<p>Every time a judge is about to answer a problem for its first time (that is, the problem has not +been submitted before, or has not received yet any YESes), when possible it is interesting to have other judges +also checking the submission, which should be analyzed with even greater care. Sometimes this is the +moment when an unfortunate issue with the <i>autojudge</i> or with inputs or outputs is discovered +(obviously nobody wants it to happen, but it might). +</p> +<p>In order to reply to <i>clarifications</i>, it is important that the judge has read and understood +the question and the problem to which it concerns. When possible, it is interesting to have the opinion +of the person or group that created the problem description, its input and output. After that, +it is necessary to think if such reply has to be sent only +to this user or to all teams in the contest. If it is a relevant issue and the contest is also offered in +other sites, then the judge shall find a way to contact other sites about the clarification. +Some examples of clarifications that are not disclosured (that is, are replied with a <i>no response</i>) +are +<i>What is the timelimit for this problem?</i>, +<i>Given this input, which is the correct output?</i>, +questions that are already explained in the problem description, +etc. +In case the judge is not certain about it, they must contact the chief judge of the site or the +chief judge of the contest for help. +</p> + +<p><font size="4"><b>About BOCA and this document</b></font></p> +<p>BOCA System and this document have been created by Cassio Polpo de Campos and can be found at +<a href="http://www.ime.usp.br/~cassio/boca/">http://www.ime.usp.br/~cassio/boca/</a>.</p> + +<hr> +<p> +<a href="http://validator.w3.org/check?uri=referer"> +<img src="http://www.w3.org/Icons/valid-html401" alt="Valid HTML 4.01!" border="0" height="31" width="88"></a> + +</p></body></html> |