diff options
Diffstat (limited to 'boca-1.5.0/doc/AUTOJUDGING.txt')
| -rw-r--r-- | boca-1.5.0/doc/AUTOJUDGING.txt | 169 |
1 files changed, 169 insertions, 0 deletions
diff --git a/boca-1.5.0/doc/AUTOJUDGING.txt b/boca-1.5.0/doc/AUTOJUDGING.txt new file mode 100644 index 0000000..5e89cb1 --- /dev/null +++ b/boca-1.5.0/doc/AUTOJUDGING.txt @@ -0,0 +1,169 @@ +(Last updated 18/oct/2011 by cassio@ime.usp.br. See copyright notice below.) + +AUTOJUDGING.txt +--------------- + +To help the process of judging, the BOCA system provides +a php script that is able to execute some user defined +script in order to compile, execute and compare the output +generated by teams with the correct output. This procedure +is not essential to the system, since each judge is capable +to download from BOCA all the files needed to evaluate a +submission. + +The propose here is not to have a automated scheme for +submission evaluation, but a supporting system that can +help judges to do their job. In this way, the autojudging +provides to the judge the data generated by the scripts +when trying to compile, to execute and to compare the results. +This data is presented in a formatted way during the judging +process inside the BOCA web interface, and can help the +judge to evaluate the submission. If the autojudging is +not enabled in the contest, the judges will see just some +"unavailable" texts when judging a submission, and they +will have to execute the whole process by theirselves. + +To start the autojudging procedure, the only thing needed +is to run the autojudging.php script from some commandline +shell. Note that it may be necessary to change the permissions +of private/conf.php file to something more readable and the php.ini +file to turn safe_mode off (on the computer running the +autojudging scheme). Furthermore, verify whether the private/conf.php +options are correctly pointing to the postgresql server/users +and if the current working directory (ie, ".") is included +in the PATH environment variable. + +$ sudo /bin/bash +# cd /var/www/boca +# php private/autojudging.php + +or alternatively +$ sudo /var/www/boca/tools/autojudge.sh + +The autojudging.php will stay running until CRTL+C is +pressed or it's killed by some other way. Autojudging waits +for a new submission, takes it and executes the +compiling/running script defined in BOCA for the +corresponding language/problem (this script should be +defined during language insertions). The following five +arguments are passed to the compiling/running script: + +$1 base_filename +$2 source_file +$3 input_file +$4 languagename +$5 problemname + +Then, this script must perform the desired steps +(commonly it compiles and runs the source code for +the given input/language/problem) and: + +* the output of the execution should be directed to + the standard output +* the standard error may be used to print anything +* the return code of the script shows what happened + during the script execution, following the rules: + FROM SAFEEXEC +# 0 ok +# 1 compile error +# 2 runtime error +# 3 timelimit exceeded +# 4 internal error +# 5 parameter error +# 6 internal error +# 7 memory limit exceeded +# 8 security threat +# 9 runtime error + other_codes are unknown to BOCA: in this case BOCA + will present an unknown error status + +After the compiling/running script is executed, the +autojudging calls the comparing script to verify if +the output generated is correct. Now the arguments +passed to comparing script are: + +$1 team_output +$2 sol_output +$3 languagename +$4 problemname + +Basicly the comparing script compares the team +output with the solution (correct) output. The last +line of the standard output of the comparing script +is read by BOCA which transmits it to judges (in +fact this information is available when the judge +takes a submission to evaluate). Moreover, the compare +script send the following codes to BOCA: + 4 - YES, files match + 5 - Presentation error + 6 - Wrong answer + +As long as the autojudging php script will execute +code from teams, these codes may contain malicious +procedures. Thus it is essential that a dedicated +computer be allocated only for the autojudging +scheme. Furthermore, users with shell on this computer +could eventually see important data which they should not +be allowed. This computer must have the same requirements +as the one running the BOCA web interface (but it +does not need postgresql neither a running web server), +and it need to be configured properly. All the steps +described in INSTALL.txt applies to this new computer, +but three important things should be noted: + +1) the computer must have a kind of firewall that +only permits connections between it and the main BOCA +server (this is an important security issue). + +2) the private/conf.php file must be configure properly +with the correct location of the boca database +(probably it will be needed tcp/ip connection to +the postgresql at the main BOCA server), the +correct user/password for database access, the +local IP number and the current contest number. + +3) the postgresql running at the main BOCA server +will have to be configured to accept connections +from the IP of the autojudging server. This can +be achieved inserting the line + +host all all IPAUTOJUDGING 255.255.255.255 md5 + +in the pg_hba.conf file at the main BOCA server. + +Talking a little bit more about the compiling/running +script, it will be responsible for running the executable +code generated from the team's source. This is exactly +where the danger lives. To decrease the risk and to have +control over the time spent by the team's code, the C +source code safeexec.c (tools directory) should be used. +It was designed to execute other programs with lower +privilegies and with a time limit set. It is enough to +"gcc -o safexec safexec.c" for compiling and it is +necessary to make a setuid root: "chown root.root safexec" +and "chmod 4555 safeexec" for running. The files run.sh +and compare.sh are good examples. + + + +Contacts and Copyrights +----------------------- +BOCA Copyright (c) 2003- Cassio Polpo de Campos (cassio@ime.usp.br) +http://www.ime.usp.br/~cassio/boca + +//////////////////////////////////////////////////////////////////////////////// +//BOCA Online Contest Administrator +// Copyright (C) 2003-2012 by BOCA Development Team (bocasystem@gmail.com) +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// You should have received a copy of the GNU General Public License +// along with this program. If not, see <http://www.gnu.org/licenses/>. +//////////////////////////////////////////////////////////////////////////////// |