(Last updated 18/oct/2011 by cassio@ime.usp.br. See copyright notice below.) AUTOJUDGING.txt --------------- To help the process of judging, the BOCA system provides a php script that is able to execute some user defined script in order to compile, execute and compare the output generated by teams with the correct output. This procedure is not essential to the system, since each judge is capable to download from BOCA all the files needed to evaluate a submission. The propose here is not to have a automated scheme for submission evaluation, but a supporting system that can help judges to do their job. In this way, the autojudging provides to the judge the data generated by the scripts when trying to compile, to execute and to compare the results. This data is presented in a formatted way during the judging process inside the BOCA web interface, and can help the judge to evaluate the submission. If the autojudging is not enabled in the contest, the judges will see just some "unavailable" texts when judging a submission, and they will have to execute the whole process by theirselves. To start the autojudging procedure, the only thing needed is to run the autojudging.php script from some commandline shell. Note that it may be necessary to change the permissions of private/conf.php file to something more readable and the php.ini file to turn safe_mode off (on the computer running the autojudging scheme). Furthermore, verify whether the private/conf.php options are correctly pointing to the postgresql server/users and if the current working directory (ie, ".") is included in the PATH environment variable. $ sudo /bin/bash # cd /var/www/boca # php private/autojudging.php or alternatively $ sudo /var/www/boca/tools/autojudge.sh The autojudging.php will stay running until CRTL+C is pressed or it's killed by some other way. Autojudging waits for a new submission, takes it and executes the compiling/running script defined in BOCA for the corresponding language/problem (this script should be defined during language insertions). The following five arguments are passed to the compiling/running script: $1 base_filename $2 source_file $3 input_file $4 languagename $5 problemname Then, this script must perform the desired steps (commonly it compiles and runs the source code for the given input/language/problem) and: * the output of the execution should be directed to the standard output * the standard error may be used to print anything * the return code of the script shows what happened during the script execution, following the rules: FROM SAFEEXEC # 0 ok # 1 compile error # 2 runtime error # 3 timelimit exceeded # 4 internal error # 5 parameter error # 6 internal error # 7 memory limit exceeded # 8 security threat # 9 runtime error other_codes are unknown to BOCA: in this case BOCA will present an unknown error status After the compiling/running script is executed, the autojudging calls the comparing script to verify if the output generated is correct. Now the arguments passed to comparing script are: $1 team_output $2 sol_output $3 languagename $4 problemname Basicly the comparing script compares the team output with the solution (correct) output. The last line of the standard output of the comparing script is read by BOCA which transmits it to judges (in fact this information is available when the judge takes a submission to evaluate). Moreover, the compare script send the following codes to BOCA: 4 - YES, files match 5 - Presentation error 6 - Wrong answer As long as the autojudging php script will execute code from teams, these codes may contain malicious procedures. Thus it is essential that a dedicated computer be allocated only for the autojudging scheme. Furthermore, users with shell on this computer could eventually see important data which they should not be allowed. This computer must have the same requirements as the one running the BOCA web interface (but it does not need postgresql neither a running web server), and it need to be configured properly. All the steps described in INSTALL.txt applies to this new computer, but three important things should be noted: 1) the computer must have a kind of firewall that only permits connections between it and the main BOCA server (this is an important security issue). 2) the private/conf.php file must be configure properly with the correct location of the boca database (probably it will be needed tcp/ip connection to the postgresql at the main BOCA server), the correct user/password for database access, the local IP number and the current contest number. 3) the postgresql running at the main BOCA server will have to be configured to accept connections from the IP of the autojudging server. This can be achieved inserting the line host all all IPAUTOJUDGING 255.255.255.255 md5 in the pg_hba.conf file at the main BOCA server. Talking a little bit more about the compiling/running script, it will be responsible for running the executable code generated from the team's source. This is exactly where the danger lives. To decrease the risk and to have control over the time spent by the team's code, the C source code safeexec.c (tools directory) should be used. It was designed to execute other programs with lower privilegies and with a time limit set. It is enough to "gcc -o safexec safexec.c" for compiling and it is necessary to make a setuid root: "chown root.root safexec" and "chmod 4555 safeexec" for running. The files run.sh and compare.sh are good examples. Contacts and Copyrights ----------------------- BOCA Copyright (c) 2003- Cassio Polpo de Campos (cassio@ime.usp.br) http://www.ime.usp.br/~cassio/boca //////////////////////////////////////////////////////////////////////////////// //BOCA Online Contest Administrator // Copyright (C) 2003-2012 by BOCA Development Team (bocasystem@gmail.com) // // This program is free software: you can redistribute it and/or modify // it under the terms of the GNU General Public License as published by // the Free Software Foundation, either version 3 of the License, or // (at your option) any later version. // // This program is distributed in the hope that it will be useful, // but WITHOUT ANY WARRANTY; without even the implied warranty of // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the // GNU General Public License for more details. // You should have received a copy of the GNU General Public License // along with this program. If not, see . ////////////////////////////////////////////////////////////////////////////////