aboutsummaryrefslogtreecommitdiff
path: root/boca-1.5.0/doc/AUTOJUDGING.txt
diff options
context:
space:
mode:
authorcassiopc <cassiopc@gmail.com>2012-08-06 09:09:10 +0000
committercassiopc <cassiopc@gmail.com>2012-08-06 09:09:10 +0000
commit866658cded5b92ddb2681dead1ebaf111d712fcc (patch)
treecf7534c4726a142189e096ce82f8d73b2a398ecc /boca-1.5.0/doc/AUTOJUDGING.txt
downloadboca-866658cded5b92ddb2681dead1ebaf111d712fcc.tar.gz
boca-866658cded5b92ddb2681dead1ebaf111d712fcc.zip
init
Diffstat (limited to 'boca-1.5.0/doc/AUTOJUDGING.txt')
-rw-r--r--boca-1.5.0/doc/AUTOJUDGING.txt169
1 files changed, 169 insertions, 0 deletions
diff --git a/boca-1.5.0/doc/AUTOJUDGING.txt b/boca-1.5.0/doc/AUTOJUDGING.txt
new file mode 100644
index 0000000..5e89cb1
--- /dev/null
+++ b/boca-1.5.0/doc/AUTOJUDGING.txt
@@ -0,0 +1,169 @@
+(Last updated 18/oct/2011 by cassio@ime.usp.br. See copyright notice below.)
+
+AUTOJUDGING.txt
+---------------
+
+To help the process of judging, the BOCA system provides
+a php script that is able to execute some user defined
+script in order to compile, execute and compare the output
+generated by teams with the correct output. This procedure
+is not essential to the system, since each judge is capable
+to download from BOCA all the files needed to evaluate a
+submission.
+
+The propose here is not to have a automated scheme for
+submission evaluation, but a supporting system that can
+help judges to do their job. In this way, the autojudging
+provides to the judge the data generated by the scripts
+when trying to compile, to execute and to compare the results.
+This data is presented in a formatted way during the judging
+process inside the BOCA web interface, and can help the
+judge to evaluate the submission. If the autojudging is
+not enabled in the contest, the judges will see just some
+"unavailable" texts when judging a submission, and they
+will have to execute the whole process by theirselves.
+
+To start the autojudging procedure, the only thing needed
+is to run the autojudging.php script from some commandline
+shell. Note that it may be necessary to change the permissions
+of private/conf.php file to something more readable and the php.ini
+file to turn safe_mode off (on the computer running the
+autojudging scheme). Furthermore, verify whether the private/conf.php
+options are correctly pointing to the postgresql server/users
+and if the current working directory (ie, ".") is included
+in the PATH environment variable.
+
+$ sudo /bin/bash
+# cd /var/www/boca
+# php private/autojudging.php
+
+or alternatively
+$ sudo /var/www/boca/tools/autojudge.sh
+
+The autojudging.php will stay running until CRTL+C is
+pressed or it's killed by some other way. Autojudging waits
+for a new submission, takes it and executes the
+compiling/running script defined in BOCA for the
+corresponding language/problem (this script should be
+defined during language insertions). The following five
+arguments are passed to the compiling/running script:
+
+$1 base_filename
+$2 source_file
+$3 input_file
+$4 languagename
+$5 problemname
+
+Then, this script must perform the desired steps
+(commonly it compiles and runs the source code for
+the given input/language/problem) and:
+
+* the output of the execution should be directed to
+ the standard output
+* the standard error may be used to print anything
+* the return code of the script shows what happened
+ during the script execution, following the rules:
+ FROM SAFEEXEC
+# 0 ok
+# 1 compile error
+# 2 runtime error
+# 3 timelimit exceeded
+# 4 internal error
+# 5 parameter error
+# 6 internal error
+# 7 memory limit exceeded
+# 8 security threat
+# 9 runtime error
+ other_codes are unknown to BOCA: in this case BOCA
+ will present an unknown error status
+
+After the compiling/running script is executed, the
+autojudging calls the comparing script to verify if
+the output generated is correct. Now the arguments
+passed to comparing script are:
+
+$1 team_output
+$2 sol_output
+$3 languagename
+$4 problemname
+
+Basicly the comparing script compares the team
+output with the solution (correct) output. The last
+line of the standard output of the comparing script
+is read by BOCA which transmits it to judges (in
+fact this information is available when the judge
+takes a submission to evaluate). Moreover, the compare
+script send the following codes to BOCA:
+ 4 - YES, files match
+ 5 - Presentation error
+ 6 - Wrong answer
+
+As long as the autojudging php script will execute
+code from teams, these codes may contain malicious
+procedures. Thus it is essential that a dedicated
+computer be allocated only for the autojudging
+scheme. Furthermore, users with shell on this computer
+could eventually see important data which they should not
+be allowed. This computer must have the same requirements
+as the one running the BOCA web interface (but it
+does not need postgresql neither a running web server),
+and it need to be configured properly. All the steps
+described in INSTALL.txt applies to this new computer,
+but three important things should be noted:
+
+1) the computer must have a kind of firewall that
+only permits connections between it and the main BOCA
+server (this is an important security issue).
+
+2) the private/conf.php file must be configure properly
+with the correct location of the boca database
+(probably it will be needed tcp/ip connection to
+the postgresql at the main BOCA server), the
+correct user/password for database access, the
+local IP number and the current contest number.
+
+3) the postgresql running at the main BOCA server
+will have to be configured to accept connections
+from the IP of the autojudging server. This can
+be achieved inserting the line
+
+host all all IPAUTOJUDGING 255.255.255.255 md5
+
+in the pg_hba.conf file at the main BOCA server.
+
+Talking a little bit more about the compiling/running
+script, it will be responsible for running the executable
+code generated from the team's source. This is exactly
+where the danger lives. To decrease the risk and to have
+control over the time spent by the team's code, the C
+source code safeexec.c (tools directory) should be used.
+It was designed to execute other programs with lower
+privilegies and with a time limit set. It is enough to
+"gcc -o safexec safexec.c" for compiling and it is
+necessary to make a setuid root: "chown root.root safexec"
+and "chmod 4555 safeexec" for running. The files run.sh
+and compare.sh are good examples.
+
+
+
+Contacts and Copyrights
+-----------------------
+BOCA Copyright (c) 2003- Cassio Polpo de Campos (cassio@ime.usp.br)
+http://www.ime.usp.br/~cassio/boca
+
+////////////////////////////////////////////////////////////////////////////////
+//BOCA Online Contest Administrator
+// Copyright (C) 2003-2012 by BOCA Development Team (bocasystem@gmail.com)
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+// You should have received a copy of the GNU General Public License
+// along with this program. If not, see <http://www.gnu.org/licenses/>.
+////////////////////////////////////////////////////////////////////////////////