1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
|
(Last updated 18/oct/2011 by cassio@ime.usp.br. See copyright notice below.)
AUTOJUDGING.txt
---------------
To help the process of judging, the BOCA system provides
a php script that is able to execute some user defined
script in order to compile, execute and compare the output
generated by teams with the correct output. This procedure
is not essential to the system, since each judge is capable
to download from BOCA all the files needed to evaluate a
submission.
The propose here is not to have a automated scheme for
submission evaluation, but a supporting system that can
help judges to do their job. In this way, the autojudging
provides to the judge the data generated by the scripts
when trying to compile, to execute and to compare the results.
This data is presented in a formatted way during the judging
process inside the BOCA web interface, and can help the
judge to evaluate the submission. If the autojudging is
not enabled in the contest, the judges will see just some
"unavailable" texts when judging a submission, and they
will have to execute the whole process by theirselves.
To start the autojudging procedure, the only thing needed
is to run the autojudging.php script from some commandline
shell. Note that it may be necessary to change the permissions
of private/conf.php file to something more readable and the php.ini
file to turn safe_mode off (on the computer running the
autojudging scheme). Furthermore, verify whether the private/conf.php
options are correctly pointing to the postgresql server/users
and if the current working directory (ie, ".") is included
in the PATH environment variable.
$ sudo /bin/bash
# cd /var/www/boca
# php private/autojudging.php
or alternatively
$ sudo /var/www/boca/tools/autojudge.sh
The autojudging.php will stay running until CRTL+C is
pressed or it's killed by some other way. Autojudging waits
for a new submission, takes it and executes the
compiling/running script defined in BOCA for the
corresponding language/problem (this script should be
defined during language insertions). The following five
arguments are passed to the compiling/running script:
$1 base_filename
$2 source_file
$3 input_file
$4 languagename
$5 problemname
Then, this script must perform the desired steps
(commonly it compiles and runs the source code for
the given input/language/problem) and:
* the output of the execution should be directed to
the standard output
* the standard error may be used to print anything
* the return code of the script shows what happened
during the script execution, following the rules:
FROM SAFEEXEC
# 0 ok
# 1 compile error
# 2 runtime error
# 3 timelimit exceeded
# 4 internal error
# 5 parameter error
# 6 internal error
# 7 memory limit exceeded
# 8 security threat
# 9 runtime error
other_codes are unknown to BOCA: in this case BOCA
will present an unknown error status
After the compiling/running script is executed, the
autojudging calls the comparing script to verify if
the output generated is correct. Now the arguments
passed to comparing script are:
$1 team_output
$2 sol_output
$3 languagename
$4 problemname
Basicly the comparing script compares the team
output with the solution (correct) output. The last
line of the standard output of the comparing script
is read by BOCA which transmits it to judges (in
fact this information is available when the judge
takes a submission to evaluate). Moreover, the compare
script send the following codes to BOCA:
4 - YES, files match
5 - Presentation error
6 - Wrong answer
As long as the autojudging php script will execute
code from teams, these codes may contain malicious
procedures. Thus it is essential that a dedicated
computer be allocated only for the autojudging
scheme. Furthermore, users with shell on this computer
could eventually see important data which they should not
be allowed. This computer must have the same requirements
as the one running the BOCA web interface (but it
does not need postgresql neither a running web server),
and it need to be configured properly. All the steps
described in INSTALL.txt applies to this new computer,
but three important things should be noted:
1) the computer must have a kind of firewall that
only permits connections between it and the main BOCA
server (this is an important security issue).
2) the private/conf.php file must be configure properly
with the correct location of the boca database
(probably it will be needed tcp/ip connection to
the postgresql at the main BOCA server), the
correct user/password for database access, the
local IP number and the current contest number.
3) the postgresql running at the main BOCA server
will have to be configured to accept connections
from the IP of the autojudging server. This can
be achieved inserting the line
host all all IPAUTOJUDGING 255.255.255.255 md5
in the pg_hba.conf file at the main BOCA server.
Talking a little bit more about the compiling/running
script, it will be responsible for running the executable
code generated from the team's source. This is exactly
where the danger lives. To decrease the risk and to have
control over the time spent by the team's code, the C
source code safeexec.c (tools directory) should be used.
It was designed to execute other programs with lower
privilegies and with a time limit set. It is enough to
"gcc -o safexec safexec.c" for compiling and it is
necessary to make a setuid root: "chown root.root safexec"
and "chmod 4555 safeexec" for running. The files run.sh
and compare.sh are good examples.
Contacts and Copyrights
-----------------------
BOCA Copyright (c) 2003- Cassio Polpo de Campos (cassio@ime.usp.br)
http://www.ime.usp.br/~cassio/boca
////////////////////////////////////////////////////////////////////////////////
//BOCA Online Contest Administrator
// Copyright (C) 2003-2012 by BOCA Development Team (bocasystem@gmail.com)
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
////////////////////////////////////////////////////////////////////////////////
|
