Skip to content

Commit 37da598

Browse files
tuupketuupke
committed
Adds ansible for end-of-contest procedure.
The readme contains some explanation on how to use the playbook. Note, this is probably still quite limited, so check the results before relying on them!
1 parent 57eaf7e commit 37da598

File tree

2 files changed

+272
-0
lines changed

2 files changed

+272
-0
lines changed

‎provision-contest/ansible/EOC/README.md‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
## DOMjudge EOC ansible helper
2+
This ansible playbook should automate must of the tasks DOMjudge needs to perform for the End Of Contest procedures during the ICPC WF's. It was created for the WF 2021 in Dhaka, which took place in 2022.
3+
4+
Two types of authentication are used and supported. Session based auth and basic auth.
5+
- Session based auth retrieves a CSRF token and fakes a login action.
6+
- Basic auth headers are constructed and used. This is to fully mitigate [annoying basic-auth issues](https://askubuntu.com/questions/1070838/why-wget-does-not-use-username-and-password-in-url-first-time/1070847#1070847), even though the `force_basic_auth` option should already prevent these issues from arising.
7+
8+
The full playbook can be run using the following command. See the setup section to see what the contents of facts.yml are supposed to be.
9+
10+
```ansible-playbook main.yml --extra-vars "@facts.yml" --tags all```
11+
12+
### Steps automated.
13+
The following steps from the EOC 2022-Oct document have been automated:
14+
15+
29. `Primary pushes results.tsv into git repo`. Tag for just this action: `results.tsv`. Note, a manual commit is still needed!
16+
32. `Compare scoreboard.json between primary and shadow`. Tag for just this action: `scoreboard`. Note, only if other (shadow or primary) credentials are provided.
17+
33. `Compare awards.json between primary and CDS`. Tag for just this action: `awards`. Note, only if CDS credentials are provided.
18+
36. `Verify that Primary and ... System Test`. Tag for just this action: `fetch`. Note, a manual commit is still needed and the `final_standings.html` is actually the static-scoreboard zip.
19+
20+
21+
## Setup
22+
To run this playbook a facts-file is required. An example of which can be seen below. Facts prefixed with:
23+
- `dj_` are for interaction with a DOMjudge instance.
24+
- `other_` are for interaction with another CLICS spec compliant CCS (commonly shadow if DOMjudge is primary, or vica versa)
25+
- `cds_` are for interaction with the CDS (also CLICS compliant).
26+
27+
Only the `repo`, `contest`, `contest_id`, and `dj_*` variables are required. When the `other_` or `cds_` facts are missing, steps which would interact with these systems are skipped.
28+
29+
- `repo` must point to where all retrieved files should be stored, not simply the repo root!
30+
- `contest` must be the externalID.
31+
- `contest_id` must be the internalID. (cID)
32+
- `jd_loc` must point to the location of a [`jd` binary](https://github.com/josephburnett/jd/releases/tag/v1.6.1). Used for comparing diffs.
33+
34+
```yaml
35+
repo: /home/mart/icpc/wf_repo
36+
contest: bapc2022
37+
contest_id: 2
38+
jd_loc: /home/mart/icpc/EOC/jd
39+
40+
dj_url: https://judge.gehack.nl
41+
dj_url_api_suffix: api/v4
42+
dj_username: mart
43+
dj_password: REDACTED
44+
45+
other_url: https://judge.gehack.nl
46+
other_url_api_suffix: api/v4
47+
other_username: mart
48+
other_password: REDACTED
49+
50+
cds_url: https://cds.gehack.nl
51+
cds_url_api_suffix: api
52+
cds_username: admin
53+
cds_password: REDACTED
54+
```
55+
56+
## Selectively running the playbook
57+
The playbook has multiple tags:
58+
- `all`: runs all tasks
59+
- `check`: Checks the consistency of both scoreboard.json and awards.json against the CDS and other CCS when configured.
60+
- `scoreboard`: Verifies scoreboard.json against the CDS and other CCS when configured.
61+
- `awards`: Verifies awards.json against the CDS and other CCS when configured.
62+
- `results.tsv`: Downloads results.tsv from DOMjudge and stores it in the repo.
63+
- `fetch`: Retrieves all files required by the EOC including results.tsv
64+
65+
66+
### Known limitations
67+
- The checks that verify whether the jsons are the same are *very* simple and I (Mart) have not yet found a reasonable way of actually checking them in a nice manner from ansible. Them matching would be a bigger red-flag than them having a difference. Still, the playbook fails when a difference is detected! To aid with (manual) verification the results of fetching the jsons is stored in `/tmp/[obj].[sys].json` with `[object]` either "awards" or "scoreboard" and `[sys]` either "dj", "cds", or "other".

‎provision-contest/ansible/EOC/main.yml‎

Lines changed: 205 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,205 @@
1+
---
2+
- hosts: localhost
3+
connection: local
4+
gather_facts: no
5+
tasks:
6+
- name: Set facts for whether enough variables are set
7+
tags: all, compare, scoreboard, awards, fetch
8+
ansible.builtin.set_fact:
9+
cds_available: "{{ cds_url is defined and cds_url_api_suffix is defined and cds_username is defined and cds_password is defined }}"
10+
other_available: "{{ other_url is defined and other_url_api_suffix is defined and other_username is defined and other_password is defined }}"
11+
12+
# Do this to prevent the dreaded issues with basic auth. Tedious but it works
13+
- name: Set DOMjudge basic auth fact
14+
tags: all, compare, scoreboard, awards, fetch
15+
ansible.builtin.set_fact:
16+
dj_basic_auth: "Basic {{ (dj_username+':'+dj_password) | b64encode }}"
17+
18+
- name: Set CDS basic auth fact
19+
tags: all, compare, scoreboard, awards
20+
ansible.builtin.set_fact:
21+
cds_basic_auth: "Basic {{ (cds_username+':'+cds_password) | b64encode }}"
22+
when: cds_available
23+
24+
- name: Set other basic auth fact
25+
tags: all, compare, scoreboard, awards
26+
ansible.builtin.set_fact:
27+
other_basic_auth: "Basic {{ (other_username+':'+other_password) | b64encode }}"
28+
when: other_available
29+
30+
31+
- name: Attempt login into DOMjudge
32+
tags: all, fetch, results.tsv
33+
block:
34+
- name: Fetch CSRF from login page
35+
ansible.builtin.uri:
36+
method: GET
37+
return_content: yes
38+
url: "{{ dj_url }}/login"
39+
register: DOMjudge_login_output
40+
41+
- name: Extract CSRF input field
42+
ansible.builtin.set_fact:
43+
DOMjudge_csrf_field: "{{ DOMjudge_login_output.content | regex_search('<input[^>]+?_csrf_token[^>]+?>') | regex_search('[^\"]{30,}') }}"
44+
45+
- name: Login into DOMjudge and retrieve cookie
46+
ansible.builtin.uri:
47+
method: POST
48+
body_format: form-urlencoded
49+
url: "{{ dj_url }}/login"
50+
status_code: 302
51+
headers:
52+
cookie: "{{ DOMjudge_login_output.cookies_string }}"
53+
body:
54+
_username: "{{ dj_username }}"
55+
_password: "{{ dj_password }}"
56+
_csrf_token: "{{ DOMjudge_csrf_field }}"
57+
register: login_response
58+
59+
- name: Test succesfull login by going to a restricted page
60+
ansible.builtin.uri:
61+
url: "{{ dj_url }}/jury"
62+
method: GET
63+
headers:
64+
cookie: "{{ login_response.set_cookie }}"
65+
66+
67+
- name: Download results.tsv
68+
tags: all, results.tsv
69+
ansible.builtin.uri:
70+
url: "{{ dj_url }}/jury/import-export/export/results.tsv"
71+
return_content: yes
72+
method: GET
73+
creates: "{{ repo }}/results.tsv"
74+
dest: "{{ repo }}/results.tsv"
75+
headers:
76+
cookie: "{{ login_response.set_cookie }}"
77+
78+
- name: Compare scoreboard.json
79+
tags: all, compare, scoreboard
80+
block:
81+
- name: "Fetch scoreboard.json from Dj"
82+
ansible.builtin.uri:
83+
url: "{{ dj_url }}/{{ dj_url_api_suffix }}/contests/{{ contest }}/scoreboard"
84+
dest: "/tmp/scoreboard.dj.json"
85+
return_content: yes
86+
force: yes
87+
headers:
88+
Authorization: "{{ dj_basic_auth }}"
89+
register: dj_scoreboard
90+
91+
- name: "Fetch scoreboard.json from other"
92+
ansible.builtin.uri:
93+
url: "{{ other_url }}/{{ other_url_api_suffix }}/contests/{{ contest }}/scoreboard"
94+
dest: "/tmp/scoreboard.other.json"
95+
return_content: yes
96+
force: yes
97+
headers:
98+
Authorization: "{{ other_basic_auth }}"
99+
register: other_scoreboard
100+
when: other_available
101+
102+
# - name: Convert scoreboard to nice json
103+
# block:
104+
# - name: Convert DOMjudge scoreboard to nice json
105+
# ansible.builtin.set_fact:
106+
# dj_scoreboard_nice: "{{ dj_scoreboard.json | to_nice_json }}"
107+
#
108+
# - name: Convert other scoreboard to nice json
109+
# ansible.builtin.set_fact:
110+
# other_scoreboard_nice: "{{ other_scoreboard.json | to_nice_json }}"
111+
# when: other_available
112+
113+
- name: Does Dj == other for scoreboard
114+
shell: "{{ jd_loc }} /tmp/scoreboard.dj.json /tmp/scoreboard.other.json"
115+
when: other_available
116+
117+
- name: Compare awards.json
118+
tags: all, compare, awards
119+
block:
120+
- name: Fetch awards.json from Dj
121+
ansible.builtin.uri:
122+
url: "{{ dj_url }}/{{ dj_url_api_suffix }}/contests/{{ contest }}/awards/"
123+
dest: "/tmp/awards.dj.json"
124+
return_content: yes
125+
force: yes
126+
method: GET
127+
headers:
128+
Authorization: "{{ dj_basic_auth }}"
129+
register: dj_awards
130+
131+
- name: Fetch awards.json from cds
132+
ansible.builtin.uri:
133+
url: "{{ cds_url }}/{{ cds_url_api_suffix }}/contests/{{ contest }}/awards/"
134+
dest: "/tmp/awards.cds.json"
135+
return_content: yes
136+
force: yes
137+
method: GET
138+
headers:
139+
Authorization: "{{ cds_basic_auth }}"
140+
register: cds_awards
141+
when: cds_available
142+
143+
# - name: Convert awards to nice json
144+
# block:
145+
# - name: Convert DOMjudge awards to nice json
146+
# ansible.builtin.set_fact:
147+
# dj_awards_nice: "{{ dj_awards.json | to_nice_json }}"
148+
#
149+
# - name: Convert cds awards to nice json
150+
# ansible.builtin.set_fact:
151+
# cds_awards_nice: "{{ cds_awards.json | to_nice_json }}"
152+
# when: cds_available
153+
154+
- name: Does Dj == CDS for awards
155+
shell: "{{ jd_loc }} /tmp/awards.dj.json /tmp/awards.cds.json"
156+
when: cds_available
157+
158+
- name: "Fetch event-feed.ndjson from Dj"
159+
tags: all, fetch
160+
ansible.builtin.uri:
161+
url: "{{ dj_url }}/{{ dj_url_api_suffix }}/contests/{{ contest }}/event-feed?stream=false"
162+
return_content: yes
163+
method: GET
164+
creates: "{{ repo }}/event-feed.ndjson"
165+
dest: "{{ repo }}/event-feed.ndjson"
166+
headers:
167+
Authorization: "{{ dj_basic_auth }}"
168+
169+
- name: "Store final standings (contest.zip)"
170+
tags: all, fetch
171+
ansible.builtin.uri:
172+
url: "{{ dj_url }}/public/scoreboard-data.zip?contest={{ contest_id }}"
173+
creates: "{{ repo }}/final_standings.zip"
174+
dest: "{{ repo }}/final_standings.zip"
175+
headers:
176+
cookie: "{{ login_response.set_cookie }}"
177+
178+
- name: "Fetch DOMjudge submissions"
179+
tags: all, fetch
180+
block:
181+
- name: Retrieve all submissions
182+
ansible.builtin.uri:
183+
url: "{{ dj_url }}/{{ dj_url_api_suffix }}/contests/{{ contest }}/submissions?strict=1"
184+
return_content: yes
185+
method: GET
186+
headers:
187+
Authorization: "{{ dj_basic_auth }}"
188+
register: "submissions"
189+
190+
- name: Extract submission endpoints
191+
ansible.builtin.set_fact:
192+
submissions: "{{ submissions.json | json_query(jmesquery) }}"
193+
vars:
194+
jmesquery: "[].files[?mime == 'application/zip'][]"
195+
196+
- name: Download submissions
197+
ansible.builtin.uri:
198+
url: "{{ dj_url }}/{{ dj_url_api_suffix }}/{{ item.href }}"
199+
return_content: yes
200+
method: GET
201+
headers:
202+
Authorization: "{{ dj_basic_auth }}"
203+
creates: "{{ repo }}/{{ item.href.split('/')[3] }}.zip"
204+
dest: "{{ repo }}/{{ item.href.split('/')[3] }}.zip"
205+
loop: "{{ submissions }}"

0 commit comments

Comments
 (0)