Overview of parameters and parameter space explored
| Parameter . | Values explored . | Description . |
|---|---|---|
| N | 3, 5, 10 | Number of reviewers on the panel |
| T | 6, 12, 24 | Number of topics (attributes for each proposal) for each reviewer to examine |
| C | 2, 3, 5 | Number of evaluation criteria |
| μ | 0.75 | Average value of the proposal attributes |
| σ | 0.2 | SD of the proposal attributes |
| r | 0, 0.5 | Correlation between the attributes of each proposal |
| ε | 0, 0.1, 0.2 | Magnitude of random errors |
| λ | 0, 0.1, 0.2 | Variability in systematic errors |
| ρ | 0, 0.05, 0.1, 0.2, 0.4 | TC-mapping diversity (proportion of links rewired) |
| s | 2, 5, 10 | Granularity of the grading language |
| h | 0, 0.1, 0.2 | Diversity between reviewers’ interpretation of the grading language |
| Parameter . | Values explored . | Description . |
|---|---|---|
| N | 3, 5, 10 | Number of reviewers on the panel |
| T | 6, 12, 24 | Number of topics (attributes for each proposal) for each reviewer to examine |
| C | 2, 3, 5 | Number of evaluation criteria |
| μ | 0.75 | Average value of the proposal attributes |
| σ | 0.2 | SD of the proposal attributes |
| r | 0, 0.5 | Correlation between the attributes of each proposal |
| ε | 0, 0.1, 0.2 | Magnitude of random errors |
| λ | 0, 0.1, 0.2 | Variability in systematic errors |
| ρ | 0, 0.05, 0.1, 0.2, 0.4 | TC-mapping diversity (proportion of links rewired) |
| s | 2, 5, 10 | Granularity of the grading language |
| h | 0, 0.1, 0.2 | Diversity between reviewers’ interpretation of the grading language |