FAQs

Q: Can I use extra data?
A: Yes. The distributed train and dev corpora were recorded using the same protocol as the test corpora, and are therefore expected to be better-matched to the test data than any other source. Competing teams may find, however, that system performance benefits from the use of other datasets; if so, teams are encouraged to describe the other datasets used, and the method of their use.

Q: Why are there two test sets (test1 and test2), and which one determines the final ranking?
A: All completed submissions are evaluated on both test1 and test2 in parallel. Test1 results are normally published within three days of submission (longer while the evaluation queue is backlogged) and are mainly for benchmarking during the competition; test2 results are withheld until the competition closes and are used for the final official ranking.

Q: How is my final ranking determined among my submissions?
A: For both tracks, all valid submissions from a team are considered for the final evaluation. We automatically identify each team’s best (Track 1) or Pareto-frontier (Track 2) submission(s), so you do not need to manually pick a final one.

Q: What metrics determine the final ranking?
A: Ranking is based on CER (Character Error Rate), our primary metric — our normalization procedure is optimized around CER. WER is reported only as a secondary/reference metric and is not combined with CER. For Track 2, latency uses TTFT-stable and TTLT with equal weight.

Q: How do I benchmark on the leaderboard?
A: Submit your system using the provided starter template through the CodaBench Challenge page (Track 1; Track 2) to receive official evaluation and appear on the leaderboard.

Q: My submission has been stuck in “Submitted” or “Preparing” for a long time. What should I do?
A: Please do not delete it. Because of intermittent errors in the Codabench backend, a submission is occasionally lost from the evaluation queue and never progresses. We check for these regularly and resubmit them for you in the background; the resubmission appears in your account as a new submission with the same file, and the original is cancelled. If you delete a stuck submission yourself, we can no longer see it and cannot resubmit it.

Q: Evaluation is taking much longer than usual. Will my submission still count?
A: Yes. Final results will be determined and announced only after every submission made before the deadline (October 24, 2026, AoE) has been evaluated, so a submission still in the queue at the deadline will be counted once its evaluation finishes.

Q: Do I need to open-source my code?
A: No. Code submitted to the challenge remains the intellectual property of the submitting team unless separately open-sourced.

Q: Is the workshop paper archival, and is it required to attend the workshop?
A: No to both. Submitting a paper is optional — teams who don’t submit one can still attend the workshop, and it is not a requirement for the competition or the prize pool. The paper is non-archival, so you are free to submit the same or an extended version of the work to other venues (e.g., Interspeech, ICASSP) afterward. Review is lightweight: papers are single-blind and are not rejected based on technical content, only if unrelated to the competition or too unclear to understand. See the Call for Papers for details and deadlines.

Q: Can I submit a workshop paper on related work that does not use the SAPC2 benchmark?
A: Yes. The purpose of the workshop is to discuss methods that work for streaming and non-streaming dysarthric ASR. The competition provides a standard benchmark for that task, but approaches to the problem that do not use the shared benchmark are also in scope.