Competition Rules

Overview

RevoASR is a Malay Automatic Speech Recognition competition. Participants build ASR models that transcribe Malay audio into text.

• Task: Malay speech-to-text recognition
• Hardware: CPU-only inference (no GPU during evaluation)
• Real-Time Factor (RTF) limit: RTF < 0.08 (your model must process audio at least 12.5x faster than real-time)

Submission Format

• Required file: inference.py — must accept audio file paths and output transcriptions
• Forbidden files: Do not include Dockerfile or requirements.txt in your submission archive
• Archive format: ZIP or tar.gz
• Maximum upload size: 2 GB
• Your archive will be extracted and inference.py will be invoked inside a Docker container

Evaluation Metrics

Submissions are evaluated on three metrics:

• WER (Word Error Rate): The fraction of words that are incorrectly predicted. Computed as (substitutions + insertions + deletions) / total reference words. Lower is better.
• CER (Character Error Rate): Same as WER but computed at the character level. Lower is better.
• RTF (Real-Time Factor): Total inference time divided by total audio duration. An RTF of 0.08 means processing takes 8% of the audio duration. Lower is better. Submissions exceeding RTF 0.08 will not qualify for the leaderboard.

Text Normalization

All predictions and ground truth are normalized before scoring:

• Convert to lowercase
• Keep only characters a-z and standard punctuation
• Remove digits, special symbols, and extra whitespace

Public / Private Split

The test set is divided into two parts:

• Public split (~30%): Scores on this subset are visible on the leaderboard during the competition. This allows you to gauge relative performance.
• Private split (~70%): Scores on this subset are hidden until the competition ends. Final rankings are determined by private split performance.

This split prevents overfitting to the visible leaderboard scores.

Prize Structure

Prizes are awarded across three tracks:

• Real-Time Track: Best WER among submissions meeting the RTF < 0.08 constraint
• Overall Track: Best WER regardless of RTF (no real-time constraint)
• Writeup Pool: Prize pool for teams submitting a technical writeup describing their approach

Submission Limits

• Rate limit: One submission per rate-limit window (see countdown on the upload page after submitting)
• Deadline: Submissions are accepted until the submission deadline (displayed on the upload page)
• Each submission is evaluated automatically. Results appear on the leaderboard and history page once evaluation completes.