The VAANI Noise Event Dataset adds 106,892 human-annotated noise events to real field recordings from Project Vaani. Use its verified_timestamps split to test whether an Indian speech recognition system still works around traffic, machinery, music, and other sound events. Do not train or score one blended random sample. Report word error rate by language and noise category, and keep speakers out of both train and test sets.
ARTPARK at the Indian Institute of Science published the dataset guide on September 7, 2026. The release gives speech teams something synthetic noise cannot: naturally occurring sound events recorded across Indian districts, with the language, location, transcript availability, annotation quality, and event timing attached to each example.
What the VAANI Noise Event Dataset contains
The September 7 dataset article describes 72,756 timestamped speech segments covering more than 122 hours, 38,541 speakers, 58 Indian languages, 30 states, and 162 districts. Annotators marked 106,892 noise events to the millisecond.
The current Hugging Face dataset card lists 90,637 segments and about 154.6 hours. Those numbers are not contradictory. The larger total includes a no_timestamps tier with 17,884 segments and about 32.4 hours. The article focuses on the 72,756 segments in the verified and unverified timestamp tiers.
There are seven top-level noise categories:
animalvehicle_trafficbaby_childsinging_musicphone_signal_alarmappliance_machinehuman_non_speech
Each row can contain several events. NoiseSubCategoryTimeStamp stores a category, a more specific tag, and start and end times. That structure matters because a clip can move from clean speech into traffic, then overlap with a horn or machine sound. A single clip-level label would hide that change.
The accompanying VAANI paper describes the broader field collection across 165 districts and 105 languages. For implementation, use the dataset card as the contract for the released annotation layer. Do not assume every language or district in the broader corpus appears in this release.
Load the gated dataset without downloading everything first
The repository is public, but access is gated. A developer must sign in to Hugging Face, accept the access terms, and share contact information with the dataset maintainers. The files total about 17.7 GB and use a CC BY 4.0 license.
Install the dataset tools and authenticate:
python -m pip install -U datasets huggingface_hub jiwer
hf auth loginStart with streaming so that a first inspection does not pull every audio shard:
from datasets import Audio, load_dataset
DATASET_ID = "ARTPARK-IISc/Vaani-Noise-Event-Dataset"
rows = load_dataset(
DATASET_ID,
split="train",
streaming=True,
token=True,
)
rows = rows.filter(
lambda row: (
row["annotationQuality"] == "verified_timestamps"
and row["isTranscriptionAvailable"]
and bool(row["NoiseSubCategoryTimeStamp"])
)
)
sample = next(iter(rows))
print(sample["language"], sample["district"])
print(sample["NoiseCategory"])
print(sample["NoiseSubCategoryTimeStamp"][:2])
Use streaming for schema checks and distribution counts. Materialize a fixed manifest before benchmarking. An iterable stream can change order across library versions or worker settings, which makes comparisons harder to reproduce.
If your decoder expects 16 kHz mono audio, cast the downloaded dataset explicitly rather than relying on whatever format a pipeline accepts:
from datasets import Audio, load_dataset
rows = load_dataset(DATASET_ID, split="train", token=True)
rows = rows.cast_column("audio", Audio(sampling_rate=16_000))
That full load is convenient, but it is not a cheap metadata query. Budget disk and cache space before running it in CI.
Build a benchmark that answers a product question
Our recommended first benchmark is not “What WER does the model get on VAANI?” It is “Which languages and noise conditions break the product we plan to ship?”
Begin with verified_timestamps, which contains 11,111 segments and about 21.8 hours. Select only rows with transcripts. Create a manifest with the audio identifier, speaker identifier if available in your accepted export, language, state, district, duration, transcript, and normalized noise categories.
Then cap the number of rows per language and category. Otherwise high-volume groups dominate the overall score. Keep a separate count for every reported cell so a good average cannot hide a category represented by six clips.
The dataset maintainers reserve an additional 10-hour, speaker-disjoint evaluation set for a challenge, but it is not part of the public dataset. That limitation changes how you should split the released data. Do not randomly divide rows and call the result independent. The same speaker or recording context can leak across both sides and make the model look better than it is.
If stable speaker IDs are unavailable in the exported fields, avoid using the release as a training and test source in the same experiment. Use it as an external evaluation set for a model trained elsewhere. That is the more defensible baseline.
If your team is building an Indian-language voice product, Axentia's generative AI development service can turn these slices into a repeatable pre-release benchmark tied to your own latency and accuracy targets.
Score by event, not just by clip
Word error rate is a useful starting point when transcripts are available:
from collections import defaultdict
from jiwer import wer
scores = defaultdict(list)
for row in evaluation_rows:
prediction = transcribe(row["audio"])
value = wer(row["transcript"], prediction)
categories = {
event["category"]
for event in row["NoiseSubCategoryTimeStamp"]
}
for category in categories:
scores[(row["language"], category)].append(value)
for key, values in sorted(scores.items()):
print(key, len(values), sum(values) / len(values))
Normalize transcripts with the same rules on references and predictions. Preserve the original text alongside the normalized form. Indian-language scripts, code switching, punctuation, and numerals can move WER substantially, so the normalization policy belongs in the benchmark report.
Also record clip duration, real-time factor, inference cost, and failure rate. A model that improves WER but takes three times longer than the audio may not fit a live assistant. The method in our NanoGPT benchmark evaluation guide applies here too: match the score and budget to the system you plan to operate.
For timestamp-aware analysis, compare words whose time ranges overlap an annotated event with words outside those ranges. That requires word-level reference or alignment output, which the dataset does not promise for every row. When alignment is unavailable, report clip-level category results and say so. Do not infer word boundaries from event timestamps.
What the dataset cannot prove
VAANI captures real field noise, but it is not a production traffic sample for your application. The distribution reflects where and how Project Vaani recordings were collected. A banking voice bot, an agricultural assistant, and a call-center transcription service will see different microphones, codecs, speaking styles, and background conditions.
The two timestamp tiers also have different confidence. About 21.8 hours have verified timestamps, while roughly 100.3 hours have unverified timestamps. The larger tier is valuable for training experiments and broad error discovery, but combining both without a quality flag makes the benchmark ambiguous.
Language coverage is wide, not uniform. An overall WER across 58 languages is especially easy to misread. Always publish per-language counts and confidence intervals or repeated-bootstrap ranges. For languages with too few samples, report that the estimate is unstable instead of ranking models.
Finally, the public release does not provide the held-out challenge evaluation audio. You can reproduce your own manifest, but you cannot claim equivalence to the official speaker-disjoint challenge evaluation.
When the VAANI Noise Event Dataset is worth using
Use it when you are selecting an ASR model for Indian field audio, checking whether a fine-tuned model became brittle around specific noises, or building a routing policy that sends hard clips to a larger model. It is also useful for testing noise-aware preprocessing, provided you score the untouched audio as the baseline.
It is not worth downloading when your product handles only clean studio speech, when none of the covered languages match your users, or when you need telephony-specific codec artifacts more than environmental noise. In those cases, a smaller dataset sampled from the real input channel will answer the decision faster.
Our position is to use VAANI as an external stress test before treating it as training data. The verified tier can expose category-specific failures quickly. Training on the same public rows too early makes leakage and overfitting harder to rule out.
The same caution applies to adaptive media systems. Our Gemini agentic video understanding guide shows why selective evidence loading must be evaluated for missed events, not only cost. For VAANI, the equivalent mistake is reporting one average while ignoring where the noise occurs and which languages carry it.
FAQ
What is the VAANI Noise Event Dataset?
The VAANI Noise Event Dataset is a human-annotated noise layer over Indian field speech collected through Project Vaani. It contains 106,892 timestamped events across seven noise categories. The released dataset includes language, district, transcript availability, annotation quality, and event timing fields for building noise-aware speech evaluations.
Is the VAANI Noise Event Dataset free to use?
The dataset is released under CC BY 4.0, but downloading it requires a Hugging Face account, acceptance of the access conditions, and sharing contact information with the maintainers. Teams should retain attribution, review the dataset card before redistribution, and document the exact revision used in each benchmark.
Should we train an ASR model on VAANI or only evaluate it?
Start with evaluation on the verified timestamp tier, especially if your model was trained elsewhere. If you later train on VAANI, create a speaker-safe split and preserve an untouched external test set. Random row splits can leak voices or recording context, producing an accuracy estimate that will not survive production audio.
The VAANI Noise Event Dataset is useful because it turns “background noise” into testable conditions. If you want help selecting the right language slices and making the result reproducible, talk to the Axentia team.