Generic Audio Classifier (MFCC + Dense Neural Network)

A reusable audio classification pipeline: supply your own labeled audio dataset (any classes - animal sounds, bird species, speech, environmental sounds) and train a classifier, with a Streamlit app for uploading a clip and seeing the predicted class alongside a reference photo. Originally built and validated on bird species recognition; nothing in the code is bird-specific.

Important limitation - read before using

This is a closed-set classifier. It can only ever output one of the classes it was trained on. If you feed it audio from a class it has never seen, it will not say "I don't know" - by design, softmax always distributes 100% probability across the trained classes. The app includes a low-confidence guardrail (predictions below a threshold are flagged as "no confident match" instead of forced), but this is a heuristic, not true out-of-distribution detection - a genuinely unfamiliar sound can still occasionally produce a confident, wrong answer.

Pipeline

Raw audio -> MFCC feature extraction (128 coefficients, mean-pooled per clip) -> dense neural network classifier (100 -> 200 -> 100 -> softmax).

Audio decoding uses soundfile + python_speech_features - no ffmpeg, pydub, or numba/JIT-compiled dependencies, which avoided Windows Application Control DLL-blocking issues encountered during development.

Project Structure

src/
  feature_extraction.py   # MFCC extraction from a single audio file
  dataset.py               # Builds the full feature dataset from a metadata CSV
  model.py                 # Dense classifier architecture
  cross_validate.py        # Stratified k-fold evaluation (see Evaluation below)
  train_final.py           # Trains the final deployable model on all data
  class_images.py          # Local image lookup + Wikipedia fallback
  app.py                    # Streamlit frontend
requirements.txt

Bring Your Own Dataset

Not included in this repository - bring your own audio, organized like this:

data/
  audio/
    <all your audio files, flat - no subfolders needed>
  Birds_Voice.csv                (or any CSV name - just point scripts at it)
  images/                        (optional, for the app's photo display)
    <ClassLabel>.jpg              # one file per class, name matches the CSV's "class" column
    # OR, for multiple photos per class:
    <ClassLabel>/
      photo1.jpg
      photo2.jpg

The metadata CSV needs at least these columns:

column	description
slice_file_name	audio filename (must exist in data/audio/)
class	class label (used as the classification target, and to match image filenames)

A fold column is not required, and is ignored if present - cross_validate.py uses StratifiedKFold instead, since a dataset's own fold assignments aren't guaranteed to keep every class represented in both train and test splits (this repo's original bird dataset had exactly that problem - see cross_validate.py's docstring).

Example domains this same pipeline works for, unchanged: bird species (the included example), other animal calls, spoken-word/command recognition, environmental/urban sound classification, mechanical fault sounds (e.g. engine knocks) - anything where you have labeled audio clips.

Setup

Python version matters: TensorFlow currently supports Python 3.10-3.13. If your environment is on a newer release (e.g. 3.14), pip install tensorflow will fail with no matching distribution.

python -m venv venv
venv\Scripts\activate          # Windows
pip install -r requirements.txt

If a locked-down Windows environment blocks streamlit.exe or similar compiled launchers (Application Control / Smart App Control), run everything through the Python module form instead of the standalone .exe:

python -m streamlit run src/app.py

Usage

1. Evaluate honestly first, from inside src/:

python cross_validate.py

Reports mean accuracy +/- standard deviation across stratified folds - the number worth reporting, not a single train/test split (which can swing wildly on small datasets).

2. Train the deployable model on all your data:

python train_final.py

Saves to saved_models/audio_classification_final.keras.

3. Launch the app:

python -m streamlit run app.py

Upload a clip, get a prediction, a confidence score, a low-confidence "no match" flag when applicable, per-class probability breakdown, and a reference photo (your own if supplied in data/images/, otherwise a live Wikipedia lookup as fallback - which only resolves for real-world named things, not arbitrary internal labels).

Evaluation Note (bird species example)

The included example dataset was small and imbalanced (66 samples across 6 bird species, ranging 3-30 samples per class), yielding 77.27% mean accuracy (+/- 9.82%) via 3-fold stratified cross-validation. Report your own numbers the same way: with the sample counts and fold count stated alongside the accuracy, not as a bare percentage. On a small dataset, that context is what makes a number credible rather than misleading.