<#>LM07 Project Update
Team: Darryl, Luo Yichi, Adeline Supervisors: Sandra, A/P Aw Paung Date: April 2026 Next Meeting: April 22, 2026, 7pm, Zoom
<##>What Are We Trying to Do?
People who cannot speak — due to ALS, stroke, or other conditions — need ways to communicate silently. We are building a brain-computer interface that decodes what a person is thinking, so they can communicate without speaking or moving.
The core challenge: Brain signals (EEG) are noisy and different for every person. A system trained on one patient fails completely on another. Calibration takes hours, and most systems can only decode single words — not natural phrases.
Our goal: Build a system that adapts quickly to new patients, transfers knowledge from easy signals to hard ones, and outputs full phrases.
<##>Our Approach
We combine three established techniques into one integrated system:
1. Signal Separation Split the brain signal into: what was said, how it was said, and who said it. We keep only the speech content — the part that should work across different people. This is the key to fast patient adaptation.
2. Cross-Mode Transfer Train on brain signals from speaking aloud (easy to decode). Then fine-tune on mouthing signals. Then adapt to imagining speech. Each step needs less data than training from scratch because the model already knows what speech looks like.
3. Phrase Decoding Decode full phrases, not single words. Model how sounds flow into each other naturally. This produces more natural output and is faster for real communication.
<##>Why Mouthing Is the Key Step
Recent research (Maghsoudi et al., 2026) made an important finding: brain patterns for mouthing and imagining are more similar to each other than either is to speaking aloud.
This means mouthing is not just a middle step — it is neurologically the right bridge between speaking and imagining. We can exploit this to make the transfer from speaking to imagining more effective.
Most published research only tests speaking vs imagining. Including mouthing as a dedicated intermediate step is a novel contribution of our project.
The transfer chain:
<##>Our Data
We have a significant advantage: previous FYP students collected brain signals from 20+ people, each doing all three types of speech tasks. This three-way dataset is rare in the literature.
Dataset details:
<##>What Success Looks Like
| Metric | What it means | |--------|---------------| | Accuracy | Does the model correctly identify the word or phrase? | | Generalisation | Does it work on people it was never trained on — without retraining? | | Transfer effectiveness | Does training on mouthing actually improve imagined speech decoding beyond training on imagining alone? | | Phrase quality | Does the output form natural, grammatically coherent phrases? | | Calibration speed | How few examples does the model need to work on a new person? |
We will compare our integrated system against partial baselines — to show that combining all three techniques outperforms any one alone.
<##>Semester Plan
| Period | What we will do | |--------|----------------| | Weeks 1-2 (Apr) | Literature review, confirm data access, finalise design | | Weeks 3-4 (May) | Data pipeline, train baseline model, get first results | | Weeks 5-8 (May-Jun) | Build signal separation, transfer chain, and phrase decoder | | Weeks 9-12 (Jun-Jul) | Full integration, evaluate on new subjects, measure all metrics | | Weeks 13-15 (Jul) | Write-up, prepare presentation, submission |
<##>What We Need From You
<##>Key References
<##>File Guide
| File | Contents |
|------|----------|
| LM07_Project_Update.pptx | Slide deck for today's meeting |
| LM07_Project_Update.md | This document |
| LM07_Pre-Meeting_Preparation.md | Full technical version with literature, methods, and detailed experimental design |