Mind-Video
Thought-to-Video with Stable Diffusion.
It uses fMRI data from the brain to recreate video images.
The process includes two stages:
- Stage A uses Sparse: Coding Masked Brain Modeling (SC-MBM) on a large brain scan dataset.
- Stage B applies a Double: Conditioned Latent Diffusion Model (DC-LDM) to generate images from brain recordings.

🧵🧠We're witnessing incredible scientific progress in image & text reconstruction from fMRI nowadays. But what about reconstructing video from fMRI? Allow me to introduce our recent preprint: Mind-Video https://t.co/VL2KXz8o9Khttps://t.co/KyNtsCxDIJhttps://t.co/bhjz0PDlS6 pic.twitter.com/3g9fmujSu3
— Zijiao Chen (@ZijiaoC) May 22, 2023
