Mind-Video

Thought-to-Video with Stable Diffusion.

It uses fMRI data from the brain to recreate video images.

The process includes two stages:

• Stage A uses Sparse: Coding Masked Brain Modeling (SC-MBM) on a large brain scan dataset.
• Stage B applies a Double: Conditioned Latent Diffusion Model (DC-LDM) to generate images from brain recordings.
  

  🧵🧠 We're witnessing incredible scientific progress in image & text reconstruction from fMRI nowadays. But what about reconstructing video from fMRI? Allow me to introduce our recent preprint: Mind-Video https://t.co/VL2KXz8o9Khttps://t.co/KyNtsCxDIJhttps://t.co/bhjz0PDlS6 pic.twitter.com/3g9fmujSu3

  — Zijiao Chen (@ZijiaoC) May 22, 2023