AI creates pictures of what people are seeing by analysing brain scans

The images in the bottom row were recreated from the brain scans of someone looking at those in the top row.

Carissa Wong in New Scientist:

A tweak to a popular text-to-image-generating artificial intelligence allows it to turn brain signals directly into pictures. The system requires extensive training using bulky and costly imaging equipment, however, so everyday mind reading is a long way from reality.

Several research groups have previously generated images from brain signals using energy-intensive AI models that require fine-tuning of millions to billions of parameters.

Now, Shinji Nishimoto and Yu Takagi at Osaka University in Japan have developed a much simpler approach using Stable Diffusion, a text-to-image generator released by Stability AI in August 2022. Their new method involves thousands, rather than millions, of parameters.

More here.