Convolutional neural networks (CNNs) were developed with some inspirations from neuroscientific insights about the brain. Recently, several studies investigated the relationship between neural acitivities in the primate brain and internal representations in CNNs. We are working on the following problems for understanding and developing more biologically-plausible vision models:

1. Relationship between brain waves in the primate visual cortex and internal representation in CNNs

2. Deep learning model for decoding various visual features from neural activities in the brain.

This project is a collaboration with Hasegawa Lab at Department of Physiology, Niigata University Graduate School of Medical and Dental Sciences.

Primate can easily recognize objects using their vision systems even in complex and dynamic environments. When we see things, complex neural activities occur in both spatial and temporal domains. For understanding what visual features are represented by these activities, we need high-resolution recording methods and plausible computational models. We analyze spatiotemporal cortical responses (ECoG: electrocorticography) using internal representations in CNNs. ECoG enables us to simultaneously record neural activities from over 100 electrodes with sub-millisecond resolution.

Modeling electrocorticography signals on the macaque inferior temporal cortex in space, time and frequency domains using hierarchical visual features of a convolutional neural network
Society for Neuroscience 2016 Annual Meeting (SfN)
San Diego, US, November, 2016
Date, H., Kawasaki, K., Ozay, M., Hongo, T., Hasegawa, I., Okatani, T.
[poster]

Correspondence between the representations of convolutional neural networks and the activities in inferior temporal cortex measured by electrocorticography
Neuroscience 2016: The 39th Annual Meeting of the Japan Neuroscience Society (JNS)
Yokohama, Japan, July, 2016
Date, H., Kawasaki, K., Ozay, M., Hongo, T., Hasegawa, I., Okatani, T.
[poster]