Improving The Memory of Neural Networks via Artificial intelligence (AI)
Using the human brain as inspiration, researchers at the University of Massachusetts Amherst and the Baylor College of Medicine have reported a way in improving neural network memory.
They write, "One solution would be to store previously encountered examples and revisit them when learning something new. Although such 'replay' or 'rehearsal' solves catastrophic forgetting," they add, "constantly retraining on all previously learned tasks is highly inefficient and the amount of data that would have to be stored becomes unmanageable quickly."
Naturally, humans are able to collect information and build on it through experiences. One way our brains remember old information is through a critical mechanism involving the ‘replay’ of neuronal activity. This mechanism is not found in AI neuronal networks. Findings were published in Nature Communications.
Learn more about neural networks:
Hava Siegelmann at UMass Amherst notes, "recognizing that replay in the brain does not store data." Rather, "the brain generates representations of memories at a high, more abstract level with no need to generate detailed memories."
"We propose a new, brain-inspired variant of replay in which internal or hidden representations are replayed that are generated by the network's own, context-modulated feedback connections,” explains first author and postdoctoral researcher Gido van de Ven. “Our method achieves state-of-the-art performance on challenging continual learning benchmarks without storing data, and it provides a novel model for abstract level replay in the brain."
Findings were published in Nature Communications.
Source: Science Daily
