NTT Research Scientist Creates Model for Discontinuous Learning

By: NTT Research, Inc. via Business Wire

July 12, 2023 at 08:07 AM EDT

Dr. Gautam Reddy quantifies “a-ha” moments in paper published by National Academy of Sciences

NTT Research, Inc., a division of NTT (TYO:9432), today announced that a paper by Physics & Informatics (PHI) Lab Scientist Dr. Gautam Reddy, which makes a novel contribution to the understanding of learning, has been published in the Proceedings of the National Academy of Sciences (PNAS), one of the world's most cited and comprehensive multidisciplinary scientific journals. The paper, titled “A reinforcement-based mechanism for discontinuous learning,” uses data on spatial navigation of labyrinths by laboratory mice to provide a simple explanation for the “a-ha” moment of sudden insight. Dr. Reddy shows that the outcome of reinforcement learning (RL) algorithms, which operate gradually, can appear discontinuous for certain kinds of tasks. Specifically, this is true when the artificial RL agent learns the direct path from point A to point B in certain structured environments.

Labyrinths or mazes have a similar complex structure to knowledge graphs, which represent how humans mentally explore and navigate a knowledge database. A recent paper by neuroscientists at Caltech describes how mice navigating through a maze to find water show “rapid learning, sudden insight and efficient exploration.” Dr. Reddy analyzes the data in that paper and produces a learning rule to explain the phenomenon of sudden insight. He quantifies the relationship between learning rule, derived from biologically based learning discontinuities; learning curve, which represents the speed of reinforcement waves; and exploration bias, or the probability that a mouse will take a certain action at a certain intersection. The paper’s novel contribution is that RL algorithms can reproduce “a-ha” moment discontinuities in simulations.

“Congratulations to Dr. Reddy on his publication in this prestigious journal and for his creative discovery,” PHI Lab Director Yoshihisa Yamamoto said. “His important findings will advance our understanding of how we reason, solve problems and gain insight.”

Long-standing RL algorithms incrementally reinforce rewarding actions through accumulated experience. Yet, past and recent experiments find sharp discontinuities in learning. Dr. Reddy sheds light on the underlying mechanism.

“We show that the nonlinear dynamics of RL-based learning, together with continuous exploration, lead to discontinuous learning curves in tree-like structured environments,” Dr. Reddy said. “We develop a quantitative theory which explains the origin and highlights the generality of the phenomenon. The theory explains existing data and provides specific testable predictions.”

Looking ahead, Dr. Reddy plans to extend the experimental framework beyond tree-like graphs to topologies with loops. He also hopes to examine how an animal learns the neural representation of its location in a maze, a prerequisite to its evaluating differing actions at a particular location.

Along with these research initiatives, Dr. Reddy also serves as the NTT Research PHI Lab lead on a five-year joint research project with scientists at Harvard University to study animal neuro-responses, in the hope of informing future artificial intelligence (AI) systems. The PHI Lab’s mission is to uncover fundamental principles and novel technologies that advance information processing beyond state of the art. Its members and partners explore the space between quantum information science, neuroscience and optical technologies. In October 2020, PHI Lab Director Yamamoto co-authored a paper in Applied Physics Letters (APL) titled, “Coherent Ising Machines: Quantum optics and neural network perspectives,” which underscored the interdisciplinary nature of the PHI Lab’s pathbreaking research agenda.

In addition to Harvard, eight other universities have agreed to conduct joint research with the PHI Lab. These include the California Institute of Technology (Caltech), Cornell University, Massachusetts Institute of Technology (MIT), Notre Dame University, Stanford University, Swinburne University of Technology, the Tokyo Institute of Technology and the University of Michigan. The NASA Ames Research Center in Silicon Valley has also entered into a joint research agreement with the PHI Lab.

About NTT Research

NTT Research opened its offices in July 2019 as a new Silicon Valley startup to conduct basic research and advance technologies that promote positive change for humankind. Currently, three labs are housed at NTT Research facilities in Sunnyvale: the Physics and Informatics (PHI) Lab, the Cryptography and Information Security (CIS) Lab, and the Medical and Health Informatics (MEI) Lab. The organization aims to upgrade reality in three areas: 1) quantum information, neuroscience and photonics; 2) cryptographic and information security; and 3) medical and health informatics. NTT Research is part of NTT, a global technology and business solutions provider with an annual R&D budget of $3.6 billion.

NTT and the NTT logo are registered trademarks or trademarks of NIPPON TELEGRAPH AND TELEPHONE CORPORATION and/or its affiliates. All other referenced product names are trademarks of their respective owners. © 2023 NIPPON TELEGRAPH AND TELEPHONE CORPORATION

View source version on businesswire.com: https://www.businesswire.com/news/home/20230712487718/en/

Can one quantify an "a-ha" moment of sudden insight? New research from @NTTResearch's @Gautam_Reddy_N published in @theNASciences seeks to do just that by using data on the spatial navigation of labyrinths by laboratory mice. #UpgradeReality

Contacts

NTT Research Contact:

Chris Shaw

Chief Marketing Officer

NTT Research

+1-312-888-5412

chris.shaw@ntt-research.com

Media Contact:

Stephen Russell

Wireside Communications®

For NTT Research

+1-804-362-7484

srussell@wireside.com

About Us

About Cabling Installation & Maintenance

Contact Cabling Installation & Maintenance

Editorial

Advertising and Sponsorship Sales

Subscriptions and Memberships

Feedback/Problems

NTT Research Scientist Creates Model for Discontinuous Learning

Contacts

More News