The One Learning Algorithm Hypothesis: Towards Universal ML and DL models and architectures

Prof. John S. Baras

Institute for Systems Research
University of Maryland College Park

The talk will be hybrid and will also be availably via Zoom:

https://tum-conf.zoom.us/j/62533387681?pwd=eVp5dU52Y2k0OVAyYzNwQXZEbUs2QT09

Meeting ID: 625 3338 7681
Passcode: 316759

Abstract:

We revisit the “One Learning Algorithm Hypothesis” of Andrew Ng (Google Brain) according to which the brain of higher level animals and of humans processes and perceives sensory data (vision, sound, haptics) with the same abstract algorithmic architecture. We develop models, based on our earlier work on automatic target recognition with radar and other sensors, face recognition and image classification, which employ a multi-resolution preprocessor, followed by a group-invariance based feature extractor, followed by a machine learning (ML) module that employs the two fundamental algorithms of Kohonen Learning Vector Quantization (LVQ), for supervised learning, and Self-Organizing Map (SOM), for unsupervised learning. In addition the model and algorithms utilize a “global” feedback from the output of the overall system back to the feature extractor and to the multiresolution preprocessor. We first summarize briefly our older results with such algorithms and their remarkable, domain agnostic, performance on various applications. We then provide our recent results on the mathematical analysis of the resulting Tree Structure Learning Vector Quantization (TSLVQ) ML architecture and algorithms. We introduce and integrate Deterministic Annealing (DA) with our older architecture and demonstrate the resulting tremendous reduction in data required for learning and application. The new algorithms allow even on-line progressive learning. We utilize Bregman divergences as dissimilarity measures, which allows us to provide direct transition from “dissimilarity distance” to probability of error, which cannot be achieved with the commonly used metric-based dissimilarity measures. We show that many DL network architectures can be mapped to this “universal” architecture. We show that the integrated algorithm converges to the true Bayes decision surface, albeit with variable resolution at various parts of it, as required. The latter brings a tight connection to integrated hypothesis testing with compressed data. We demonstrate the results in various applications in communication and control systems and close with future directions and extensions.

Biography:

John S. Baras is a Distinguished University Professor, holding the Lockheed Martin Chair in Systems Engineering and a Permanent Joint Appointment with the Institute for Systems Research (ISR) and the ECE Department at the University of Maryland College Park (UMD). He received his Ph.D. degree in Applied Mathematics from Harvard University, in 1973, and he has been with UMD since then. From 1985 to 1991, he was the Founding Director of the ISR. Since 1992, he has been the Director of the Maryland Center for Hybrid Networks (HYNET), which he co-founded. He is a Fellow of IEEE (Life), SIAM, AAAS, NAI, IFAC, AMS, AIAA, Member of the National Academy of Inventors (NAI) and a Foreign Member of the Royal Swedish Academy of Engineering Sciences (IVA). Major honors and awards include the 1980 George Axelby Award from the IEEE Control Systems Society, the 2006 Leonard Abraham Prize from the IEEE Communications Society, the 2017 IEEE Simon Ramo Medal, the 2017 AACC Richard E. Bellman Control Heritage Award, and the 2018 AIAA Aerospace Communications Award. In 2016 he was inducted in the University of Maryland A. J. Clark School of Engineering Innovation Hall of Fame. In June 2018 he was awarded a Doctorate Honoris Causa by his alma mater the National Technical University of Athens, Greece. His research interests include systems, control, optimization, autonomy, machine learning, artificial intelligence, communication networks, applied mathematics, signal processing and understanding, robotics, computing systems, formal methods and logic, network security and trust, systems biology, healthcare management, model-based systems engineering. He has been awarded twenty patents and honored with many awards as innovator and leader of economic development.