TIGP (SNHCC) -- Explainable AI and Large Language Models
- LecturerDr. Florian Dubost (Google)
Host: TIGP (SNHCC) - Time2023-05-15 (Mon.) 14:00 ~ 16:00
- LocationOnline
Live Stream
Abstract
The first part of the presentation covers explainable AI. Attention--or attribution--maps methods are methods designed to highlight regions of the model's input that were discriminative for its predictions. They can consequently be used for explaining the network's behavior. However, different attention maps methods can highlight different regions of the input, with sometimes contradictory explanations for a prediction. This effect is exacerbated when the training set is small. This indicates that either the model learned incorrect representations or that the attention maps methods did not accurately estimate the model's representations. In this presentation, we review state-of-the-art attention methods, and we present an unsupervised fine-tuning method that optimizes the consistency of attention maps and show that it improves both classification performance and the quality of attention maps.
The second part of the presentation covers large language models (LLM). We review the recent evolution of language models (GPT-4, Bard), compare their architecture nomenclature (encoder, decoder, encoder-decoder, auto-regressive decoder and encoder-decoder), and limitations.
The second part of the presentation covers large language models (LLM). We review the recent evolution of language models (GPT-4, Bard), compare their architecture nomenclature (encoder, decoder, encoder-decoder, auto-regressive decoder and encoder-decoder), and limitations.
BIO
Dr. Florian Dubost is a Senior Machine Learning Software Engineer at Google. His interest lies at the intersection of machine learning, computer vision and neurology. He previously developed machine learning algorithms for video processing, object detection, semantic segmentation with natural and medical images and other modalities including brain MRI, brain X-rays, or EEG. His methodological work focused on semi-supervised, weakly-supervised, and self-supervised learning and neural network interpretability.
Dr. Dubost holds a PhD degree in machine learning from Erasmus University Rotterdam, Netherlands, an MSc. in engineering from the Technical University of Munich, Germany, and another MSc. in engineering from Ecole Centrale Marseille, France. He carried out part of his PhD at Harvard Medical School, US, in collaboration with MIT, and a one-year postdoc at Stanford University. Dr. Dubost published over 30 peer-reviewed full articles in international journals and conferences. He is the recipient of several deep learning competition awards and co-supervised over 20 international students. He is a reviewer for machine learning and computer vision conferences such as CVPR, ECCV, AAAI and MICCAI and journals such as Nature Machine Intelligence, IEEE Transactions on Medical Imaging, Medical Image Analysis, Neuroimage, and IEEE Transactions on Biomedical Engineering. He has been on the program committees of conference workshops in NeurIPS and MICCAI and has co-organized multiple international deep learning competitions, such VALDO for MICCAI for brain lesion detection and segmentation. Dr. Dubost is also a junior editor at Frontiers.
Dr. Dubost holds a PhD degree in machine learning from Erasmus University Rotterdam, Netherlands, an MSc. in engineering from the Technical University of Munich, Germany, and another MSc. in engineering from Ecole Centrale Marseille, France. He carried out part of his PhD at Harvard Medical School, US, in collaboration with MIT, and a one-year postdoc at Stanford University. Dr. Dubost published over 30 peer-reviewed full articles in international journals and conferences. He is the recipient of several deep learning competition awards and co-supervised over 20 international students. He is a reviewer for machine learning and computer vision conferences such as CVPR, ECCV, AAAI and MICCAI and journals such as Nature Machine Intelligence, IEEE Transactions on Medical Imaging, Medical Image Analysis, Neuroimage, and IEEE Transactions on Biomedical Engineering. He has been on the program committees of conference workshops in NeurIPS and MICCAI and has co-organized multiple international deep learning competitions, such VALDO for MICCAI for brain lesion detection and segmentation. Dr. Dubost is also a junior editor at Frontiers.