×
Lecture 4 - Learning with multiple representations
Axel Ngonga &
Barbara Hammer
Abstract
Recent machine learning models excelled excel for complex tasks. For example, AlphaFold, targets the prediction of protein 3D structures based in the sequence, resulting in the 2024 Nobel prize in chemistry. One key component of this network is its reliance on a combination of different representations, including structural components, homological information, and structural operations implementing equivariance. More generally, machine learning methods often go beyond classical representations such as tabular data, vectorial representations, or real-valued cost functions, and they harvest the advantage which comes with more complex representations. Learning wirh multiple representations (LMR) refers to ML methods that leverage several representations of the data and models or several formalizations of the learning task simultaneously, orchestrate them in a coordinated fashion, and use them in a synergistic way to solve a single problem. Taking advantage of the complementarity, the redundancy, and specific characteristics of different representations, LMR seeks to improve performance in comparison to methods operating on a single representation only. Typical examples of LMR include the combination of numerical and symbolic formalisms, the representation of data and models on different levels of abstraction, and the combination of different types of supervision of the learner.
Within the tutorial, we will explore two different examples of this framework: We will explore dimensionality reduction methodologies in machine learning and how this can be extended to incorporate specific auxiliary knowledge. This enables an inspection of the suitability of a chosen representation for a given task, and it can be extended towards a visualization of the function implemented by supervised deep networks. Moreover, it allows an inspection of the effect of finetuning on foundational deep models such as language embedding.
Furthermore, we will explore the impact of using multiple representations on concept learning. After a brief presentation of classical approaches based on reasoning in description logics, we begin the discussion of multiple representations by introducing concept retrieval via SPARQL. In particular, we show that concept retrieval can be reduced to querying graph databases when full forward chaining has been performed. We then give some brief insights into using embeddings for rapid but approximate reasoning. To this end, we show how ideas based on conjunctive queries in embeddings spaces can be exploited to develop a full approximate reasoner for expressive description logics. Insights into the tradeoff between explainability and scalability complete this section of the presentation.