Jacopo de Berardinis, PhD

In Music Information Retrieval (MIR) we develop algorithms for the automatic analysis of music. This can either operate at the musical content level (audio recordings, scores, etc.) and/or at the metadata level (tags, contextual descriptions, relationships, etc.), to automatically organise, search, and manipulate musical information. Examples include: music transcription, genre recognition, source separation, key detection, query by humming, etc.
Despite the general interest in MIR, my research focuses more on music structure analysis (detecting sections, phrases, motifs in a piece), music emotion recognition (predicting induced or expressed emotions from music) and music similarity. This encompasses the design of both computational models to address these tasks and any application built on top of them.

The recent rise of generative systems (e.g. DALL-E, MusicGen, etc.) has sparked commercial interest and opened up new opportunities for the creative sector. However, Generative AI is also creating serious implications and concerns for creative professionals: from the ethical dimensions and the ownership of the generated material, to the reuse of copyrighted material as training data for music models.
To address this challenge, my research focuses on responsible generative methods for computational creativity that are designed with the artists and for the artists, to preserve their active role in the creation process and give them proper recognition to the music that was contributed. I am also particularly interested in new evaluation methods and frameworks for music generation systems and their deployment in interactive workflows.

Music is known as a universal language that unites people, allowing us to manifest our feelings, recall memories and past experiences, and providing a highly expressive medium to unleash creativity. Among its benefits, music has been demonstrated to (1) improve mental and physical health (e.g. reducing stress, anxiety, depression, muscle pain, blood pressure), (2) increase memory capabilities, sleep quality, cognitive performance, and endurance, (3) while serving as an effective intervention to various disorders (Alzheimer's disease, dementia, etc.).
Current computational systems for music lack personalisation and interpretability - providing black-box solutions where humans are often seen as end users rather than active players. Here, I seek to creating personalised and engaging experiences where users are able to naturally search for music, navigate its related content and generate playlists depending on their interests, likings, mood, activities and goals, while interacting with the system to expand their awareness and knowledge behind this process.

Not only can knowledge be inferred from musical content, but informative and complementary types of knowledge are already available on the Web across different modalities (text, images, videos, links) and resources (MusicBrainz, Genius, YouTube, Songfacts, etc.). To date, this knowledge is scattered, poorly interconnected, and represented via different conventions and formats. Knowledge Graphs hold the potential to address this problem.
We are creating the largest interconnected Music Knowledge Graph by integrating music data from different sources and modalities. This provides the opportunity to study multimodality at scale and design Trustworthy AI applications for information retrieval, knowledge discovery, and generative machine learning.

In Machine Learning, the field of Representation Learning deals with designing computational models, paradigms, and training objectives to learn numerical representation (alias embeddings) that can be reused for a number of downstream tasks and applications. For example, in NLP, pre-trained text embeddings are used as starting point for semantic search, paraphrase detection, sentiment analysis, syntactic parsing, etc.
In the music domain, the goal is to learn general representations from musical content (audio, symbolic) and/or contextual information (e.g. metadata) to support Music Information Retrieval tasks (search, analysis, and recommendation). Here, I am interested in self-supervised methods to learn music embeddings from audio, symbolic, or multimodal data (e.g. text, images, videos, and Knowledge Graphs); with a focus on representations to address complex tasks requiring large amounts of expert annotations.