This repository provides the software that is necessary to produce MEMUK, a melodic music knoledge graph, i. e. a knowledge graph that contains information about the horizontal aspect of music. To represent music data, three different datasets containing symbolic digital music formats (MusicXML, MIDI and kern) have been integrated together. To enable interoperability across these three original datasets all files have been converted to the MEI file format. MEI stands for Music Encoding Initiative, an open source markup language to write and edit music in a digital way. In opposition to other formats, like MusicXML, MIDI and kern it allows to represent all kind of western musical notation, even including the ancient ones. MEMUK was then created from the MEI files by using the Polifonia Ontology Network.
-
In the folder Dataset_Selection there are the script used to select datasets useful for creating MEMUK. filtered_datasets.yaml contains the original list of datasets taken from the repository: https://github.com/ismir/mir-datasets
-
The folder melody contains the three datasets Melosol, ATEPP-1.1, Clean LMD in the partitions folder In the subfolder src is contained the software for bulk converting MIDI, kern and MusicXML files to MEI
-
kg-generation contains all necessary tools to convert MEI files into the RDF format used for Knowledge Graphs. query2.sparql is the query that allows this conversion thanks to (SPARQL Anything)[https://github.com/SPARQL-Anything/sparql.anything] tools.
-
The folder validation contains the code used to check the accuracy of the conversion from the different file tipes to mei.
In order to run the software, ensure that Python is installed on your system. Moreover, pip is essential for installing the necessary project modules: Music21, (PrettyMidi)[https://pypi.org/project/pretty_midi/], and (Converter21)[https://pypi.org/project/converter21/]. After setup is complete, the program can be started from the command line using this command:
python main.py /your/path/to/dataset <execute_function> All arguments are necessary to execute the code correctly. To avoid producing errors, the main dataset directory should comply with this format: datasetName/raw, where raw is the folder containing all subdirectories as well as files for conversion, and datasetName is linked with the appropriate function to assign titles and composers to each file.
As an <execute_function>, several functions are currently available and can be run by specifying their names in the command:
- convert_files: transforms single-track MIDI, MusicXML, or kern files into MEI files;
- split_midis: separates multi-track MIDI files into single-track MIDI files;
- correct_file: resolves metronome notation errors in Melosol files.
Additional functions may be developed according to user needs and incorporated into the list of executable functions in the main file.
Above an image explains the workflow for using and expanding the functionality of this software:
To create MEMUK on your local machine and test the software that genaretes it, SPARQL Anything requires a version equal or greater than JAVA 17. The SPARQL Anything executable jar can be found in the official release page on GitHub: https://github.com/SPARQL-Anything/sparql.anything/releases. In fact, two different version of SPARQL Anything could be downloaded: one is the sparq-anything-<version>.jar, and the other one is the sparql-anything-server-<version>.jar. The first version is the one used in this projects. Meanwhile, the second one is a server and its UI can be accessed at the address http://localhost:3000/sparql after running the jar file from the Command Line Interface (CLI) as follows:
$ java -jar sparql-anything-server-<version>.jar This UI can be very handy to get familiar with the SPARQL syntax and to try out queries. However, the first version is the one used in this project, and should be downloaded into the bin folder.
While a Docker installation is optional, it is highly recommended, both to provide a secure and consistent environment for running the code and for minimising the risk of compatibility issues caused by different programming setups. The Dockerfile takes care of installing the correct Python version and all the required libraries for a smooth code execution.
To run the MEMUK generation software first check that entrypoints contains the following commands:
#!/bin/bash
curl -L "https://github.com/SPARQL-Anything/sparql.anything/releases/download/0.9.0/sparql-anything-0.9.0.jar" -o ./bin/sa9.jar
python3 kg_generation.py ./your/input/path ./your/output/path \
--query_path query2.sparql \
--sparql_anything_path ./bin/sa9.jar \
--rdf_serialisation TTL \
--handle_error \
--save_metadataAlso ensure that the MEI files are located in your folder ./your/input/path inside of the kg-generation folder, otherwise no output will be generates.
The to run the software with docker use the following commands:
$ docker docker build -t memuk .
$ docker docker run -v /your/absolute/local/path:/srv/app/your/output/path/your/absolute/local/path is the folder on your pc where you want to save the RDF files in output (in .ttl file format).
This image shows how the Polifonia Ontology Network was used to model relationships between music notation elements.
Link to my thesis: Ontology-Based Data Integration: memuk, the melodic music knowledge graph
This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.