Manual¶
This page is a detailed guide for using ms3 for different tasks. It supposes you are working in an interactive Python interpreter such as IPython, Jupyter, Google Colab, or just the console.
Good to know¶
Terminology¶
Measure counts (MC) vs. measure numbers (MN)¶
Measure counts are strictly increasing numbers for all <measure> nodes in the score, regardless of their length. This information is crucial for correctly addressing positions in a MuseScore file and are shown in the software’s status bar. The first measure is always counted as 1 (following MuseScore’s convention), even if it is an anacrusis.
Measure numbers are the traditional way by which humans refer to positions in a score. They follow a couple of conventions which can be summarised as counting complete bars. Quite often, a complete bar (MN) can be made up of two <measure> nodes (MC). In the context of this library, score addressability needs to be maintained for humans and computers, therefore a mapping MC -> MN is preserved in the score information DataFrames.
Quarter Beats¶
A quarter beat always has the length of a quarter note. It is used as a standard unit to express positions and durations independently of the beat size suggested by the time signature (e.g. three eighths), and can be converted to a different beat size.
If the guidelines say “xy is expressed as/in quarter beats”,
it actually means “as fractions of a whole note”. So the duration of a half note, for example, is expressed
as 1/2
, and not as 2
(which but be the multiplier of quarter beats, or understanding quarter beats as unit).
This is simply a terminological convention to speak consistently of beat sizes.
Using the library¶
Parsing a single score¶
Import the library.
To parse a single score, we will use the class
ms3.Score
. We could import the whole library:>>> import ms3 >>> s = ms3.Score()
or simply import the class:
>>> from ms3 import Score >>> s = Score()
Locate the MuseScore 3 score you want to parse.
Make sure it is uncompressed. i.e. it has the extension
.mscx
and not.mscz
.Tip
MSCZ files are ZIP files containing the uncompressed MSCX. A later version of ms3 will be able to deal with MSCZ, too.
In the examples, we parse the annotated first page of Giovanni Battista Pergolesi’s influential Stabat Mater. The file is called
stabat.mscx
and can be downloaded from here (open link and keyCtrl + S
to save the file or right-click on the link toSave link as...
).Create a
ms3.Score
object.In the example, the MuseScore 3 file is located at
~/ms3/docs/stabat.mscx
so we can simply create the object and bind it to the variables
like so:>>> from ms3 import Score >>> s = Score('~/ms3/docs/stabat.mscx')
Inspect the object.
To have a look at the created object we can simply evoke its variable:
>>> s MuseScore file -------------- ~/ms3/docs/stabat.mscx Attached annotations -------------------- 48 labels: staff voice label_type 3 2 dcml 48
Parsing options¶
-
Score.
__init__
(musescore_file=None, infer_label_types=['dcml'], read_only=False, labels_cfg={}, logger_cfg={}, parser='bs4')[source] - Parameters
musescore_file (
str
, optional) – Path to the MuseScore file to be parsed.infer_label_types (
list
ordict
, optional) – Determine which label types are determined automatically. Defaults to [‘dcml’]. Pass[]
to infer only main types 0 - 3. Pass{'type_name': r"^(regular)(Expression)$"}
to callms3.Score.new_type()
.read_only (
bool
, optional) – Defaults toFalse
, meaning that the parsing is slower and uses more memory in order to allow for manipulations of the score, such as adding and deleting labels. Set toTrue
if you’re only extracting information.labels_cfg (
dict
) – Store a configuration dictionary to determine the output format of theAnnotations
object representing the currently attached annotations. SeeMSCX.labels_cfg
.logger_cfg (
dict
, optional) – The following options are available: ‘name’: LOGGER_NAME -> by default the logger name is based on the parsed file(s) ‘level’: {‘W’, ‘D’, ‘I’, ‘E’, ‘C’, ‘WARNING’, ‘DEBUG’, ‘INFO’, ‘ERROR’, ‘CRITICAL’} ‘file’: PATH_TO_LOGFILE to store all log messages under the given path.parser ('bs4', optional) – The only XML parser currently implemented is BeautifulSoup 4.
Parsing multiple scores¶
Import the library.
To parse multiple scores, we will use the class
ms3.Parse
. We could import the whole library:>>> import ms3 >>> p = ms3.Parse()
or simply import the class:
>>> from ms3 import Parse >>> p = Parse()
Locate the folder containing MuseScore files.
In this example, we are going to parse all files included in this older version of ms3’s Git repo which has been cloned into the home directory and therefore has the path
~/ms3
.Create a
ms3.Parse
objectThe object is created by calling it with the directory to scan, and bound to the variable
p
. By default, scores are grouped by the subdirectories they are in and one key is automatically created for each of them to access the files separately.>>> from ms3 import Parse >>> p = Parse('~/ms3') >>> p 10 files. KEY -> EXTENSIONS ----------------------- docs -> {'.mscx': 4} tests/MS3 -> {'.mscx': 6} No mscx files have been parsed.
As long as you always want to perform actions on all files, it may be convenient to assign a simple key. This might be also useful if you want to add several directories to the object using
p.add_dir()
:>>> p = Parse('~/ms3', key='test') >>> p.add_dir('~/other_folder', key='other') >>> p 237 files. KEY -> EXTENSIONS ------------------- test -> {'.mscx': 10} other -> {'.mscx': 227} No mscx files have been parsed.
Note that the same 10 files that were distributed over two keys in the previous example are now grouped under the key ‘test’.
Parse the scores.
In order to simply parse all registered MuseScore files, call
p.parse_mscx()
. Instead, you can pass the argumentkey
to parse only one (or several) selected group(s) to save time. The argumentlevel
controls how many log messages you see; here, it is set to ‘critical’ or ‘c’ to suppress all warnings:>>> p.parse_mscx(keys='test', level='c') >>> p KEY -> EXTENSIONS ------------------- test -> {'.mscx': 10} other -> {'.mscx': 227} 10/237 MSCX files have been parsed. 7 of them have annotations attached. KEY -> ANNOTATION LAYERS ------------------------- test -> staff voice label_type -> 2 1 dcml 167 -> 3 1 dcml 26 -> 2 dcml 48 -> 1 1 0 7 -> 3 166 -> dcml 568
As we can see, only the 10 files with the key ‘test’ were parsed and the table shows an overview of the counts of the included label types in the different notational layers (i.e. staff & voice). For example, the 7 files that include labels, have in their respective upper layers (staff 1, voice 1), 568 DCML harmony labels, 166 absolute chord labels (type 3) and 7 random strings (type 0) overall.
Extracting score information¶
Each of the DataFrames holding score information can be
automatically stored for every score. To select one or several aspects out of
{notes, measures, rests, notes_and_rests, events, labels, chords, expanded}
,
it is enough to pass the respective _folder
parameter to
store_lists()
distinguishing where to store the TSV
files. Additionally, the method accepts one _suffix
parameter per aspect,
i.e. a slug added to the respective filenames. If the parameter
simulate=True
is passed, no files are written but the file paths to be
created are returned. Since corpora might have quite diverse directory structures,
ms3 gives you various ways of specifying folders which will be explained in detail
in the following section.
Briefly, the rules for specifying the folders are as follows:
absolute folder (e.g.
~/labels
): Store all files in this particular folder without creating subfolders.relative folder starting with
./
or../
: relative folders are created “at the end” of the original subdirectory structure, i.e. relative to the MuseScore files.relative folder not starting with
./
or../
(e.g.rests
): relative folders are created at the top level (of the original directory or the specifiedroot_dir
) and the original subdirectory structure is replicated in each of them.
To see examples for the three possibilities, see the following section.
Specifying folders¶
Consider a two-level folder structure contained in the root directory .
which is the one passed to Parse
:
.
├── docs
│ ├── cujus.mscx
│ ├── o_quam.mscx
│ ├── quae.mscx
│ └── stabat.mscx
└── tests
└── MS3
├── 05_symph_fant.mscx
├── 76CASM34A33UM.mscx
├── BWV_0815.mscx
├── D973deutscher01.mscx
├── Did03M-Son_regina-1762-Sarti.mscx
└── K281-3.mscx
The first level contains the subdirectories docs (4 files) and tests (6 files in the subdirectory MS3). Now we look at the three different ways to specify folders for storing notes and measures.
Absolute Folders¶
When we specify absolute paths, all files are stored in the specified directories. In this example, the measures and notes are stored in the two specified subfolders of the home directory ~, regardless of the original subdirectory structure.
>>> p.store_lists(notes_folder='~/notes', measures_folder='~/measures')
~
├── measures
│ ├── 05_symph_fant.tsv
│ ├── 76CASM34A33UM.tsv
│ ├── BWV_0815.tsv
│ ├── cujus.tsv
│ ├── D973deutscher01.tsv
│ ├── Did03M-Son_regina-1762-Sarti.tsv
│ ├── K281-3.tsv
│ ├── o_quam.tsv
│ ├── quae.tsv
│ └── stabat.tsv
└── notes
├── 05_symph_fant.tsv
├── 76CASM34A33UM.tsv
├── BWV_0815.tsv
├── cujus.tsv
├── D973deutscher01.tsv
├── Did03M-Son_regina-1762-Sarti.tsv
├── K281-3.tsv
├── o_quam.tsv
├── quae.tsv
└── stabat.tsv
Relative Folders¶
In contrast, specifying relative folders recreates the original subdirectory structure.
There are two different possibilities for that. The first possibility is naming
relative folder names, meaning that the subdirectory structure (docs
and tests
)
is recreated in each of the folders:
>>> p.store_lists(root_dir='~/tsv', notes_folder='notes', measures_folder='measures')
~/tsv
├── measures
│ ├── docs
│ │ ├── cujus.tsv
│ │ ├── o_quam.tsv
│ │ ├── quae.tsv
│ │ └── stabat.tsv
│ └── tests
│ └── MS3
│ ├── 05_symph_fant.tsv
│ ├── 76CASM34A33UM.tsv
│ ├── BWV_0815.tsv
│ ├── D973deutscher01.tsv
│ ├── Did03M-Son_regina-1762-Sarti.tsv
│ └── K281-3.tsv
└── notes
├── docs
│ ├── cujus.tsv
│ ├── o_quam.tsv
│ ├── quae.tsv
│ └── stabat.tsv
└── tests
└── MS3
├── 05_symph_fant.tsv
├── 76CASM34A33UM.tsv
├── BWV_0815.tsv
├── D973deutscher01.tsv
├── Did03M-Son_regina-1762-Sarti.tsv
└── K281-3.tsv
Note that in this example, we have specified a root_dir
. Leaving this argument
out will create the same structure in the directory from which the Parse
object was created, i.e. the folder structure would be:
.
├── docs
├── measures
│ ├── docs
│ └── tests
│ └── MS3
├── notes
│ ├── docs
│ └── tests
│ └── MS3
└── tests
└── MS3
If, instead, you want to create the specified relative folders relative to each
MuseScore file’s location, specify them with an initial dot. ./
means
“relative to the original path” and ../
one level up from the original path.
To exemplify both:
>>> p.store_lists(root_dir='~/tsv', notes_folder='./notes', measures_folder='../measures')
~/tsv
├── docs
│ └── notes
│ ├── cujus.tsv
│ ├── o_quam.tsv
│ ├── quae.tsv
│ └── stabat.tsv
├── measures
│ ├── cujus.tsv
│ ├── o_quam.tsv
│ ├── quae.tsv
│ └── stabat.tsv
└── tests
├── measures
│ ├── 05_symph_fant.tsv
│ ├── 76CASM34A33UM.tsv
│ ├── BWV_0815.tsv
│ ├── D973deutscher01.tsv
│ ├── Did03M-Son_regina-1762-Sarti.tsv
│ └── K281-3.tsv
└── MS3
└── notes
├── 05_symph_fant.tsv
├── 76CASM34A33UM.tsv
├── BWV_0815.tsv
├── D973deutscher01.tsv
├── Did03M-Son_regina-1762-Sarti.tsv
└── K281-3.tsv
The notes
folders are created in directories where MuseScore files are located,
and the measures
folders one directory above, respectively. Leaving out the
root_dir
argument would lead to the same folder structure but in the directory
from which the Parse
object has been created. In a similar manner,
the arguments p.store_lists(notes_folder='.', measures_folder='.')
would create
the TSV files just next to the MuseScore files. However, this would lead to warnings
such as
Warning
The notes at ~/ms3/docs/cujus.tsv have been overwritten with measures.
In such a case we need to specify a suffix for at least one of both aspects:
p.store_lists(notes_folder='.', notes_suffix='_notes',
measures_folder='.', measures_suffix='_measures')
Examples¶
Before you are sure to have picked the right parameters for your desired output,
you can simply use the simulate=True
argument which lets you view the paths
without actually creating any files. In this variant, all aspects are stored each
in individual folders but with identical filenames:
>>> p = Parse('~/ms3/docs', key='pergo')
>>> p.parse_mscx()
>>> p.store_lists( notes_folder='./notes',
rests_folder='./rests',
notes_and_rests_folder='./notes_and_rests',
simulate=True
)
['~/ms3/docs/notes/cujus.tsv',
'~/ms3/docs/rests/cujus.tsv',
'~/ms3/docs/notes_and_rests/cujus.tsv',
'~/ms3/docs/notes/o_quam.tsv',
'~/ms3/docs/rests/o_quam.tsv',
'~/ms3/docs/notes_and_rests/o_quam.tsv',
'~/ms3/docs/notes/quae.tsv',
'~/ms3/docs/rests/quae.tsv',
'~/ms3/docs/notes_and_rests/quae.tsv',
'~/ms3/docs/notes/stabat.tsv',
'~/ms3/docs/rests/stabat.tsv',
'~/ms3/docs/notes_and_rests/stabat.tsv']
In this variant, the different ways of specifying folders are exemplified. To demonstrate all subtleties we parse the
same four files but this time from the perspective of ~/ms3
:
>>> p = Parse('~/ms3', folder_re='docs', key='pergo')
>>> p.parse_mscx()
>>> p.store_lists( notes_folder='./notes', # relative to ms3/docs
measures_folder='../measures', # one level up from ms3/docs
rests_folder='rests', # relative to the parsed directory
labels_folder='~/labels', # absolute folder
expanded_folder='~/labels', expanded_suffix='_exp',
simulate = True
)
['~/ms3/docs/notes/cujus.tsv',
'~/ms3/rests/docs/cujus.tsv',
'~/ms3/measures/cujus.tsv',
'~/labels/cujus.tsv',
'~/labels/cujus_exp.tsv',
'~/ms3/docs/notes/o_quam.tsv',
'~/ms3/rests/docs/o_quam.tsv',
'~/ms3/measures/o_quam.tsv',
'~/labels/o_quam.tsv',
'~/labels/o_quam_exp.tsv',
'~/ms3/docs/notes/quae.tsv',
'~/ms3/rests/docs/quae.tsv',
'~/ms3/measures/quae.tsv',
'~/labels/quae.tsv',
'~/labels/quae_exp.tsv',
'~/ms3/docs/notes/stabat.tsv',
'~/ms3/rests/docs/stabat.tsv',
'~/ms3/measures/stabat.tsv',
'~/labels/stabat.tsv',
'~/labels/stabat_exp.tsv']
Column Names¶
General Columns¶
mc Measure Counts¶
Measure count, identifier for the measure units in the XML encoding. Always starts with 1 for correspondence to MuseScore’s status bar.
mn Measure Numbers¶
Measure number, continuous count of complete measures as used in printed editions. Starts with 1 except for pieces beginning with a pickup measure, numbered as 0.
onsets¶
The value for onset
represents, expressed as quarter beats, a position in a measure where 0
corresponds to the earliest possible position (in most cases beat 1), and some other fraction corresponds to an onset’s offset from 0
.
Quarter beats can be converted to beats, e.g. to half beats or dotted eighth beats;
However, the operation may rely on the value of mc_offset.
Tip
When loading a table from a file, it is recommended to parse the text of this
column with fractions.Fraction
to be able to calculate with the values.
MS3 does this automatically.
Measures¶
act_dur Actual duration of a measure¶
The value of act_dur
in most cases equals the time signature, expressed as a fraction; meaning for example that
a “normal” measure in 6/8 has act_dur = 3/4
. If the measure has an irregular length, for example a pickup measure
of length 1/8, would have act_dur = 1/8
.
The value of act_dur
plays an important part in inferring MNs
from MCs. See also the columns dont_count and numbering_offset.
barline¶
The column barline
encodes information about the measure’s final bar line.
breaks¶
The column breaks
may include three different values: {'line', 'page', 'section'}
which represent the different
breaks types. In the case of section breaks, MuseScore
dont_count Measures excluded from bar count¶
This is a binary value that corresponds to MuseScore’s setting Exclude from bar count
from the Bar Properties
menu.
The value is 1
for pickup bars, second MCs of divided MNs and some volta measures,
and NaN
otherwise.
keysig Key Signatures¶
The feature keysig
represents the key signature of a particular measure.
It is an integer which, if positive, represents the number of sharps, and if
negative, the number of flats. E.g.: 3
: three sharps, -2
: two flats,
0
: no accidentals.
mc_offset Offset of a MC¶
The column mc_offset
, in most cases, has the value 0
because it expresses the deviation of this MC’s
onset 0
(beginning of the MC)
from beat 1 of the corresponding MN. If the value is a fraction > 0, it means that this MC is part of a MN which is
composed of at least two MCs, and it expresses the current MC’s offset in terms of the duration of all (usually 1) preceding MCs
which are also part of the corresponding MN. In the standard case that one MN would be split in two MCs, the first MC
would have mc_offset = 0
, and the second one mc_offset = the previous MC's
act_dur .
next¶
Every cell in this column has at least one integer, namely the MC of the subsequent bar, or -1
in the cast of the last.
In the case of repetitions, measures can have more than one subsequent MCs, in which case the integers are separated by
', '
.
The column is used for checking whether irregular measure lengths even themselves out because otherwise the inferred MNs might be wrong. Also, it is needed for MS3’s unfold repeats functionality (TODO).
Developers
Within MS3, the next
column holds tuples, which MS3 should normally store as strings without parenthesis. For
example, the tuple (17, 1)
is stored as '17, 1'
. However, users might have extracted and stored a raw DataFrame
from a Score
object and MS3 needs to handle both formats.
numbering_offset Offsetting MNs¶
MuseScore’s measure number counter can be reset at a given MC by using the Add to bar number
setting from the
Bar Properties
menu. If numbering_offset
≠ 0, the counting offset is added to the current MN and all subsequent
MNs are inferred accordingly.
Scores which include several pieces (e.g. in variations or a suite),
sometimes, instead of using section breaks, use numbering_offset
to simulate a restart for counting
MNs at every new section. This leads to ambiguous MNs.
repeats¶
The column repeats
indicates the presence of repeat signs and can have the values
{'start', 'end', 'startend', 'firstMeasure', 'lastMeasure'}
. MS3 performs a test on the
repeat signs’ plausibility and throws warnings when some inference is required for this.
The repeats
column needs to have the correct repeat sign structure in order to have a correct next
column which, in return, is required for MS3’s unfolding repetitions functionality. (TODO)
timesig Time Signatures¶
The time signature timesig
of a particular measure is expressed as a string, e.g. '2/2'
.
The actual duration of a measure can deviate from the time signature for notational reasons: For example,
a pickup bar could have an actual duration of 1/4
but still be part of a '3/8'
meter, which usually
has an actual duration of 3/8
.
Tip
When loading a table from a file, time signatures are not parsed as fractions because then both
'2/2'
and '4/4'
, for example, would become 1
.
volta¶
In the case of first and second (third etc.) endings, this column holds the number of every “bracket”, “house”, or volta, which should increase from 1. This is required for MS3’s unfold repeats function (TODO) to work.
The MNs for all voltas except those of the first one need to be amended to match those of the first volta. In the case where these voltas have only one measure each, the dont_count option suffices. If the voltas have more than one measure, the numbering_offset setting needs to be used.