Drumparse failing on some tracks of the GMD
Drumparse fails to parse some tracks of the GMD. I tried to parse the 491 4/4 "beat" tracks. I ignored other time signatures, and "fill" tracks.
Script used to test all tracks
#!/bin/bash
set -e
export DRUMPARSE="$HOME/qparselib/build/drumparse"
export GRAMMAR="$HOME/grammar.wta"
export GENRE_COL=4
export TEMPO_COL=5
export BEAT_COL=6
export TS_COL=7
export MIDI_COL=8
if [ "$#" -ne 2 ]; then
echo "Usage: $0 GMD_DIRECTORY OUTPUT_FILE"
exit 1
fi
export groove_dir="$1"
export output_file="$2"
filter_and_append() {
row=$(echo -n "$1" | tr -d '\r' | tr -d '\n') # remove carriage return and newline
midi_file=$(echo "$row" | cut -d, -f$MIDI_COL)
tempo=$(echo "$row" | cut -d, -f$TEMPO_COL)
genre=$(echo "$row" | cut -d, -f$GENRE_COL | cut -d/ -f1) # if genre is "genre/subgenre", keep only "genre"
if bash -c "$DRUMPARSE -midi '$groove_dir/$midi_file' -schema '$GRAMMAR' -genre '$genre' -ts 4/4 -tempo '$tempo' -max -v0 > /dev/null 2>&1"; then
echo "$row,OK"
else
>&2 echo "Error in $midi_file"
echo "$row,ERROR"
fi
}
export -f filter_and_append
# keep only BEAT_COL==beat and TS_COL==4-4
echo "$(head -n 1 "$groove_dir/info.csv"),parsing_status" > "$output_file"
awk -F "," '($6 == "beat" && $7 == "4-4")' "$groove_dir/info.csv" | parallel --bar --progress -j+0 filter_and_append >> "$output_file"
In total, there are 47 tracks that lead to an error during the parsing, so around 47/491 = 9.6 %.
drummer1/session2/35_punk_128_beat_4-4.mid
drummer1/session2/62_punk_144_beat_4-4.mid
drummer1/session2/142_gospel_120_beat_4-4.mid
drummer1/session2/161_afrocuban-rhumba_110_beat_4-4.mid
drummer1/session2/180_rock_50_beat_4-4.mid
drummer1/session3/1_rock-prog_125_beat_4-4.mid
drummer1/session3/6_dance-disco_120_beat_4-4.mid
drummer1/session3/9_rock_135_beat_4-4.mid
drummer1/session3/13_rock_115_beat_4-4.mid
drummer1/session3/14_rock_94_beat_4-4.mid
drummer2/session1/2_rock_102_beat_4-4.mid
drummer2/session2/1_rock_130_beat_4-4.mid
drummer2/session2/10_rock_130_beat_4-4.mid
drummer3/session1/30_rock_120_beat_4-4.mid
drummer3/session1/47_rock_120_beat_4-4.mid
drummer3/session2/15_rock_100_beat_4-4.mid
drummer3/session2/18_rock_100_beat_4-4.mid
drummer3/session2/22_rock_92_beat_4-4.mid
drummer3/session2/26_hiphop_92_beat_4-4.mid
drummer3/session2/29_rock_92_beat_4-4.mid
drummer3/session2/42_rock_110_beat_4-4.mid
drummer3/session2/47_rock_110_beat_4-4.mid
drummer4/session1/7_blues-shuffle_134_beat_4-4.mid
drummer5/eval_session/7_pop-groove7_138_beat_4-4.mid
drummer5/session1/1_latin-brazilian-bossa_127_beat_4-4.mid
drummer5/session1/2_latin-brazilian-bossa_127_beat_4-4.mid
drummer5/session1/14_latin-brazilian-songo_112_beat_4-4.mid
drummer5/session1/19_middleeastern_126_beat_4-4.mid
drummer5/session1/20_latin-brazilian-baiao_110_beat_4-4.mid
drummer5/session2/4_reggae_141_beat_4-4.mid
drummer5/session2/8_neworleans-chacha_124_beat_4-4.mid
drummer5/session2/19_latin-brazilian-ijexa_108_beat_4-4.mid
drummer5/session2/23_latin-brazilian-songo_120_beat_4-4.mid
drummer7/session1/11_rock_93_beat_4-4.mid
drummer7/session1/13_funk_102_beat_4-4.mid
drummer7/session3/92_hiphop_70_beat_4-4.mid
drummer7/session3/120_rock_93_beat_4-4.mid
drummer7/session3/129_rock_93_beat_4-4.mid
drummer7/session3/151_soul_98_beat_4-4.mid
drummer7/session3/159_soul_98_beat_4-4.mid
drummer8/session1/5_hiphop_90_beat_4-4.mid
drummer8/session1/16_afrobeat_90_beat_4-4.mid
drummer8/session1/17_afrobeat_90_beat_4-4.mid
drummer8/session1/23_hiphop_70_beat_4-4.mid
drummer8/session1/28_rock_102_beat_4-4.mid
drummer8/session2/28_latin-bounce_106_beat_4-4.mid
drummer8/session2/41_highlife_126_beat_4-4.mid
The case of drummer3/session1/47_rock_120_beat_4-4.mid
is a bit special (see #89 for more details), as it results in:
munmap_chunk(): invalid pointer
Abandon (core dumped)
All the other ones results in the parsing error correctly handled by drumparse.
EDIT: The command that I initially used to parse the tempo from the filename was wrong when there were a digit in the style (like 1_funk-groove1_138_beat_4-4.mid
for example). You can find below the initial issue, which report more errors that there really is.
Initial issue
The command run for each track was the following:
midi_name=$(basename "$midi_file")
tempo=$(echo "$midi_name" | grep -oP "\d+" | sed -n "2p") # get tempo from name
genre=$(echo "$midi_name" | grep -oP "[a-z]+" | sed -n "1p") # get genre from name
drumparse -midi "$midi_file" -schema "$GRAMMAR" -genre "$genre" -ts 4/4 -tempo "$tempo" -max -v0
In total, there are 86 tracks that lead to an error during the parsing, so around 86/491 = 17.5 %.
Tracks that do not work
drummer1/eval_session/1_funk-groove1_138_beat_4-4.mid
drummer1/eval_session/2_funk-groove2_105_beat_4-4.mid
drummer1/eval_session/3_soul-groove3_86_beat_4-4.mid
drummer1/eval_session/4_soul-groove4_80_beat_4-4.mid
drummer1/eval_session/5_funk-groove5_84_beat_4-4.mid
drummer1/eval_session/6_hiphop-groove6_87_beat_4-4.mid
drummer1/eval_session/7_pop-groove7_138_beat_4-4.mid
drummer1/eval_session/8_rock-groove8_65_beat_4-4.mid
drummer1/eval_session/9_soul-groove9_105_beat_4-4.mid
drummer1/eval_session/10_soul-groove10_102_beat_4-4.mid
drummer1/session2/35_punk_128_beat_4-4.mid
drummer1/session2/62_punk_144_beat_4-4.mid
drummer1/session2/142_gospel_120_beat_4-4.mid
drummer1/session2/161_afrocuban-rhumba_110_beat_4-4.mid
drummer1/session2/180_rock_50_beat_4-4.mid
drummer1/session3/1_rock-prog_125_beat_4-4.mid
drummer1/session3/6_dance-disco_120_beat_4-4.mid
drummer1/session3/9_rock_135_beat_4-4.mid
drummer1/session3/13_rock_115_beat_4-4.mid
drummer1/session3/14_rock_94_beat_4-4.mid
drummer2/session1/2_rock_102_beat_4-4.mid
drummer2/session2/1_rock_130_beat_4-4.mid
drummer2/session2/10_rock_130_beat_4-4.mid
drummer3/session1/30_rock_120_beat_4-4.mid
drummer3/session1/47_rock_120_beat_4-4.mid
drummer3/session2/15_rock_100_beat_4-4.mid
drummer3/session2/18_rock_100_beat_4-4.mid
drummer3/session2/22_rock_92_beat_4-4.mid
drummer3/session2/26_hiphop_92_beat_4-4.mid
drummer3/session2/29_rock_92_beat_4-4.mid
drummer3/session2/42_rock_110_beat_4-4.mid
drummer3/session2/47_rock_110_beat_4-4.mid
drummer4/session1/7_blues-shuffle_134_beat_4-4.mid
drummer5/eval_session/1_funk-groove1_138_beat_4-4.mid
drummer5/eval_session/2_funk-groove2_105_beat_4-4.mid
drummer5/eval_session/3_soul-groove3_86_beat_4-4.mid
drummer5/eval_session/4_soul-groove4_80_beat_4-4.mid
drummer5/eval_session/5_funk-groove5_84_beat_4-4.mid
drummer5/eval_session/6_hiphop-groove6_87_beat_4-4.mid
drummer5/eval_session/7_pop-groove7_138_beat_4-4.mid
drummer5/eval_session/8_rock-groove8_65_beat_4-4.mid
drummer5/eval_session/9_soul-groove9_105_beat_4-4.mid
drummer5/eval_session/10_soul-groove10_102_beat_4-4.mid
drummer5/session1/1_latin-brazilian-bossa_127_beat_4-4.mid
drummer5/session1/2_latin-brazilian-bossa_127_beat_4-4.mid
drummer5/session1/14_latin-brazilian-songo_112_beat_4-4.mid
drummer5/session1/19_middleeastern_126_beat_4-4.mid
drummer5/session1/20_latin-brazilian-baiao_110_beat_4-4.mid
drummer5/session2/4_reggae_141_beat_4-4.mid
drummer5/session2/8_neworleans-chacha_124_beat_4-4.mid
drummer5/session2/19_latin-brazilian-ijexa_108_beat_4-4.mid
drummer5/session2/23_latin-brazilian-songo_120_beat_4-4.mid
drummer7/eval_session/1_funk-groove1_138_beat_4-4.mid
drummer7/eval_session/2_funk-groove2_105_beat_4-4.mid
drummer7/eval_session/3_soul-groove3_86_beat_4-4.mid
drummer7/eval_session/4_soul-groove4_80_beat_4-4.mid
drummer7/eval_session/5_funk-groove5_84_beat_4-4.mid
drummer7/eval_session/6_hiphop-groove6_87_beat_4-4.mid
drummer7/eval_session/7_pop-groove7_138_beat_4-4.mid
drummer7/eval_session/8_rock-groove8_65_beat_4-4.mid
drummer7/eval_session/9_soul-groove9_105_beat_4-4.mid
drummer7/eval_session/10_soul-groove10_102_beat_4-4.mid
drummer7/session1/11_rock_93_beat_4-4.mid
drummer7/session1/13_funk_102_beat_4-4.mid
drummer7/session3/92_hiphop_70_beat_4-4.mid
drummer7/session3/120_rock_93_beat_4-4.mid
drummer7/session3/129_rock_93_beat_4-4.mid
drummer7/session3/151_soul_98_beat_4-4.mid
drummer7/session3/159_soul_98_beat_4-4.mid
drummer8/eval_session/1_funk-groove1_138_beat_4-4.mid
drummer8/eval_session/2_funk-groove2_105_beat_4-4.mid
drummer8/eval_session/3_soul-groove3_86_beat_4-4.mid
drummer8/eval_session/4_soul-groove4_80_beat_4-4.mid
drummer8/eval_session/5_funk-groove5_84_beat_4-4.mid
drummer8/eval_session/6_hiphop-groove6_87_beat_4-4.mid
drummer8/eval_session/7_pop-groove7_138_beat_4-4.mid
drummer8/eval_session/8_rock-groove8_65_beat_4-4.mid
drummer8/eval_session/9_soul-groove9_105_beat_4-4.mid
drummer8/eval_session/10_soul-groove10_102_beat_4-4.mid
drummer8/session1/5_hiphop_90_beat_4-4.mid
drummer8/session1/16_afrobeat_90_beat_4-4.mid
drummer8/session1/17_afrobeat_90_beat_4-4.mid
drummer8/session1/23_hiphop_70_beat_4-4.mid
drummer8/session1/28_rock_102_beat_4-4.mid
drummer8/session2/28_latin-bounce_106_beat_4-4.mid
drummer8/session2/41_highlife_126_beat_4-4.mid