Qmidiroute patched for some GS use

Chat about anything related to the QuestStudios Archive, Classic PC Games, MIDI, Etc.

Moderator: Quest Studios Archive moderators

Post Reply
jaffa225man
Quest Studios Veteran
Quest Studios Veteran
Posts: 138
Joined: Mon Jul 13, 2015 6:26 pm
Location: Chippewa Falls, Wisconsin

Qmidiroute patched for some GS use

Post by jaffa225man » Mon Nov 11, 2019 12:44 pm

I bought a D-05 and wanted to experiment with it by playing my usual midi playlist. Sadly, I didn't know of a way to automatically forward certain program numbers to it. For instance, I wanted to send all piano (PC#1-6) to it, while using the SC-8850 for the rest. Qmidiroute can be used, but based on the channel, not the program number, and that would mean creating a new configuration for every midi file that changes where the pianos are. Anyway, since a solution wasn't immediately apparent, I decided to build this functionality into qmidiroute with another patch. It's finally done!

As far as features go, it now has this "Program Forward" map, where an input value (PC#, 0-127) may be matched to output to, in addition to the alsa port mapped to whatever device, a specific (or offset) channel, a specific value PC# (or range), and a fixed specific GS Variation (0-65) (or -1 can be used to just pass the input variations).

I also added a "SysEx Forward" map so that the SC-8820 can be on equal footing when used with the SC-8850. It takes no input arguments, and just outputs to the specified port.

SysEx Forward is also useful when used additionally with the "Drum Forward" map. Drum Forward takes an input channel range and matches, like Program Forward, with the PC#. For simplicity it just outputs to the same channel as was input. SysEx Forward is needed because GS SysEx can be used to alter any channel into becoming drums. Without SysEx Forward, drums may be forwarded to a channel other than 10 and would play as piano or whatever instrument happened to be there. Anyway, Drum Forward's output value (PC#) range can be set (0-127) to select the wanted Drums. Of course, the alsa port is set to specify the output device.

Qmidiroute has 2 virtual output ports by default, but if you need more, the amount can be given on the command line with -p, like so:
qmidiroute -p3

There you have it! The uses are many, but I've considered testing certain SC-8820 instruments along with the rest on the SC-8850. So far, I've just been playing with the D-05 and SC-8850, though. It's a neat perk to be able to "Program Forward" SC-8850 default instruments into more favorable variations, such as "Wide FreHrns", without having make modifications to every MIDI file. I'll include a test qmidiroute mapping that plays with that idea a bit, sending pianos 1-6 (0-5) to SC-8850 Part A as "St.Soft EP", and to Part B as "EP Legend", while French Horns are all changed to "Wide FreHrns" and play on both Part A and Part B. To set this up, I use aconnect to connect the SC-8850 parts to the virtual ports, in a terminal, like so ("aconnect -l" will list all ports available):
aconnect 129:1 32:0 ; aconnect 129:2 32:1
Also, you need to be sure your player is connected to qmidiroute's input (which in my example is 129:0), and that, likely, is done through the GUI of the player, but otherwise could be done with aconnect similarly to the above commands (where 14:0 is the player's output port):
aconnect 14:0 129:0

Again, I based this patch on the standard debian qmidiroute 0.4.0 source, but have also included one that will incrementally patch on top of my previous Note Off patch. Both retain the Note Off map functionality too, to fix the CM-32L/CM-64 overflow assign.

I hope it's useful to someone other than me. Enjoy:
qmidiroute-0.4.0-progch-port-forwarding-patches.7z
(15.5 KiB) Downloaded 351 times
Edit 2019-11-13 Found 2 bugs already, and fixed them. I'm sorry to those of you that already downloaded it, if you noticed what I just did: After my gs-reset.mid file, with the example .qmr file the non-variation SC-8850 Parts were being changed to variations 8, and 10). Also, the Drum Forward was stuttering because it was checking for changes in the variation. Drums not having a variation, it was never setting a value. What a silly mistake, but I guess that's what last minute changes lead to... more changes after tests are repeated.

Final Edit (I hope, for a long time) 2019-11-14 After the drums were working, I noticed and fixed a bug where the GS Map being set on drum channels would be retained when the user specifies one that exists in a more recent map, so just like with standard GS instruments, I decided to let the user choose which map (through their module's front panel buttons) by first sending the drum map bank as "0" (and rewriting all possibly conflicting changes) when the GUI-set output PC# ("Value") maps to any PC# higher than zero.

Edit 2020-03-31 Everything else still seems to be working and now I've patched a segmentation fault that happened when time-shifting audacious. It probably occurred under other conditions too, but for me, audacious made it obvious. Now, I can once again enjoy the very useful feature of audacious to be able to skip to anywhere in a track without playing the whole song (although, skipping SysEx should be avoided). If I'm not mistaken, this bug has existed for a very long time, and is in the original, unpatched, version of qmidiroute-0.4.0 and probably well before that.

Thanks if you're trying it out,

Luke

jaffa225man
Quest Studios Veteran
Quest Studios Veteran
Posts: 138
Joined: Mon Jul 13, 2015 6:26 pm
Location: Chippewa Falls, Wisconsin

Re: Qmidiroute patched for some GS use

Post by jaffa225man » Wed Apr 01, 2020 4:22 pm

Because my edit yesterday didn't trigger an updated date, I'm posting here for anyone else who has experienced the qmidiroute crashes I have. I think that latest patch solves them. It's much more stable for me anyway.

I hope it helps,

Luke

jaffa225man
Quest Studios Veteran
Quest Studios Veteran
Posts: 138
Joined: Mon Jul 13, 2015 6:26 pm
Location: Chippewa Falls, Wisconsin

Re: Qmidiroute patched for some GS use

Post by jaffa225man » Sat Nov 07, 2020 9:04 pm

At long last, I've upgraded everything I can think of, and they seem to be working well!
This changed enough, that I thought a version bump of .1 was justified, so this patch upgrades debian's qmidiroute-0.4.0 source (or any of my previous patched versions) to 0.5.0.
qmidiroute-0.4.0-to-0.5.0-patches.7z
(106.69 KiB) Downloaded 196 times
I added Channel & Polyphonic Aftertouch matching. It's a neat thing to play with: Polyphonic Aftertouch to Note On creates a fun effect, at least with instruments that naturally decay. :) Of course, you can, instead, use these to change any controller numbers into aftertouch, although most synths probably can trigger effects more easily from controller numbers than aftertouch of either type. You can also match "Note On" input to "Polyphonic Aftertouch" output to re-record aftertouch information with a velocity-only keyboard (but when doing this you may want to only match input velocities greater than zero, since my keyboard sends "Note Off" as "Note On" with a velocity of 0, and the aftertouch will turn off as soon as each key isn't held. On the other hand, if you do this, the aftertouch will never turn off, but if you play softly, it can be subtle).

The 65 variations I'd allowed sending to, previously, for the SC-8850 was too low to use many of the INTEGRA-7's banks, so I expanded them (as well as the banks) to the full MIDI range of 0-127. Along with those expansions, I also tried to align my design more with qmidiroute's, and so now Program & Drum Forwarding can match on input bank and variation, rather than just program number.

SysEx delays are now able to be set, differently (whether unmatched, forwarded, or edited).

SysEx Edit is my nicest addition, but its syntax is pretty particular. My example .qmr files should help anyone wanting to write their own. First, SysEx bytes (in hexadecimal characters without the leading "0x", i.e. just "f0" or "F0") are separated by spaces, while multiple messages are separated by commas ",".
A message may be matched from input to output as a list of constant bytes, or something more fancy may be done. If you want to change one message into more than one, you can include the message to be matched on input twice or more (each separated by a comma), and for each of those, put the desired message on output. The opposite works too. You can match multiple input messages and change each into the same output message. In the INTEGRA-7 qmr I'm including, I match both GS and GM1 resets to replace each with a GM2 reset. In input, I have "f0 41 10 42 12 40 00 7f 00 41 f7, f0 7e 7f 09 01 f7" (GS & GM1 reset), and in output there's "f0 7e 7f 09 03 f7, f0 7e 7f 09 03 f7" (GM2 reset for each input).
While that can be useful, better still is the ability to change a message with ranges. To get the MT-32's MASTER VOLUME to a nice level, I match 00-64 (0-100 in decimal) to 0-30 (0-48 in decimal), but since this changes the checksum, we also need some more special syntax. I got this idea from the Behringer BCR2000, but had to expand it for my use: "cks-1 05:08" specifies the type of checksum (1 being Roland, 2 or 3 for other types, as the BCR specifies - described on page 55 of "B-Control MIDI Implementation" by Mark van den Berg @ https://mountainutilities.eu/), along with the address start byte offset as hexadecimal (byte number 0 in the message is F0), separated by a colon (:) from the data start byte offset as hexadecimal (directly after the address in this case, numbered again from 0 on byte F0). So for the complete example of MASTER VOLUME editing, I have "f0 41 10 16 12 10 00 16 00-64 cks-1 05:08 f7" as input and "f0 41 10 16 12 10 00 16 00-30 cks-1 05:08 f7" as output. The checksum will be written to the offset "cks-#" (the necessary "##:##" isn't counted in this offset) is from the end f7/F7 byte. This way, a longer message including the MASTER VOLUME byte somewhere in the middle, will still write the checksum to the byte just before the f7/F7. If one wanted to set a constant MASTER VOLUME, that could be done with "f0 41 10 16 12 10 00 16 00-64 cks-1 05:08 f7" as input , and "f0 41 10 16 12 10 00 16 30-30 cks-1 05:08 f7" as output. If the input edit includes a range (-), the output must too, even if the output range creates a constant value like that. Either of the checksum (cks-#) or range (-) bytes may be at different offsets (from input to output), although there's no utility I've seen for that behavior.

SysEx Edit also has a rarely needed feature of removing sequentially duplicate SysEx messages (I encountered it in some ZUN SC-8850 SMFs). It's best to not enable it, in most cases, though as playback containing only GS reset SysEx messages will only output the first GS reset.

In debugging some playback on the INTEGRA-7, I decided to add the option to delay between bank and variation messages (this setting is found on the unmatched tab, even though it affects all). Although I solved the INTEGRA-7 issue I'd been having another way (holding back these messages until the program change is received so it is known where to send them), I read in Yamaha's "XG Format Music Data Production Recommendations" ("sisin") manual that XG requires a delay between these CC0 and CC32 messages (of "1/480", however I haven't researched if that's in seconds or some other unit, it's on page "5", or 9, of the version 1.15 pdf).

Another change I almost forgot to mention is that the check boxes in the Event Log portion of the window is now saved, so you can disable the logging messages for a headless device and save CPU usage.

Finally, and this is a big internal change, but hopefully unnoticeable: While trying to debug some anomalies with SC-8850 variations being set, I decided to port the entire input procEvents() loop from Qt to pthreads, just in case Qt's event loop wasn't processing input fast enough and missing some messages. With this change, I also used aseqdump's idea to put ALSA in non-blocking mode, in hopes it will help. In the end, I'm not sure if Qt's event loop being slow was the issue, but it doesn't hurt, and I was already using pthreads for the SysEx delays code, having wanted to learn it for a while.

Sadly, the addition of these match types makes save files incompatible with prior qmr format, so it's a good idea to keep the version you have saved your previous .qmr files in, to manually migrate values (with each version's window open) and save new .qmr files. I updated the ones I'm including again, though, so no worries if you've just used them. It's actually not too hard to examine the saves as text files and edit them accordingly, but it's probably not for the uninitiated. By releasing all these changes at once, I hope to avoid future .qmr file incompatibilities.

Finally, with the input bank/variation matching, I've included a few different capital tone fallback qmr files. (One just performs SC-55 fallback, although it's really to give a closer SC-55 approximation on an SC-55mkII since the original SC-55 does its own fallback. One makes the SC-55mkII "fallback" (well, forward and back, in this case) to "Fl.Key Click" to emulate the oldest SC-55. And the final one just performs SC-55mkII fallback so it never sees a variation it doesn't recognize, as it would've done if Yamaha hadn't had capital tone fallback patented then.)

Edit 2020-11-11: I decided to fix a bug due to not realizing ALSA may split SysEx into 256 byte chunks, so although it hasn't been downloaded yet, it would be best to wait until I patch it. I'll let you know when done. Thanks for your patience. :) -Done! (Edit 2020-11-14) Split SysEx messages are recombined so that internal checksum calculation will always work. It can be used to standardize MIDI file recording with arecordmidi, which would have recorded them split: now you can connect qmidiroute between MIDI input ports and your recording program to recombine each SysEx message to its original entirety. I also took the opportunity to speed up the input function substantially so the chances of lost SysEx are minimal (even compared to aseqdump somehow!). Audacious-3.4.3 (the last version I can find with the necessary AMIDI-Plug ALSA Backend, for some reason) may skip fast SysEx on a file's first play, but replaying it (even right away, before it finishes) seems to send all SysEx messages. However, aplaymidi, itself, is reliable and always seems to send all SysEx.

I hope you are as excited as I am about this upgrade.
Thanks,

Lucas
Last edited by jaffa225man on Tue Mar 16, 2021 3:46 am, edited 2 times in total.

jaffa225man
Quest Studios Veteran
Quest Studios Veteran
Posts: 138
Joined: Mon Jul 13, 2015 6:26 pm
Location: Chippewa Falls, Wisconsin

Re: Qmidiroute patched for some GS use

Post by jaffa225man » Mon Dec 28, 2020 1:17 am

I had a fairly hard time installing this on another debian machine of mine today, so I feel obligated to tell anyone trying this for themselves the steps I took:

In a terminal, go to the directory the patches were downloaded to

Code: Select all

cd ~/Downloads/
Become root:

Code: Select all

su
& enter your root password
or

Code: Select all

sudo su
& enter your user account password

Extract the patches to /usr/src with something like this:

Code: Select all

7z x -o/usr/src ./qmidiroute-0.4.0-to-0.5.0-patches.7z
Go where we want to extract the source

Code: Select all

cd /usr/src/
Download & extract debian's source for qmidiroute-0.4.0

Code: Select all

apt-get source qmidiroute
Just to keep the patch level obvious

Code: Select all

mv /usr/src/qmidiroute-0.{4,5}.0
Go into the build directory

Code: Select all

cd /usr/src/qmidiroute-0.5.0
Apply the patch

Code: Select all

patch -p1 </usr/src/qmidiroute-0.4.0-to-0.5.0-patches/qmidiroute-0.5.0.patch
Install some build requirements

Code: Select all

apt-get install qttools5-dev-tools
Sanitize the build system for your install

Code: Select all

aclocal
autoconf
automake
To be sure we're using qt5 for everything and rebuild the .moc files for your specific qt5 version

Code: Select all

export QT_SELECT=5
qmake -project
Configure it

Code: Select all

./configure --enable-qt5
Build it

Code: Select all

make
Install it

Code: Select all

make install
And that's all there is to it, but it sure took me some time to figure out everything! I hope not too many people stumbled onto the issues I had, but if so I'm sorry.

jaffa225man
Quest Studios Veteran
Quest Studios Veteran
Posts: 138
Joined: Mon Jul 13, 2015 6:26 pm
Location: Chippewa Falls, Wisconsin

Re: Qmidiroute patched for some GS use

Post by jaffa225man » Tue Mar 16, 2021 3:44 am

The patch version I uploaded today (0.5.1) adds and modifies about three main things:
qmidiroute-0.4.0-to-0.5.1-patches.7z
(113.34 KiB) Downloaded 197 times
1. SysEx Edit: If you have multiple offset values that do (or could) occur on the same message, you can now group them with a new syntax to denote it. That is, wrap each comma-separated message in parentheses. Because just describing it can seem complicated, I'll give the example of my first reason for it. MT-32 family Part Volumes are set through a range of semi-close addresses, so in at least one occurrence, they might be just in a single, long, SysEx message like so (from KQ4.SYX):
f0 41 10 16 12 03 00 00 01 0c 18 32 0c 00 01 00 64 06 00 00 00 00 00 00 01 30 18 32 0c 00 01 00 00 07 00 00 00 00 00 00 01 14 18 32 0c 00 01 00 64 0a 00 00 00 00 00 00 01 1d 18 32 0c 00 01 00 60 09 00 00 00 00 00 00 00 3b 18 32 0c 00 01 00 64 07 00 00 00 00 00 00 00 20 18 32 0c 00 01 00 64 05 00 00 00 00 00 00 00 24 18 32 0c 00 01 00 64 0b 00 00 00 00 00 00 01 3a 18 32 0c 00 01 00 50 0e 00 00 00 00 00 00 31 f7

I include the old Master Volume translation first, but then (contained in parentheses) lower all the parts' volumes. Here's what goes in SysEx Edit Input:
f0 41 10 16 12 10 00 16 00-64 cks-1 05:08 f7, (f0 41 10 16 12 03 00 08 00-64 cks-1 05:08 f7, f0 41 10 16 12 03 00 18 00-64 cks-1 05:08 f7, f0 41 10 16 12 03 00 28 00-64 cks-1 05:08 f7, f0 41 10 16 12 03 00 38 00-64 cks-1 05:08 f7, f0 41 10 16 12 03 00 48 00-64 cks-1 05:08 f7, f0 41 10 16 12 03 00 58 00-64 cks-1 05:08 f7, f0 41 10 16 12 03 00 68 00-64 cks-1 05:08 f7, f0 41 10 16 12 03 00 78 00-64 cks-1 05:08 f7, f0 41 10 16 12 03 01 08 00-64 cks-1 05:08 f7)

For this to work properly, it is important to match each comma-separated input message to the same number of comma-separated messages for the output, including putting the parentheses around the same number of messages for the output as was in the input:
f0 41 10 16 12 10 00 16 00-30 cks-1 05:08 f7, (f0 41 10 16 12 03 00 08 00-5a cks-1 05:08 f7, f0 41 10 16 12 03 00 18 00-5a cks-1 05:08 f7, f0 41 10 16 12 03 00 28 00-5a cks-1 05:08 f7, f0 41 10 16 12 03 00 38 00-5a cks-1 05:08 f7, f0 41 10 16 12 03 00 48 00-5a cks-1 05:08 f7, f0 41 10 16 12 03 00 58 00-5a cks-1 05:08 f7, f0 41 10 16 12 03 00 68 00-5a cks-1 05:08 f7, f0 41 10 16 12 03 00 78 00-5a cks-1 05:08 f7, f0 41 10 16 12 03 01 08 00-5a cks-1 05:08 f7)

That will remap Master Volume 0-100 to 0-48, and each Part Volume 0-100 to 0-90. In my real test, I had Part 2 outputting 5a-5a (a constant 90), so the muted timpani that Ari discovered wasn't. I'm happy to report that KQ4_5.mid played without clipping, after also adding a rule tab to eliminate all controller 7 messages, by sending them to an unused port. Here's the actual output when using the examples above (Part 2-timpani is muted, but otherwise it should serve help understanding):
f0 41 10 16 12 03 00 00 01 0c 18 32 0c 00 01 00 5a 06 00 00 00 00 00 00 01 30 18 32 0c 00 01 00 00 07 00 00 00 00 00 00 01 14 18 32 0c 00 01 00 5a 0a 00 00 00 00 00 00 01 1d 18 32 0c 00 01 00 56 09 00 00 00 00 00 00 00 3b 18 32 0c 00 01 00 5a 07 00 00 00 00 00 00 00 20 18 32 0c 00 01 00 5a 05 00 00 00 00 00 00 00 24 18 32 0c 00 01 00 5a 0b 00 00 00 00 00 00 01 3a 18 32 0c 00 01 00 48 0e 00 00 00 00 00 00 75 f7

2. Program & Drum Forward now has a Velocity Multiplier value to help in supporting non-GM/GS compliant instruments. If some just seem a little quiet or louder than you are used to, just increase or decrease the multiplier. 1.00 is the default. With it, output velocities remain the same as input. I thought this might help with some non-GS/GM quiet sounds on the INTEGRA-7, but haven't remembered which yet.

3. The Unmatched rules tab now has a check box that disables Note Off Velocity for all (matched and unmatched) messages. I just was thinking that it would be easy enough to implement now, and allows simpler consolidation for multi-device maps supporting the CM-32L/CM-64 at the same time as the Sound Canvas family. Leave it unchecked, and those weird messages will go through unscathed. I think they're a relic of glitches or flaws in devices, though, and would be interested to hear of some practical use for Note Off having velocities set.

Sadly, changes 2 and 3 break the save format again, but I've updated all my included .qmr files and added a few more I've had kicking around.

Expect to always use the build instructions above (viewtopic.php?f=4&t=17790&p=18542#p18511), because I don't know a way to automate it, although that's what autoconf and automake were designed for.

Thanks for trying it, and I hope all is well! :)

Post Reply