Missing notes after live MIDI recording

Discussions about our next-generation scoring application, Dorico.
GregS
Junior Member
Posts: 118
Joined: Fri Jul 03, 2020 1:39 pm
Contact:

Missing notes after live MIDI recording

Post by GregS »

Just trying Dorico Elements 3.5 for the first time, and I'm experiencing a strange problem - I'm noticing that notes are sometimes missing, after recording MIDI live. I have tried it with MIDI Thru enabled and disabled - same either way.
With Thru enabled, I disable Local Control on my keyboard, and DURING the recording, I do not observe any missing notes.
I'm actually noticing similar symptoms with another notation application, so it's looking like a system problem.
On the other hand, a DAW (Reaper) is working fine. (although in order to get the best quantization results in Reaper, I have to delay all recorded MIDI events by about 80ms, and THEN do the quantization - that seems like a huge delay to have to compensate for! I am using Reaper's inbuilt metronome during the recording).

Setup: Dell XPS 15 9560, Ultimate performance plan
Yamaha P-515 digital piano, connected via USB
I am using pure MIDI - no VSTs.

Question: with MIDI Thru enabled, does the MIDI driver do the echoing, or is it done by the application? (I suspect the latter)
Elements 3.5.11.1054, Windows 10

User avatar
Daniel at Steinberg
Moderator
Posts: 18228
Joined: Mon Nov 12, 2012 10:35 am
Contact:

Re: Missing notes after live MIDI recording

Post by Daniel at Steinberg »

Dorico is responsible for the echoing when MIDI thru is enabled.

I suggest you try the troubleshooting steps shown here that will hopefully reveal whether there's anything untoward going on in the MIDI stream.

GregS
Junior Member
Posts: 118
Joined: Fri Jul 03, 2020 1:39 pm
Contact:

Re: Missing notes after live MIDI recording

Post by GregS »

Thanks. I've already had a look at the MIDI stream a bit in a standalone instrument - Pianoteq, and I can't see anything wrong.
What I really need to do is capture the raw MIDI that Dorico received, wait for the problem to occur, and then inspect THAT MIDI stream. (I know MIDI-OX can be used for that type of thing, but I've never tried it yet)
It would be good if Dorico itself had a raw MIDI dump facility actually.
Elements 3.5.11.1054, Windows 10

GregS
Junior Member
Posts: 118
Joined: Fri Jul 03, 2020 1:39 pm
Contact:

Re: Missing notes after live MIDI recording

Post by GregS »

Interesting! It's working wonderfully with MIDI-OX running, but as soon as I close down MIDI-OX, I get missing notes again.
I notice "Active Sense" messages are being transmitted by the keyboard - is it possible that MIDI-OX is filtering these out, and those messages are causing the problem for Dorico? (I certainly haven't tried to get MIDI-OX to filter anything out yet)
Elements 3.5.11.1054, Windows 10

Rob Tuley
Grand Senior Member
Posts: 3910
Joined: Fri May 20, 2016 12:41 am

Re: Missing notes after live MIDI recording

Post by Rob Tuley »

Active sense messages are weird.

If a transmitting device using active sense doesn't send any other MIDI data for 0.3 seconds, it sends an active sense message to say "I'm still working OK". That seems harmless enough, but....

If a receiving device that handles active sense messages receives one, it assumes there will be more coming from the same source, and if it doesn't receive another message in the expected time frame it terminates any notes previously sent that are still playing, assuming the device has been switched off so it will never get proper "note off" messages to end them!

That might account for your random missing notes, if the timing is going wrong somehow.

Usually the best thing to do with active sense is disable it. There should be a setting on your keyboard do to that.

User avatar
PaulWalmsley
Steinberg Employee
Posts: 2060
Joined: Tue May 17, 2016 9:24 pm
Location: Steinberg, London
Contact:

Re: Missing notes after live MIDI recording

Post by PaulWalmsley »

Dorico does have a mechanism for saving a copy of the MIDI that you record, so hopefully that means you can recover the things you recorded, but also if you have a case that isn't working, you can send to us. Open a File Explorer window and type %TEMP% in the path bar, then navigate to the Dorico 3.5 directory. There should be a subdirectory called "MIDI captures". What you can then do is create a new solo piano project in Dorico, add a number of empty bars and then switch to Play Mode. You can drag and drop the MIDI file onto the lane in Play Mode.

Given that you had a very high latency in Reaper that suggests that you may need to set a similar latency value in Dorico too, in Preferences > Play > Recording > MIDI input latency compensation. Try recording just playing on the beat and tweaking the value until the notes in play mode are aligned to the barline.
Architect & Developer - Steinberg London

GregS
Junior Member
Posts: 118
Joined: Fri Jul 03, 2020 1:39 pm
Contact:

Re: Missing notes after live MIDI recording

Post by GregS »

Thanks for the rapid assistance everyone.
As far as I can tell, I can't disable Active Sense transmission on the P-515 - I've tried disabling SYSEX and "System Realtime" transmission, but the messages are still sent.
It appears that in the worst case, I'll have to run a MIDI filter (such as MIDI-OX) - that is an acceptable workaround for me.
Aside from this problem, I like the look of Dorico so far, and the documentation is excellent.
Elements 3.5.11.1054, Windows 10

User avatar
PaulWalmsley
Steinberg Employee
Posts: 2060
Joined: Tue May 17, 2016 9:24 pm
Location: Steinberg, London
Contact:

Re: Missing notes after live MIDI recording

Post by PaulWalmsley »

Dorico does filter out sysex and active sensing so it _shouldn't_ be a problem. If the capture files that I mention above don't have any missing notes then that suggests that the MIDI side is fine, and it's just a matter of setting the latency correctly.
Architect & Developer - Steinberg London

GregS
Junior Member
Posts: 118
Joined: Fri Jul 03, 2020 1:39 pm
Contact:

Re: Missing notes after live MIDI recording

Post by GregS »

Ok I'll follow your instructions re the capture files and send to Support. Just btw, the notes are missing even when I play back using "Played durations".
I don't think I'll need nearly as much (if any) latency compensation in Dorico, but I will have a look at that closely.
Elements 3.5.11.1054, Windows 10

GregS
Junior Member
Posts: 118
Joined: Fri Jul 03, 2020 1:39 pm
Contact:

Re: Missing notes after live MIDI recording

Post by GregS »

I have uploaded a project with one missing note. In Reaper, the note IS present. The timing looks good to me - I do not feel that I need to add any MIDI latency compensation in Dorico.
If I drag the MIDI file onto the lane in a new Dorico project, the offending note is still missing.
The missing note is NN 59, in measure 9, beat 2. (the pattern that I'm playing is obvious)
Looks like I have to contact my Steinberg distributor for official support, so I'll just use this forum for the time being.
https://drive.google.com/drive/folders/ ... sp=sharing
Elements 3.5.11.1054, Windows 10

User avatar
PaulWalmsley
Steinberg Employee
Posts: 2060
Joined: Tue May 17, 2016 9:24 pm
Location: Steinberg, London
Contact:

Re: Missing notes after live MIDI recording

Post by PaulWalmsley »

This does appear to be a latency issue - I've opened the MIDI file in Cubase and all the notes look to be about 30ms early. I tried shifting them back by 30ms and saving the file, then loaded that into Dorico and all the notes now appear. So whatever latency setting you currently have in Dorico, try reducing it by about 30ms. I've attached the adjusted MIDI file.
Clipboard Image.jpg
(29.86 KiB) Not downloaded yet
There is a bug here, which I'll log, but experimenting with getting the correct latency should let you get better results.

Regarding support, generally you will get very good support on the forum - there's a very good community here. Also, I wrote the MIDI import feature, so this report has got to me a lot faster than if you had gone through your distributor.
Attachments
MIDI Capture.zip
(1.36 KiB) Downloaded 3 times
Architect & Developer - Steinberg London

GregS
Junior Member
Posts: 118
Joined: Fri Jul 03, 2020 1:39 pm
Contact:

Re: Missing notes after live MIDI recording

Post by GregS »

Thanks Paul - excellent!
I haven't entered anything in the MIDI latency field yet - it's at zero. I haven't tried yet, but given that the notes are early, does that mean I need to enter a *negative* value? (will it accept that?)
Elements 3.5.11.1054, Windows 10

User avatar
dankreider
Grand Senior Member
Posts: 5561
Joined: Thu Jun 28, 2018 3:45 pm
Contact:

Re: Missing notes after live MIDI recording

Post by dankreider »

Yes, the field accepts a negative value. But if the notes are early, use a positive value. The field represents the amount of latency that a note needs to align to the beat.
Dan Kreider
hymnworks.com
dankreider.com/dorico-resources
Dorico 3.5.11, Windows 10

User avatar
PaulWalmsley
Steinberg Employee
Posts: 2060
Joined: Tue May 17, 2016 9:24 pm
Location: Steinberg, London
Contact:

Re: Missing notes after live MIDI recording

Post by PaulWalmsley »

See the earlier post about this: viewtopic.php?f=246&t=172681
Architect & Developer - Steinberg London

GregS
Junior Member
Posts: 118
Joined: Fri Jul 03, 2020 1:39 pm
Contact:

Re: Missing notes after live MIDI recording

Post by GregS »

Adding the 30ms delay has indeed fixed the problem - thanks. Just by the way, the competitor's app doesn't have such a setting.
Elements 3.5.11.1054, Windows 10

Rob Tuley
Grand Senior Member
Posts: 3910
Joined: Fri May 20, 2016 12:41 am

Re: Missing notes after live MIDI recording

Post by Rob Tuley »

GregS wrote:
Fri Jul 03, 2020 3:41 pm
As far as I can tell, I can't disable Active Sense transmission on the P-515
Looking at the documentation, there doesn't seem to be any way to disable Active Sense.

It's a bit odd that a modern keyboard is still generating it. It was useful back in the day when MIDI cables were for one-directional data transfer only and had non-locking connectors, and you had a rats nest of identical and unlabeled cables connecting your MIDI devices.

If you were playing live on stage and somebody tripped over a cable and pulled out a plug, at least you didn't have some notes left stuck on while you were trying to guess which synth in the rack was playing them, because Active Sense killed them as soon as it noticed the cable was disconnected.

But with MIDI over USB cables and wireless data links it is probably past its sell-by date now.

GregS
Junior Member
Posts: 118
Joined: Fri Jul 03, 2020 1:39 pm
Contact:

Re: Missing notes after live MIDI recording

Post by GregS »

Paul: I'm still getting missing notes - they occur more often playing normal music, because I'm not playing as accurately.
I've gone through that procedure you referred to, and I've bumped the latency up to 40ms. I've also tried varying it up & down, but I always get missing notes, and of course, if I vary it too much, the notes then quantize on to the wrong beats as well. You said there's a bug, so I'll see how I go when that's been addressed. In the meantime this isn't preventing me from working.
By the way, I suspect as I get more skilled with the package (I'm a raw beginner), I'll do less MIDI input, and more keyboard & mouse input. However, I'd like to get the MIDI input working as well as it can.
I can upload more test recordings if you like.
Elements 3.5.11.1054, Windows 10

User avatar
PaulWalmsley
Steinberg Employee
Posts: 2060
Joined: Tue May 17, 2016 9:24 pm
Location: Steinberg, London
Contact:

Re: Missing notes after live MIDI recording

Post by PaulWalmsley »

If you have any other MIDI files that have missing notes, then please do attach them, along with a note to say which bar contains the missing notes, and I can add to the list to look at.
Architect & Developer - Steinberg London

GregS
Junior Member
Posts: 118
Joined: Fri Jul 03, 2020 1:39 pm
Contact:

Re: Missing notes after live MIDI recording

Post by GregS »

I've found the problem I think. My keyboard has a tri-sensor action, which is meant to simulate the double-escapement of a grand piano. In what is, I think, a pretty ugly hack, it sends contiguous Note-Ons, for repeats where the key is not lifted up far enough. (i.e - for legato repeats, where the dampers don't fall back on the strings in between)
When the key IS lifted fully, it will then spit out all the outstanding Note-Offs in a burst.
Yamaha has done it like this for a long time I believe.

I don't see any way to turn this behaviour off.

I'll be expecting a fix for this by tomorrow. Thanks. 🤣
Elements 3.5.11.1054, Windows 10

GregS
Junior Member
Posts: 118
Joined: Fri Jul 03, 2020 1:39 pm
Contact:

Re: Missing notes after live MIDI recording

Post by GregS »

No - it's not the *only* problem. Dorico is definitely not detecting these partial repeats, but it's still missing normal notes.
I think it's best if I just wait for you to fix the acknowledged bug first, because I'm confident that the problem is simply my sloppy playing, so perhaps when it's fixed, I'll see the missing notes, but not in my desired rhythmic position - and that will then be my problem.

The partial repeat thing won't occur very often I don't think - I don't do that kind of repeat very often.

EDIT: just btw, when I tried a Roland FP60, which also has a tri-sensor action, I noticed that it transmitted back to back Note-Off & Note-On pairs for legato repeats. This is much easier for DAWs to handle, but there is actually some logic in how Yamaha do it, IMHO.
Elements 3.5.11.1054, Windows 10

User avatar
PaulWalmsley
Steinberg Employee
Posts: 2060
Joined: Tue May 17, 2016 9:24 pm
Location: Steinberg, London
Contact:

Re: Missing notes after live MIDI recording

Post by PaulWalmsley »

I've found the problem. It happens due to a very specific combination of note timings. The notes for the very short chord in bar 9 has all the note on events in one order, but the note off events are in a different order. In general this shouldn't be a problem, but it led to a case where one of the notes was omitted. I should have a fix for this now, but if you have another MIDI file that it happens with, then I would be interested to see it so I can verify the underlying bug is fixed.
Architect & Developer - Steinberg London

GregS
Junior Member
Posts: 118
Joined: Fri Jul 03, 2020 1:39 pm
Contact:

Re: Missing notes after live MIDI recording

Post by GregS »

Thanks again Paul. Things have changed again, unfortunately. I first tried resetting the latency compensation back to zero (to make the problem easier to reproduce), but using that same pattern, it worked fine.
Right now, I do have a sequence that makes it fail readily (a different pattern to the one I sent you before), HOWEVER, this time, the MIDI data *is* actually missing the notes, and moreover, I can reproduce this in Reaper as well.

I'll see if I can get a raw MIDI dump using MIDI-OX, but I hope that it doesn't change the behaviour like it did last time.

I've reverted to using MIDI-THRU, so I can be sure that the keyboard did send the notes - I simply do not detect any missing notes in the audio.
Elements 3.5.11.1054, Windows 10

GregS
Junior Member
Posts: 118
Joined: Fri Jul 03, 2020 1:39 pm
Contact:

Re: Missing notes after live MIDI recording

Post by GregS »

I think I'm simply doing these blasted legato repeats much more often than I thought. When I play the same problematic passage in a much more deliberate fashion, the problem goes away. I did actually do a test a few days ago like this, and I thought I had eliminated this possibility, but I think I got it wrong. I will now get a MIDI dump to try to prove it.

The problem you found Phil I suspect is happening much more infrequently, and it was sheer chance that we caught that one.

These two problems, in combination with simple bad quantization (partly due to lack of MIDI latency compensation, initially), all added up to a total mess and confusion on my part.

Just btw, if there are any MIDI-OX experts here who are willing to write a script to transform the real time MIDI stream into conventional Note-On/Note-Off pairs, please volunteer here by way of reply, and once I have found someone, we could then take it off line. I'm willing to negotiate a fee for your work. (I'm not sure whether MIDI-OX has the capability of doing this type of thing mind you - it's going to have to be able to maintain state in between MIDI events)

EDIT: if MIDI-OX can't do it, perhaps there is another utility that can.
Elements 3.5.11.1054, Windows 10

GregS
Junior Member
Posts: 118
Joined: Fri Jul 03, 2020 1:39 pm
Contact:

Re: Missing notes after live MIDI recording

Post by GregS »

I've reproduced the simple missing note problem again. (playing nice clean staccato). I set the latency in the opposite direction, to really try and bring it out.
In the attached, the following notes are missing in the score:
Bar 2, 16th-note 1, NN 55
Bar 3, 16th-note 1, NN 48

The MIDI looks fine in a DAW.
Attachments
missing_notes.zip
(390.64 KiB) Downloaded 3 times
Elements 3.5.11.1054, Windows 10

GregS
Junior Member
Posts: 118
Joined: Fri Jul 03, 2020 1:39 pm
Contact:

Re: Missing notes after live MIDI recording

Post by GregS »

Regarding the tri-sensor problem, the Bome MIDI Translator appears to have the capability to do the transformation, and if I attempt to do it, this is what I'll be trying first, because it seems newer & fresher than MIDI-OX. It's scripting language is very rudimentary, but it does appear to have the capability - I've asked them.
It would of course be ideal if Dorico could support it. It suspect it would be easy to simply do the transform to Note-On/Note-Off, but to RETAIN the repeated Note-Ons, and show it properly in the event window, and allow it to be edited, requires a lot of thought I think.

I wonder whether "legato repeats" are ever notated? I.e, explicitly requesting that the dampers are not lifted? I very much doubt it - I suspect that it's just something that pianists can exploit on grand pianos, but not notated.
Elements 3.5.11.1054, Windows 10

Post Reply

Return to “Dorico”

Who is online

Users browsing this forum: benwiggy, pianoleo, Thijs Boehme and 8 guests