Listen to each track solo. You can isolate, which track causes the draggin. If the dragging does not appear in solo play, it must be a problem with all tracks playing together. So, polyphony...
Too much notes at the same time might be too much for the polyphony, causing the sequencer to thin out the less important notes (depending on your settings). Since this procedure happens in realtime, this little drag could be a result of this. You should then manually erase certain notes to remain within the polyphony range of your sequencer.
Hope this helps... even if it's only to find out what is NOT the reason^^