What if I were to set a column as push-pull right before setting it high, then checking the rows, then setting the column back to open drain before moving to the next column. Presumably this would retain speed while ensuring the safety of the ports, no?
NO, you do not need "a column" high, but "all but one column" high. Pressing two keys simultaneously with a no diode keypad (I assume that as usual this is what it is) will connect two rows and two columns = connecting a high PP with a low PP (and two inputs, but that is irrelevant).

Erik