When there is no key pressed, the inputs (rows) are high and the outputs (columns) are low. Pressing a button will pull the row low. Then I set each coulmn high in turn and see which one restores the row to a high state.
Is this backwards from the way I should be doing this?

the issue here is that any '51 input that float (in your case no key) will be high, thus you can not see "which one restores the row to a high"

Ok, your post does not make total sense to me, but let me try another way:
assuming that you select column and detect row (the reverse works just as well)

1) set all columns low, if no row low, no key
2) set all columns high
3) loop setting one column low at a time, check rows - any row low is 'key found'

Erik