When you can only beat a bluff? Can you call?

[Theinvestment]

Hello all, An interesting situation came up when I was playing 2,3 NL cash games. I had about 500-550 in front of me. The mid position player raises to $15 (a very standard raise on this table) the cut off calls, I call from the button with 10 9 Suited diamonds, then the BB calls. The flop comes 4 J Q with 2 spades, everyone checks the flop.
The turn is a 5, everyone checks and I decide that there was weakness and fire out a bet trying to take the pot down with a $50 bet into a $60 pot assuming everyone was drawing and was not strong. The two early and mid positions quickly fold but then the cut off was thinking about a play, He ended up announcing hes all in for $165, which suprised me. He was playing relatively tight that game but I couldnt have put him on the J or the Q or the spade draw becuase he would have generally bet on the turn. I called time and was thinking I had the best hand, It was another $115 to call and I was getting about 2.5 to 1 and if he had me beat with a high card I still had outs to beat him come the river. I decided to fold and then he turned over 7 3 off suit, which absolutely baffled me, my inital read was that he was weak, but my major concern was that he may have a high card or ateast a small pocket pair which beats me at that stage anyway. Im still confused as to what I should have done, any input would be great, I feel I made the right fold but I think his play was very risky considering it was a stone cold bluff and I had been playing very well with solid cards etc. Feedback would be great thanks????