Bad laydown?

[Wes64]

I was playing 2-5 No Limit at Harrah's in New Orleans. I had been playing about 3 hours. The game was 5 locals who all seemed to know each other, 4 people who seemed to be from out of town, and me (from Dallas).

Immediately to my right is the tightest, quietest local. He was playing very few hands and was aggressive in those he played. The only hands he had shown down he won, and they had been big hands with one of the tourists calling his raises. He is our villian.

Hero is on the button with Q [img]/images/graemlins/spade.gif[/img] J [img]/images/graemlins/spade.gif[/img]. Early position raises to $20. (That had been a fairly standard opening raise in this game and the $20 raises had been getting callers). All fold to villian who calls. Hero calls, SB folds, BB calls. $82 in the pot. I have approximately $440 after I call. The other 3 players all had between $330 and $450.

Flop comes 4 [img]/images/graemlins/spade.gif[/img] 7 [img]/images/graemlins/spade.gif[/img] J [img]/images/graemlins/diamond.gif[/img]. So I have top pair and a flush draw.

BB checks. Initial raiser checks. Cut-off goes all-in for $330.

Hero thinks and folds. All fold to villian who turns over A [img]/images/graemlins/club.gif[/img] T [img]/images/graemlins/spade.gif[/img].

I thought he must have had either had A J or a set. Either way my only outs are the spades. There were several weak players at the table and I thought that the villian was trying to deprive them of proper odds to draw the flush but hoping one of us would call with just a flush draw.

Should I have called? The pot was laying 740:330 if I called. But thinking he must have a J or a set, don't I need better odds (say 1000:330) to call?

Be brutal.