grammar question regarding the use of "at" and "in"

[BiCuriousGeorge]

my girlfriend called me and asked where i was. i said "i'm at the gym". it then dawned on me that i could also have said "i'm IN the gym" and that would have been an appropriate sentence too.

but if she had called me and asked where i was and i had been sitting in a cardboard box, it would be fine for me to say "i'm in a cardboard box" but would sound weird if i said "i'm AT a cardboard box"

"i'm in my house" and "i'm at my house" both sound fine, but if you were on a road trip and i called you asking where you were, you would say "i'm in north carolina" but never "i'm AT north carolina"

what's the rule here that i can't seem to figure out in my head regarding what makes at or in appropriate?