Here's the nearest IBM club
http://www.ring76.com/index.html
The nearest SAM assembly is in Long Beach from what I can see. Then outside of that is the SAM in Las Angeles and of course the Magic Castle.
Also you are lucky to have 2 magic shops near San Diego. There's the Sea Port Village Magic Shop as well as Abney's Magic Shop. Magic shops will have lectures, meet ups and at the very least they'll be able to point you in the direction of the other magicians in the area.
I hope that helps out a bit.