Are they allowed to jump their fence to use specfic resident only access?

This is rather extreme, but you could put up barbed wire on the top of your fence. Other than that, I don't think there's much you can do. It is trespassing if they are entering your private yard, but unless a cop actually witnesses them doing it, I don't think you'd have much luck trying to press charges.