Try a different brand.
Some stick better than others. Some stick too well. The Band-Aid brand goes from one extreme to another in how well they work (or don't) depending on what type you buy.
If you're really allergic to the adhesive, talk to a doctor. Maybe they know of more "hypo-allergenic" bandages on the market for people like you.