It depends on what you get-they're not all created equal. And most ANC headphones are designed to cut out background noise but still allow you to hear people talking to you, so they don't just universally block noise out, they cut out certain frequencies. You have to have realistic expectations-you probably are going to be disappointed if you expect them to cut out absolutely everything with just the noise cancelling (and no music) on. Sometimes in particularly noisy environments you can think they don't seem effective, until you take them off and realise how much noise they are cutting out.
I've just got some for the first time (Mpow H17 headphones) and I love them. There are various things that still come through-people talking in close proximity in particular-but it filters out background noise so well. I've tested mine in a variety of environments and they're generally very good. I have a boat, which is 10ft from a railway line, and recently the wind and rain have been causing a lot of noise, plus there is building work nearby and a busy road...but with my headphones on it's reduced to a low rumble when the trains go past and very faint sounds from the building site. It doesn't make everything silent by a long shot, but it does make it very quiet.