Make sure you're calibrated, and turn your sound up louder than you'd normally have it during the calibration.
As for cadence when speaking, adding a slight pause between "Xbox" and the command is important. The other thing a lot of people do, is they start spitting out commands one after another and get mad that it doesn't work. It can be a little weird, but there is a circular indicator in the top right corner of the screen. When you issue a command it turns white. When it is ready for the next command, it will switch back to green. The way the UI is setup, the next interface has to fully load, including loading all of the onscreen voice cues before the indicator will turn green and allow the next command to be issued. Hopefully they can tweak it, but that's the way it is now. I still use it almost all of the time unless I'm playing a game. That way I can issue commands while I'm using my phone, eating, or looking around on my laptop, without messing with the controller.
There are standard voice commands that you can issue by saying, "Xbox, [command]". You can also say, "Xbox, Select" on any screen, and it will highlight all of the elements in the UI that can be activated with a voice command by showing green text. Once you have issued the "Xbox, Select" command, any green text can be spoken and the command will issue, but it does not have to be preceded with "Xbox." So, if you're on the Home screen and you say, "Xbox, Select" you can navigate to the Store by saying, "store", without preceding it with Xbox. The easiest way to explain it, is if you don't see the command on screen, you have to precede the command with "Xbox". If the command is highlighted on screen, you don't.
You don't have to yell or whisper. You should be able to speak normally. Also, for whatever reason, some people raise their voice an octave when speaking to it. I'm not sure why. I was at my bother's place, and he said it didn't work. I sat down with him and he was trying to give it commands, but he was talking like Peter Pan. I told him to try speaking to it like a normal human being and it worked. There must be some weird psychological thing that prevents some people from interacting with it correctly in terms of how they speak. Like some people shout when they're on the phone, because they subconsciously know the person they're talking to is far away. As an aside, the Kinect on Xbox 360 recognized you when you waved at it. A friend of mine came over, and I say, "Wave to the Kinect" and he says, "It's not working," but what I was seeing was the most freakshow non-wave I had ever witnessed. I went and stood beside the Kinect and asked him to wave at me and he did, and it worked.
Your best case will probably not be 100%, but it can be close to it. Mine is. I very occasionally have hiccups where it doesn't seem to hear anything for about 30 seconds, and then it'll start working again. It happens maybe once a month. Besides that, I have issues with the "Xbox On" command and "Close." Neither is used often, so it's not much of an issue, but they fail for me far more often than anything else. My window air-conditioner (somewhat loud) does lower my hit rate a little. It's in the next room, through a doorway, but it's facing Kinect directly. I guess the noise cancelling can't cut it out perfectly. They're probably 25 feet apart.
Now, if you go to use the new Netflix app, which I'm assuming is made by Netflix, it is a huge piece of shit for voice control. It looks like they carried over whatever they used on PS4, and it's junk. The video content is shown in thumbnails. When you say, "Xbox Select" to show the different commands you can issue, it will place a small banner at the bottom of each thumbnail (eg. "Play 1") to let you know which command will play each movie. Well, on a lot of the thumbnails, the banner covers the title of the movie ... The thumbnails are so small, that with the banner on top, you won't even be able to guess which movie you're looking at from the remainder of the thumbnail. Good job Netflix. The Microsoft app was kind of a turd as well, but at least navigation worked correctly.