If you don't understand why CoD has become such a big franchise you probably don't understand what CoD players like and what IW has been doing to "feed" that consumer.
First, CoD isn't for everyone. While CoD was a quality series I didn't like CoD1-3., so I understand not liking the series.
But if you go back and look at the series, though, CoD 1-2 offered some high quality story telling and stood out in the WWII market. CoD had a number of campaigns with roughly related story archs with an action oriented "authentic" experience with an emphasis on the viceral. Yes, there was a lot of scripting--but that also allowed IW to control pacing, story telling, and the eb-and-flow of the action. Every element is scripted in detail. The multiplayer in CoD1-2 was always one of the better classic non-vehicular shooters. Good gun balance, great maps, solid mechanics, and new twists (e.g. death cam prevented camping).
For you non-PC folks CoD1-2 were quite popular on the PC. CoD2 was also a BIG seller for the 360 launch, probably the biggest game in the launch window in terms of hype and sales. There were some online issues that hindered the online community component but it was clearly one of the best online games on any console at the time. This planted the seed for CoD4.
CoD4 just took off from there.
- It shed WWII for the more familiar and popular modern combat.
- The Engine. 60fps and looking better than most shooters helps a lot. CoD4 is a good example of "smoke and mirrors" as they aren't doing a lot of "advanced" techniques, but instead are maximizing "older" techniques at high qualities. Looking at indivudal components of an image can be pretty depressing (for me!) but the visual package as a whole is really nice. The result is a really smooth engine (even on the PS3, which helped marketing a lot) that lent itself well to game design.
- The 6-9hr campaign, while short, had a lot of variety, frequent change of pace, a coherent story that was equally well told through the gameplay and story boards, offered various perspectives, was gritty and viceral with excellent set pieces, and can be summerized as "intense." Yes, the AI isn't designed to be cutting edge "free thinking sandbox" AI, and expecting such will leave you absolutely frustrated. But the scripting services the story and theme. In this regards it feels very much like the "Rock" or other over the top action paramilitary movie. You take the good with the bad, for every "spawn bunker" there is a scripted moment you won't find in a more free form AI sandbox (e.g. the sniping missions had some very movie-esque moments). CoD4 didn't excell in every element of SP, but the areas it did well it clearly stood head and shoulders above your normal shooter. Compared to Halo 3, for example, the story telling is just night and day.
CoD4 SP has some really, really memoriable moments. The fallout crash is one of those really eerie moments in gaming thaqt is hard to shake, as is the ending. The 50 cal. flash back was well done, and who can forget the AC-130 segment?
- Gameplay is very balanced. IW really nailed gun balance and feel with map design. Everything from how the health works, ammo availability, accessible cover, etc works well. There are some annoyances (e.g. nades in SP) but in general the game feels very well thought out.
- MP ... If you don't play MP then you probably wouldn't understand why the title is so popular (and should refrain from being critical of its success without understanding this component). In context there has been a shift in MP games with the advent and popularity of titles like Battefield. CoD4 is more traditional finding a middleground between twitchy quake-style shooters and slower paced tactical shooters like RB6. In many ways CoD4 is a spiritual sequal to CounterStrike with more gameplay modes and an evolution of the gameplay. You have your military terrorist/anti-terrorist theme with average moving (but quick aiming) players who have various movement penalties and recoil penalties. Gun range and configuration, as well as zoom/unzoomed, all play into the mechanics that create a game that is accessible but has a huge learning curve. I think CoD4 is the first time a lot of the major principles that made CS so popular have been translated well to a console.
CoD4 really straddles the "accessible but rewarding" curve well. e.g. The ranking allows even newbs to get level-50 (something Halo 3 would never, ever allow). Yet your less skilled players, while being rewarded with level-50, don't gain some major advantage as the 5 base kits are all well done. Instead the perks and accessories allow you to tailor your classes to your play styles and preferences. This is a good example of rewarding players for playing more and allowing them to "claim" the game as their own. CoD4 allows the newbs to play more team oriented games (always a plus) as well as providing some perks for retribution (one of the few games that can reward the blitz tactic ... and forces you not to rush and to back off rushing opponents). The fact when you do well you get popups noting your accomplishments--quite regularly--is a big re-inforcement. Much like leaderboards, achievements, unlocks, etc CoD4 keeps toying with you "one more game and you get a new perk!" It is a pokemon game for shooting fans.
For the more skilled gamers you constantly rewards you. Sneak up on a small group with superior positioning and a handful of well placed shots and you get a 3 kill streak--and reward. Good, smart players can be totaly devastating. On the one hand you have Halo, which passively encourages teamwork and a "waltz" to the rebounding shields and gun reloads, and CoD4 which is a totally different style of shooter the modern warfare "overwhelming force" can shatter the back of an opponent within seconds. Every gun in the right hands can feel like a sniper rifle. Cover, loadout, magazine management, etc are all still required but with a different skillset from Halo. This is why you see both competiting, but not necessarily canabolizing, each other.
I am partial, but I don't find it surprising that the most popular MP shooters are at one moment accessible but also brutally rewarding to the best of players. Where Halo 3 has this really well done balance of gun power/range with these super bunny hopping marines with a spattering of vehicles CoD4 is all about modern weapons and utilizing your loadouts well on the map. Both reward map knowledge and reading the "eb-and-flow" but they play very, very differently and decisively reward smart gameplay. Clearly they are both MP games where the MP isn't just the SP tied together with some netcode.
Modern warfare isn't going to appeal to everyone in the same way. Some may find the lack of vehicles lame, dislike the gun balance (everything is lethal), have no interest in RPG-lite perks and unlocks, etc. But there is no denying CoD4 is really deep, fully fleshed out, and a robust and balanced experience. When looking at other MP packages in other shooters it is really clear IW has a good grasp on what a lot of consumers like--and importantly--know where not to drop the ball. They have a complete, coherent, package and don't overlook such basic, elementary things like player movement and responsiveness. No element of the MP seems really disconnected or to destroy the overall experience. Some games may have cooler elements, some individually better ideas, and fresh new experiences. But on most cases they flop at key moments which really kills longevity. IW has shown they can avoid these sort of train wrecks by proving the right features for the product.