Fit is the most important component of how your clothing looks on you. The other elements--color and style--should be afterthoughts to how it fits. A great looking suit on my dad, for example, would be a bad choice for me because our sizes are different and the fit would be poor. A fairly obvious observation, but the point is that the nicest of suits will look bad if the fit is wrong. This is true with suits, and it's also true of shirts. A shirt, like a suit, should fit properly. Too large and it looks billowy, sloppy, unkempt, unattractive. Too small and it pulls at the buttons, is short on the arms, and is generally unflattering. All this regardless of whether it is a good color or pattern.
From November's GQ
The majority of off-the-rack shirts are made to very general specifications, and are designed to fit as many people as possible. The result is that even if you are a 15 neck and neither skinny nor large, the waist and chest will always be several inches too big, and the arms will usually be too wide. You can tuck it in as best you can, but the folds and ballooning effect of all the extra material only gets worse as you go about your business. There is a better way, however. You can find fitted and semi-fitted options with many retailers. Stores like Banana Republic and Express gear most of their shirts toward a more fitted look.
The point is to find a fit that works for your body type. If you're a bit larger, a regular cut could be a good choice because you won't end up with lots of extra material in the arms and torso. For guys that have normal or skinny frames, get a shirt that doesn't look like it was meant for someone 40 pounds heavier. The neck should not be loose yet unrestrictive. Cuffs should go down to the ends of your wrists where your thumb meets the wrist.
Like suits, the best option with shirts is to get it tailor-made for you. Not only do you get the right fit in the torso and arms, as well as your preferred length in the sleeves and tails, you also get a bunch of other options. You choose your fabric, collar and cuff, and even the smaller details like pocket, yoke, placket, pleats, and button options. And you can have it monogrammed if you like. Sound expensive? You can get a custom shirt for the same price as an off-the-rack shirt at Nordstrom or Banana Republic.