There are times when I get so focused on what I am taking a picture of, I don’t remember that I can simply rotate my camera ninety degrees and I will have a very different image, that will sometimes be better. When I see many hobbyists taking pictures, it seems especially rare for them to play with the orientation of the camera. Most of the images are landscapes with a regular camera, or portrait style with a camera phone (such as the iPhone). They get trapped by their habits and miss an easy way to change how dynamic their photographs are.
Rotating the camera can add background or landscape to help define an image, give it a sense of place. It can also be a simple way of removing unwanted features that are in the frame with one orientation of the camera, but not another.
Vertical and horizontal pictures also flow differently from each other. It can be possible for lines in the image to lead the eyes better with a portrait orientation instead of a landscape and vice-versa. If a river cuts through a scene from left to right, maybe a landscape is best. If it recedes into the distance, vertical may be best. There is no hard rule, play around with it. For a little while, with every picture you take, rotate the camera and take a second shot, then compare the two.
Finally, magazines prefer portrait orientations for covers and landscapes for inside two page spreads. If you want to sell for a particular story, offer the magazine options (just not too many options).