If you are American, this one may come as quite a surprise to you. Although fashion trends are changing rapidly to make the baseball hat much more popular, that wasn't always the case. As baseball is a very American sport and one that is played in very few countries outside of the US, most countries also do not wear baseball hats as a result. They opt for other headware.
If you are traveling abroad and wearing a baseball hat, there is a good chance that people will assume that you are an American tourist just because of the hat.