Yes skewed data does matter. There could be multiple consequences including inability for your model to generalize. Your model may end up incorrectly favoring the wrong features in order to do well on the training data, but then when it comes to unseen data, your model is going to make a lot of mistakes.

There are some pretty good responses here on how skewed data can be damaging: https://www.quora.com/What-problems-do-skewed-training-data-sets-lead-to