Vermont hasn't suffered nearly as much as a lot of its neighbors when it comes to the number of people leaving the state. However, that's not to say it doesn't have some problems. Rents and home prices are pretty high in the state, and like a lot of other places, there just aren't enough good-paying jobs to keep a lot of people content in Vermont. So, many choose to leave.
Still, it could be worse, and Vermont isn't going to have a population collapse anytime soon. That said, a lot of retirees are moving away in search of warmer climates.