I spent quite some time on a vegan diet. The only time I felt like I was missing out was when I went to a restaurant, and that was only because most restaurants have very few - if any - vegan options.
Oh yeah, that's definitely the worst part. When I was abroad there were hardly ANY even vegeterian dishes at restaurants, so I usually had to eat before or after, sometimes I couldn't even go. It wasn't very fun. It's super easy to be vegetarian/vegan in Sweden though! About one in seven swedes are vegetarian/vegan now!