Art is always changing, and so a collection should too: searching, combining and trying. You can't make errors when you collect. All purchases are part of your development as a collector.
How to choose from the wide range of artworks: see a lot, talk a lot, and follow your instincts and intuition. Try to decide on what you would really like to have, also when you don't buy it.
Financial boundaries: financial constraints naturally lead you to buy young artists, who often break with established art-making processes. These restrictions make collecting more exciting.
Your possibilities: Do not consider your budget or the space in your home a limitation. Compare it with the way a poet limits himself when making a sonnet or an epic tale: it sharpens you, you have to make choices, it puts your decision making to the test, just like the artist whose work you want to buy. He also constantly has to make decisions.
Buying art in the artist's atelier can be inspiring because it gives you the opportunity to talk with the artist and see what are his or her ideas, but the many impressions you get in the atelier can be overwhelming.
Buying art in a gallery has the advantage that the gallery owner already made a first selection. This can start you in the right direction, but be aware, it's your decision.
As a starting collector you have to realize that contemporary art is a broad concept. You have to be well-informed about what is happening in the art world, but you also have to stay true to your own preferences. The red line will come with time.
Enjoy choosing for yourself, there's no purchase committee looking over your shoulder. Buying art though, means that you can't buy something else. That's the choice you make.
Art is not a good investment. It's a hobby, a passion. If you do it right you can earn money, but by selling you will loose a part of yourself. Time will tell. Orientate yourself on the fiscal opportunities of buying and gifting artworks.
The intensity of your first impression of a work of art, that often makes you decide to buy it, says a lot about if it keeps fascinating you in the future. This does not mean you have to understand the work at a glance. It is the unfamiliarity, the elusiveness that keeps the work interesting.