Sometimes starting a new collection can be intimidating simply because of the number of different items that are available. Most collectibles can be collected by some kind of theme within the collectible space - baseball cards can be collected by team or year, rocks can be collected by color or type, toy cars can be collected by manufacturer or type of vehicle - and world coins are no exception. Because of the large variety of coins that exist, it's easy to pick one (or more) themes to collect instead of trying to collect anything and everything. A themed collection can also be more interesting to display, because the coins are connected to each other. Children especially may enjoy building a themed collection because it's easier to know what to look for.
Picture Themes
A collection based around the pictures on the coins requires no reading and allows the collector to make decisions about whether to include or exclude a certain coin. Some good picture themes are animals, plants, people, buildings, or transportation (boats, cars, airplanes, etc.). You can narrow down the theme even further (only birds, only flowers, only boats) but I think you'll find this will seriously limit the size of your collection. Still, for a child's collection, or a collection for a very specific purpose (to frame and hang in a boat, for example), a very small, very specific theme can be a great way to start.
Date Themes
For a collection with personal significance, you could collect only coins from a certain year (your birth, child's birth, wedding, etc.). The coins in a collection like this would be quite varied, and the number of coins available to collect would be larger than with a picture theme.
Coin Property Themes
Beyond collecting based what's on the coins themselves, you can form a collection based on the physical properties of the coins.
For a visually striking collection, you can collect coins that aren't round. I believe that non-round coins are less common now than they have been over the past 50 years, but you can find square coins, octagonal (and other multi-sided) coins, and scalloped coins (with a wavy edge).
Several countries over the last 50 years have produced coins with holes, including Spain and Denmark. Japan's 5-yen coin has had with a hole in the center for many years (it's the only coin I can think of that is still in circulation). Between 1900 and 1950, more countries had coins with holes (including Belgium and France).
You can also create a collection around the color of the coins. A large percentage of all coins are silver in color (though very few are made of actual silver), so it can be fun to collect only the other colors - copper or bronze (brownish) or gold-colored (you're very unlikely to find any coins made of actual gold). Older coins that look black are probably made of zinc, which looks silvery when it's fresh but can tarnish to black.
Location Themes
Because coins come from specific areas of the world, you can build a collection based on where the coins are from. They could be based on specific countries (coins from countries you've visited; coins from where your family comes from) or specific continents (Europe, Asia, Africa, South America) if you have a particular affinity for some part of the world.
For children, it can be fun to try to collect one coin from every country. You can have a map of the world that can either be colored in or marked with pins as each country is collected, which can also help get children introduced to world geography.
Beyond Themes
Even if you start with a theme, you eventually may expand your collection and start trying to collect every different world coin. The question then becomes - what constitutes a "different" coin? As a collector, you will have to make decisions about what kinds of changes you care about collecting and which you will pass up. While major changes (coin size, color, or picture) more obviously deserve to be in your collection, there are some less-obvious changes that you may or may not want to include.
Mint marks - Some countries (including the US) put a different letter or symbol on coins to indicate where they were made. Except for the mint mark, the coins are usually identical.
Language differences - Pre-Euro Belgium coins came in 2 varieties each year - one with Belgium written in Flemish (Belgie) and one with Belgium written in French (Belgique). The rest of the coins are identical.
Font changes - From one year to another, the font (letter or number style) used to write on the coin may change, even if the rest of the coin (picture, denomination, etc.) is the same. This can be hard to spot unless you compare 2 coins side-by-side, or use a reference guide that mentions these changes.
Minor design changes - Sometimes a coin's picture may undergo a slight change - made a little bigger, a little smaller, or simply re-done so that it is slightly different than it was before. These can be very hard to spot.
Alignment changes - I noticed recently that Switzerland's coin alignment (see my earlier post about terminology) changed from coin to medal in the 1980s. Except for the alignment, the coin design (pictures and etc.) look the same.
For these kinds of small changes, it is up to you to decide if it is worth having each variety in your collection or not (they very rarely affect the value of a coin).