Sometimes trees send up additional shoots from the rootstock at the base of the trunk of especially newer trees.  Many trees are actually clones where a part of the desirable tree is grafted onto a rootstock that is a different tree.  In some cases, like a crabapple the rootstock may be more aggressive than the selection that was grafted onto it.  The rootstock may send up shoots.  Those shoots are called suckers. 

It is best to cut off those suckers by pulling back some of the soil and with a sharp shears, cut the sucker growth under the soil surface and then pull the soil and mulch back up the tree.

In the pic, you can see where a male selection of a Gingko was grafted onto a Ginkgo seedling.  The main trunk is between the two green suckers and the bamboo stake is to the left of the main trunk of this young tree.  Cut off those two green shoots as far below the soil surface as you can to force all of the energy into the grafted portion of the tree for best results. 

It is not a good idea to spray the suckers with chemicals to kill them off as you do not want to kill off the rootstock.  Your persistence in removing those suckers when young and small will be a simple and easy task and will benefit your tree greatly. 

Removing those suckers on trees it the opposite thing to do for your shrubs where you should cut out the largest stems and allow those suckers to remain and renew the shrub with new young shoots.