Most businesses are required to register for the GST (and collect and remit the GST or HST).
However, if your business qualifies as a Small Supplier, then you are not required to register for the GST/HST.
The Canada Revenue Agency defines a GST Small Supplier as a sole proprietor, partnership, or corporation whose total taxable revenues before expenses are $30,000 or less annually. (Read the complete Small Supplier definition.) Note that the Small Supplier GST registration rule doesn't apply to all types of businesses; taxi and limousine operators, for instance, must always register for the GST.
But even if your business does qualify as a GST Small Supplier, you will probably want to register for the GST anyway. If you register for the GST/HST, you can them "reclaim" the GST/HST you've paid on business purchases, on everything from capital property through office supplies, through GST/HST Input Tax Credits. Therefore, it's wise to register for the GST right away when you're starting a business.
See Grappling With The GST and/or HST for a fuller explanation of the Small Supplier GST exemption and how GST Input Tax Credits work.