Yes, starting a business in Canada when you're nonresident in Canada is possible - but certain requirements have to be met.
First, are you Canadian? If you are, then all you need is a Canadian address (not a Post Office box, but a real Canadian address). Then you can register your new business in the province that it's located in.
If you personally have or establish a Canadian address and are a Canadian citizen, you can register your business as a sole proprietorship.
Or you could form a partnership with a Canadian living in Canada; then you can use his/her address for starting your business in Canada.
Another approach is to start an incorporated business. You would still need a Canadian address to enjoy the tax benefits of having a Canadian Controlled Private Corporation (see Types of Corporations in Canada and Corporate Tax), as well as have the correct number of Canadians on your Board of Directors and meet all the other requirements for such a corporation.
The correct number of resident Canadians depends on the jurisdiction you incorporate in. In Canada you may incorporate federally or provincially for starters. How to Incorporate Your Business in Canada explains the advantages and disadvantages of each of these forms of incorporation and the basic procedure.
"Foreign investors need to be aware of the residency requirements imposed on the directors of companies incorporated in Canada. The federal statute requires that 25 per cent of the directors be resident in Canada. In case there are fewer than four directors, then the CBCA (Canada Business Corporations Act) requires that one director be resident in Canada. Each province has different residency requirements and an investor wishing to incorporate in Canada should consider this issue. For example, some provinces do not impose residency requirements for directors (e.g.: New Brunswick, Quebec, Nova Scotia and Yukon)." (Frequently Asked Questions: Corporations Canada)
These residency requirements for the directors of companies incorporated in Canada apply to all types of Canadian corporations, not just Canadian Controlled Private Corporations.
See Forms of Business Ownership in Canada for more on the differences between sole proprietorships, partnerships and corporations.
Everything written above assumes that you are going to continue to be a nonresident. If you are not a Canadian citizen you cannot just come to Canada and start a business. Instead, you would have to apply to immigrate to Canada through Canada's Business Immigration Program or stay where you are but team up with one or more Canadian citizens or landed immigrants to start a business in Canada; see As a Non-Canadian, How Can I Open a Business in Canada?