Very small set-ups can work without the use of fans, although they will still require some sort of ventilation, but if you are using anything more than 1 CFL lamp, you will also need to look at air circulation, which generally means a combination of fans (to draw in and circulate air), filters (to remove the smell) and ducts (to remove stale air).
It is essential to have clean air in the growing area to give the plants the best possible chance of growing healthily and avoiding hazards such as mould and infestations of mites, particularly spider mites.
Good ventilation (and in particular the use of a filter) will also help to keep the neighbours happy. Cannabis has a very strong and distinctive smell and some people dislike it intensely. While fans and ducts will last over the long-term, filters need to be changed regularly to be effective. The packaging will state how long they can be used and it can be worth marking down the date by which a filter must be changed when you install it, so that you're prepared before it actually runs out.
When setting up your ventilation system, you should think about air as being like light. You want all the plants to benefit from it, more or less equally, but you want to avoid over-exposure. Basically you should be able to put your hand in to any plant and feel a pleasant breeze. If there is no breeze or you are being blasted with air then you either need to rearrange your plants or rethink your ventilation set up.
While fans come in various sizes, most ducts are 6” meaning that for this reason alone, it makes sense to go with 6” fans. How strong your fans need to be depends on the size of your grow-space, how much heat your lights produce and the overall conditions in the grow-space, i.e. how much humidity or ventilation does it have naturally. It also depends on the plants' growth stage, in that smaller plants need less active ventilation than larger ones. There are charts and tables available for guidance, however, regularly checking that each plant can feel a soft breeze and making adjustments if not is at least as effective and probably easier.