Correct me if I am wrong on this, but while all the leaves work to absorb light and help convert sugars for its food supply, not every leaf goes to the total growth of the plant. If you cut a leaf on a left branch, it wont affect the growths on the other branch. Similarly, if you cut a fan leaf thats attached to the main stalk, you are slowing the bud branch's growth, but not the buds growth since the buds growth is affected by light to its sugar leaf. So, you are able to remove all of your stalk fan leaves once your plant switches its focus to bud development over vegetative growth.
I also find it funny that a lot of defoliation's detractors are people who dont even top their plant. Obviously you dont need to remove leaves when you grow a single branch...