If one of the female comes from regular seeds there are male alleles in the cross,as for crossing 2 feminized plants(which i've later discovered being related)i haven't noticed any loss of vigour,these plants are actually very vigourous,i'm harvesting the second one in a couple of weeks.This is the first try for me so take it with a grain of salt.
I really don't understand what you mean, the y chromosome isn't carried by female plants regardless of what pollen parent made them, what "Male alleles" are carried by females?
If you breed only using reversed females you will lose the genes the males had, but those genes would only appear in males anyway...