I have not personally heard of any side effects in men although I guess you should do your own research if this concerns you. I know that men who cycle steroids for bodybuilding often use Clomid to help keep estrogen levels down in their system. Too much estrogen can cause things like “bitch tits”. Here is a snippet on Clomiphene Citrate (clomid):
Clomiphene Citrate
Clomiphene citrate (brand name Clomid, Serophene) is used to induce ovulation. It revolutionized the field of infertility in the late 1950s.
Description
Clomiphene citrate is an orally administered medication. The initial dosage is 50 mg per day for five days, from day three to seven of the woman’s cycle. The dose may be increased in subsequent cycles if the minimum dose does not result in ovulation.
Clomiphene citrate appears to act on the hypothalamus and is useful for women who do not ovulate because of hypothalamic or pituitary problems. Given early in the menstrual cycle (day three to seven), it suppresses the amount of naturally circulating estrogen. This “tricks” the pituitary into producing more follicular stimulating hormone (FSH) and luteinizing hormone (LH). These hormones then stimulate the ovary to ripen a follicle and release an egg. Of patients who are properly screened for use of this drug, about 70 percent will ovulate, and 40 percent of those will become pregnant. If a patient ovulates but does not become pregnant, the physician should check cervical factors. The anti-estrogenic effect of clomiphene citrate can create a “hostile” environment for conception.
Note, increasing FSH and LH in men results in increased sperm count and production of seminal fluid.