Objective: The objective of the study was to explore the prevalence of different conditions in infertile men with severe male factor, based on endocrine evaluation and scrotal ultrasound. Materials and methods: The retrospective study was carried out on 86 infertile men with azoospermia and severe oligospermia who attended the Male infertility clinic of the department of Reproductive Endocrinology and Infertility of Bangabandhu Sheikh Mujib Medical University. They had evaluation with serum testosterone, follicle stimulating hormone (FSH), luteinizing hormone (LH), prolactin and thyroid stimulating hormone (TSH). In addition they had scrotal ultrasound. Results: Most (88.37%) of them had azoospermia. Severe oligospermia was present in 11.63%. Testosterone deficiency was detected in 34.9%. Non-obstructive forms (pre-testicular and testicular causes) comprised 62.8%. Scrotal abnormalities suggestive of obstructive and non-obstructive varieties were overlapping with endocrine diagnoses. Conclusion: Majority of men with severe male factor infertility has non-obstructive azoospermia which has poor chance of sperm retrieval for in vitro fertilization and intracytoplasmic sperm injection (IVF-ICSI). Substantial proportion of them has testosterone deficiency which needs to be treated if they do not opt for IVF-ICSI with poor prognosis or donor oocytes.