Hot isostatic pressing (HIP) is commonly applied to additively manufactured Ti-6Al-4V to eliminate lack-of-fusion (LoF) defects; however, significant differences in fatigue performance are often observed even when residual porosity is comparable. This study investigates whether fatigue behaviour in PBF-LB Ti-6Al-4V material with varying defect content is controlled by process defects or by microstructure-dependent crack initiation mechanisms. Specimens were produced using two process themes, a minimal porosity (MP) and a high productivity (HP) condition with reduced volumetric energy input, and subsequently HIP treated below or above the -transus temperature. X-ray computed tomography confirmed that HIP effectively healed LoF defects in the HP material, reducing residual porosity to levels comparable to MP builds. Fatigue testing revealed that, once defects were eliminated, fatigue life depended strongly on the HIP microstructure: sub- HIP consistently produced longer lives than super- HIP across all stress levels. EBSD analysis showed that sub- HIP generated a fine  structure that distributed cyclic plasticity across many grains, whereas super- HIP formed large prior- grains containing coarse  colonies with very fine lamellar structure within each colony. These colonies acted as microstructural units, concentrating cyclic slip into long bands and promoting early crack initiation. The results show that HIP reduces defect density and the remaining fatigue failures are predominantly governed by microstructural mechanisms. After excluding a few samples with defect initiated failures, fatigue life was controlled by colony slip localisation, identifying the  colony size as the dominant microstructural fatigue length scale. Tailoring HIP temperature provides a route to achieving wrought-comparable fatigue performance in high-productivity PBF-LB Ti-6Al-4V.