Residual stress measurements using x-ray diffraction is a well established method used within the industrial and academic community to verify the performance of different processes for metallic materials. The measurement gives an absolute value of the stress state which can be used to design and optimize the process route to induce beneficial compressive residual stresses and avoid detrimental tensile stresses. Investigating the uncertainty and accuracy of the measurement system, operator and the material is therefore of high relevance both from an industrial and scientific point of view. Round robin testing is an important way to quantify the uncertainties that could affect the quality of the measured results and hence how a process is optimized and tuned. Such an investigation allows the operator to understand and reduce variations. Current round robin test includes results from five different laboratories using comparable equipments located in Sweden, Finland, Germany and United States. This work focuses on five shot-peened tool steel specimens produced with identical process settings. Additionally, an investigation of the repeatability of the system, influence of the operator, variations within the specimen, and the long time stability of the specimens has been measured.