Human practices of all kinds - substance use, gambling, sex, even eating - are increasingly being reframed through the language of addiction. This 'addicting' of contemporary society is achieved, in part, through the screening and diagnostic tools intended to identify and measure addiction. These tools are a key element in the expert knowledge-making through which realities of addiction emerge. Promoted as objective and accurate, the tools are given legitimacy through application of scientific validation techniques. In this article, we critically examine the operations of these validation techniques as applied to substance addiction tools. Framed by feminist and other scholarship that decentres the epistemological guarantees of objectivity and validity, we structure our analysis using Ian Hacking's (1999) concepts of 'refuting' (showing a thesis to be false) and 'unmasking' (undermining a thesis). Under 'refuting', we consider the methodological validation processes on their own terms, identifying contradictory claims, weak findings and inconsistent application of methodological standards. Under 'unmasking', we critically analyse validation as a concept in itself. Here we identify two fundamental problems: symptom learning and feedback effects; and circularity and assumptions of independence and objectivity. Our analysis also highlights the extra-theoretical functions and effects of the tools. Both on their own terms and when subjected to more searching analysis, then, the validity claims the tools make fail to hold up to scrutiny. In concluding, we consider some of the effects of the processes we identify. Not only do these tools make certainty where there is none, we contend, they actively participate in the creation of social objects and social groups, and in shaping affected individuals and their opportunities. In unpacking in detail the legitimacy of the tools, our aim is to open up for further scrutiny the processes by which they go about making (rather than merely reflecting) the disease of addiction.