Grading Gems: Appraising the Quality of Research for Social Work and Social Care

The impetus towards basing practice and policy decisions moreexplicitly on sound research requires tools to facilitate thesystematic appraisal of the quality of research encompassinga diverse range of methods and designs. Five exemplar toolswere developed and assessed in terms of their usefulness inselecting studies for inclusion in a systematic review. Thewidely used ‘hierarchy of evidence’ was adaptedand used to appraise internal validity. Four tools were thendeveloped to appraise the external validity dimensions of generalizability(two scales) and methods of data collection (two scales). Methodsof combining the scores generated by each tool were explored.Qualitative and quantitative studies were appraised, not separatedinto two spheres but by using complementary tools developedto appraise different aspects of rigour. There was a high levelof agreement between researchers in applying the tools to twenty-twostudies on decision making by professionals about the long-termcare of older people. The scales for internal validity and generalizabilitydiscriminated between the qualities of studies appropriately.The two tools to appraise data collection gave diverse results.Excluding studies that scored in the lowest category on anyscale appeared to be the scoring system that was most justifiable.This approach is presented to stimulate debate about the practicalapplication of the evidence-based initiative to social workand social care. This study may assist in developing clearerdefinitions and common language about appraising rigour thatshould further the process of selecting robust research forsynthesis to inform practice and policy decisions.