- Scale development is one the vital approaches for associating abstract concept(s) to empirical indicants according to specific rules and appropriate overarching theory. It is about the accuracy of a measuring instrument and whatever it measures rather than truthfulness of the measurement.
- You need to define the concept and its possible sub-components. In so doing, you need to search and review the previous literature to find out if there is a current scale exists, but also how the concept has been defined formerly. Then, you would ask questions such as: Is there any scale for measuring the potential concept you have in your mind?; Why do we need a new scale?; What is wrong with previous scales?; Is it a multidimensional or monodimensional concept?; and what are the potential application of the new scale?
- You typically conduct a deductive/inductive qualitative approach using series of semi-structured interviews. These interviews can be in one stage or could have a follow-up interviews based on the nature of the study. You then can generate an initial pool of items reflecting different aspects of the concept. Content or thematic qualitative analysis frequently used for open coding and carefully evaluating individual words, phrases and sentences. However, some studies also used deductive approach employing Spearman correlation coefficient to compare different groups of informants. Again, this depends on the nature of the study and authors.
- You need to test for content and face validity to find out if a particular set of items reflect on a content domain. Some studies also called this stage as initial purification or expert panel. Again, you could use judgment panel or Delphi study or small survey of expert judges to evaluate the conceptual definition of the concept or possible dimensions. You then ask expert judges to rate the items (typically) using a 3-point scale (1 = not at all representative, 2 = somewhat representative, and 3 = completely representative). This procedure will help in the retention of items with regards to the concept as well as the likelihood of identifying potential items under each possible dimension.
- You can then conduct the first main survey of informants in regards to content and context of your study. It is crucial to select right types of informants to avoid different kinds of biases, particularly social desirability bias. The appropriateness of the items for capturing the concept (or potential dimensions) should be tested using exploratory factor analysis (EFA). EFA results might interpret differently based on different types of measurement scale including reflective, formative and higher-order scales. Please, seek out for further information on these modes in scale development papers and books.
- Next step is scale validation. The purpose of this new follow-up study is to confirm the dimensionality structure of the scale and also to establish convergent, discriminant, nomological and predictive validity. In so doing, you normally employ a confirmatory factor analysis (CFA); suggested fit indexes; HTMT and/or Fornell and Larcker’s discriminant validity approaches; average variance extracted (AVE); Cronbach’s Alpha; and Competitive reliability (CR) to confirm the nature of the concept and its possible dimensions. You then use correlation and/or multiple regression/structural equation modelling (SEM) to test relationships between the new scale and other related constructs based on previous studies. You may also compare the scale and its potential dimensions (sub-scales) across (normally) two sub-groups of informants. Again, this may change based on different types of measurement scale. Please, seek out for further information on statistical and scale development papers and books.
- The last stage is replication and generalisation stage. You need to conduct a new study to compare the performance of the new scale with the other existing scales in several cultures/countries (i.e., cross-validation in large sample) based on the previous theoretical framework by using correlation and/or SEM. In other words, this is for further justification of the predictive and nomological validity of the new scale. There are various ways to select countries for the cross-cultural analysis, but it all depends on the nature of the study.
- Last but not least, an excellent literature review and overarching theory is the key for scale development papers. You should follow recommended scale development steps as closely as possible. You should carefully explain different steps including samples and why selected, how many items kept/skipped and on what basis, any issues etc. Please, have a look at previous papers while you are writing your paper. As there is no consistency, please note advantage and disadvantage of the different papers and reflect on your paper. And always ask: Do I really need to develop a new scale? Why?
Picture by Sonny Abesamis under CC license.