An empirically-based study by Coenen et al. In a recent methodological study gated , we followed a similar approach used by Guest et al. We found the majority of themes were identified within the first focus group, and nearly all of the important read most frequently expressed themes were discovered within the first three focus groups Figure 2. Figure 2. Average number of new codes identified per focus group focus groups randomly ordered Guest et al. Note that these sample sizes, for both interviews and focus groups, apply per sub-population of interest.
Note too that thematic saturation will vary based on a number of factors keep watch for a future blog post and sample size should be adjusted accordingly. In those cases, a homogeneous sample would typically be single gendered.
In contexts where gender might play a very minor role in response variability — say, for e. My 2 cents…. I agree with Greg. Keep an eye out for a blog post discussing how the level of sample homogeneity — and other factors — might affect thematic saturation, and therefore sample size!
Very interesting. Very useful. If I understood the conclusions correctly, it does go back to traditional rule of thumb approaches; i. The catch, or course, is that one should be carefully reviewing scripts as data is collected.
I like the idea that it confirms one may make advance decisions on set targets. Thanks, Jane! Wonderful overview of the literature, thank you so much for this! However, I always take these numbers with a pinch of salt, saturation will obviously vary greatly with topic area and research question. Yes, absolutely, Daniel! Thanks for your comment and link. Stay tuned for the next post in this series that will address some of the factors that affect saturation — to help identify whether a small or large pinch of salt should go into saturation-based qualitative sample size calculations!
Thanks for this post — really useful. One question I have is, did any of these studies consider the interviewing skills of the interviewer? Or mention what training the interviewers had in advance of doing the semi-structured interviews? Hi Mia, great questions! As an aside from the sampling discussion, interviewer training really is key to generating good qualitative data — and not just training in terms of interviewing skill, but also in making sure that everyone has a common understanding of the research objectives.
Some people have a more natural affinity for interviewing than others, but if you can provide your interviewers with pretty immediate feedback e. May I ask if you were conducting surveys via email — what would be an acceptable number of completed surveys to aim for? Thanks for this illuminating post. I have now collected data from 2 different states were I conducted 13 interviews and 3 FGDs in the first and 8 interviews and 1 FGD in the next.
When I started collecting data from the second state, I reached data saturation much quicker and for the FGDs i was not getting much different data from the first state. I am now wondering if I have taken the right approach. Was the interviews and FGDs recommended per each round of data collection? Be careful not to conflate dyads and triads with a small focus group. Dyads are conversations that involve two participants who may or may not know each other. Triads are comprised of three participants who may or may not know each other.
Similar to in-depth interviews or focus groups, triads are good formats for moderators to drill deeper into the subject or topic being explored.
Triads also allow moderators to observe how triangulation may influence perceptions and responses. When it comes to focus groups, more is not always better.
Often, the advice given is to follow the rules of thumb but to suggest a slightly higher number to be "on the safe side". Some authors suggest that the researchers then use point of saturation as they go along to decide the final number of groups needed [ 2 , 4 ]. While some of these text books still refer to Glaser and Strauss, others do not.
In these cases the concept of theoretical sampling and theoretical saturation and the theory behind it seem to have been lost on the way and replaced with the less theoretically sustained terms of "purposive sampling" and "data saturation" or merely "saturation".
However, the basic procedure of selecting informants and deciding on the number of groups through a constant process of analysing data and obtaining new data until no new essential information is found, remains. It has been claimed by critics within the field of qualitative methodology that the concept of saturation is often misused in research reports [ 23 , 26 ].
Charmaz [ 25 ] claims that "Often, researchers invoke the criterion of saturation to justify small samples - very small samples with thin data. Whether "saturation" has thus become the magic word which obscures faulty designs in focus group reports, as Charmaz insinuates, has yet to be investigated. In general, empirical research into how focus group methods are used and work is scarce. We have found only two review studies that document and discuss sample size in focus group studies [ 7 , 8 ] and to our knowledge no study has yet assessed how decisions about sample size in focus group studies are reported.
Nor does the effectiveness of different sample sizes appear to have been evaluated. This lack of empirical evidence suggests that advice offered with regard to sample size is, as a rule, based on common assumptions or personal experience with the method. Our objective is twofold. First, we aim to survey sample size in current focus group studies in health science journals. Second, we aim to survey and assess the extent to which researchers describe and justify the number of focus groups they carry out.
We carried out a structured search for papers that included focus group methods, and extracted and analysed quantitative and qualitative information about the authors' use of these methods.
We searched PubMed Central in for primary studies that used focus groups and were published in We chose PubMed Central because it offers good coverage of medical journals and allowed us to apply a "Free Full Text filter" to limit searches to publications from open-access journals. This approach gave us a manageable sample of studies and easy and immediate access to papers, and allows others to easily access publications cited in this review and check our conclusions.
As mentioned, definitions of focus group interviews usually refer to researcher-initiated gathering of a small group of people with the aim of facilitating discussion about a given topic [ 1 , 4 — 6 ]. However, we included any study where the authors themselves used the term "focus group" to refer to their method of data collection. We included studies using focus groups only and studies using mixed methods. We chose studies published in as we started working with the review in and wanted to review the state of the art in the field.
In addition we repeated the same search for and , i. We excluded papers that described planned studies and papers that described internet-based focus groups as we assumed that recruiting participants for such groups possibly invokes other challenges than recruiting participants to face-to-face groups.
The search was conducted on December 21 st One review author BC independently assessed the relevance of full text versions of all papers identified from the electronic searches. Decisions to exclude papers were checked by the other review author CG. Both authors independently extracted data from the first ten included studies using a standard form based on the variables we aimed at analysing. The data were then checked against each other, and, if necessary, reference was made to the original paper.
Any discrepancies between the two data extraction sheets were discussed by the two reviewers and resolved by consensus. For the remaining studies, one reviewer BC extracted the data alone and checked with the other author when in doubt. In addition, we carried out a matrix-based qualitative analysis of the texts. Here, we explored the study reports adopting the same approach that we would have used for transcribed interview material, using a cross-case thematic analysis described by Miles and Huberman, among others [ 27 ].
Specifically, we extracted and condensed relevant passages from all included papers in a separate document with our comments regarding level and style of reporting and the authors' explanations for sample size.
We especially focussed on:. Whether and how the number of groups were referred to by the authors when discussing study strengths and limitations. Through our electronic database search we identified papers published in In comparison, our search for and resulted in 23 and 62 studies respectively.
We considered the full text version of each of the papers from for inclusion in the review. Twenty papers were excluded after reading the full text version.
Of these, six were excluded because they were planned studies, four were excluded because the focus groups were Internet-based, three were excluded because they were not primary studies and seven were excluded because they did not, in fact, report the results of focus group studies even though the term "focus groups" appeared in their abstracts.
Some of these referred to focus groups as part of an intervention that was evaluated in the reported study. Two hundred and twenty papers met our inclusion criteria. See Additional file 1 for a list of included studies. These were published in different journals. In addition, it was sometimes difficult for us to find focus group and participant numbers as this information was sometimes reported in different sections of the papers See for example [ 28 , 29 ].
For mixed method studies, it was sometimes difficult to separate between sample size information for different data collection methods See for example [ 30 ]. Those papers that did report numbers of groups and participants showed a great range in these numbers, but data distribution was positively skewed, i. Authors' explanations of how they had decided on or ended up with the number of focus groups carried out varied, but were often unclear or completely lacking e.
See Table 2. When authors used mixed methods, explanations of sample size for the quantitative part of the study were often meticulous, while the sample size of the focus interviews in contrast was often unclear and superficial, as in this example:.
Often, authors explained the number of participants rather than the number of focus groups [ 34 , 35 ]. Typical for these cases were situations where focus group studies were carried out alongside clinical intervention studies and where the authors had invited all participants from the intervention study to participate in the focus group interviews [ 34 , 36 ]. As these accounts explain the number of participants rather than the number of focus groups, this was categorised as "no explanation".
From Table 2 we also see that no study justified the number of groups by referring to the need to balance data quality and quantity capacity to analyse data. The table also shows that all the explanations for sample size were found in studies that had between two and 13 focus groups. We tested if there was any correlation between the presence of an explanation for number of focus groups and the number of groups conducted Pearson correlation.
We found no linear relationship. None of the eleven single-group studies attempted to justify why one group was sufficient. These studies all use mixed methods and the qualitative assessment showed that they typically used the focus group study as a pilot for developing questionnaires etc. As noted, none of the studies that included more than 13 groups gave any explanations for sample size and the qualitative assessment showed that none of the studies referred to a high number of groups as a study limitation.
On the contrary, when authors considered their sample size to be relatively high, this was seen as advantageous:. By conducting a large number of focus groups with a significant cross-section of the facility's employees, we reduced the possibility that results would be dramatically affected by a single focus group or methodological choice during coding [ 37 ]. On the other hand, when authors had included only a small number of focus groups, they frequently described this as a limitation.
However, in several of these studies, authors also claimed the number of focus groups had been determined by data saturation. This should, in theory, imply that the number of focus groups was, in fact, appropriate.
See for example [ 38 — 43 ]. But while authors often described a small number of focus groups as a weakness, several studies justified their choice of method with reference to the possibilities that focus groups gave to go in depth and provide a thick description of the issue.
Among the 37 studies that did give an explanation for the number of focus groups, 28 claimed that they had stopped once they had reached a point of saturation. However, more than half of these explanations 15 of 28 did not report convincingly that an iterative process, involving data collection and analysis that ended with saturation, had taken place. The reason for this was mainly inconsistencies in the description of the methodological procedures.
One common example was that, despite their reference to point of saturation, their number of focus groups was pre-determined, as the extraction below shows See also [ 44 , 45 ] :.
Three focus groups were conducted with residents of a specific community. It was a small community, and it was determined that after three focus groups, there would be a saturation point regarding their barriers and suggested solutions [ 46 ]. Other common examples where claims of saturation appeared unsubstantiated were studies that used convenience sampling, and included everyone who volunteered to participate, instead of purposive or theoretical sampling, and where, in addition all data seemed to have been gathered before the analysis See for example [ 41 , 42 , 47 — 49 ].
We also noted that no author reported or discussed the number of focus groups that had been conducted with no new relevant information before it was decided that a point of saturation had been reached. On the contrary, authors normally gave superficial and vague references to the process of reaching point of saturation, many of them referring to a "feeling" of having reached this point:.
Focus groups were [ At the end of the fifth focus group, we felt that theme saturation had been met, so we stopped recruitment We also found some examples of adequate reporting regarding point of saturation.
For example Barimani et al [ 51 ] explained their use of grounded theory and described how a constant comparison between empirical findings and theory had guided their sampling procedure until theoretical saturation was met.
This paper was, however, unusually long, around words. Another example is Shuval et al [ 52 ]. In their paper of around words, the authors have not specifically referred to the term "saturation" but have offered a clear description of the sampling procedure and how saturation was met:.
We used purposeful sampling [with reference to Patton ] to select key informants on the basis of researchers' acquaintance , promote group interaction, and capture the diverse characteristics of participants i.
All students whom we approached agreed to participate in focus groups. Focus groups were held until no new themes emerged 9. Six of the 37 studies that gave an explicit explanation for the number of focus groups referred to rules of thumb in the literature.
These were mostly references to pragmatic guidelines of how many groups are necessary to reach a point of saturation e. As the example below illustrates, authors reported that the field lacks consistent guidelines. Recommendations for both the number of focus groups and sample size vary. The number of recommended sessions depends on the complexity of the study design and the target sample's level of distinctiveness [ 18 — 21 ]. Stewart et al. We felt that two groups at each site would limit bias that might be seen in a single group or site and allow us to examine themes common across groups [ 54 ].
However, when we checked with the literature, referrals to recommendations were not always accurate. For example Gutterling et al, conducted three focus groups and explain the number thus:. For good results, just a few focus groups are sufficient, as data become saturated and little new information emerges after the first few groups [Morgan 96] [ 55 ].
Looking up Morgan [ 1 ], we found that he claims that most studies use four to six groups because they then reach saturation, but he also underlines that the more categories of participants and less standardisation of questions, the higher number of focus groups. Three of the 37 studies reported practical reasons for the number of focus groups conducted.
The information offered regarding recruitment constraints was incomplete as, for example, in this explanation:. The number of available participants was limited and the number of focus groups was therefore few [ 56 ]. Two of the explanations appeared to be tied to difficulties in recruiting participants to additional groups [ 56 , 57 ], while one mentioned limited resources "budgetary and staffing constraints" which led the researchers to decide pre-study to conduct five focus groups [ 38 ].
This last study also claimed to have reached saturation without stating this as the procedure for deciding number of groups :. However, data analysis indicates that all themes reached saturation, meaning additional participants would likely not have added to the depth or breadth of parent responses [ 38 ]. An additional eight studies also described recruitment limitations, but only as an explanation for the total number of participants, not for how many groups the participants were divided into.
In these studies, the size of the groups seems to have been decided beforehand, due to text book recommendations, and thus the number of groups was given by the total number of already recruited participants divided by the number of participants per group.
The results from our searches from , and support the claims that there has been an increase in focus group studies over the last ten years. The wide range of health journals publishing focus group studies in indicates that this method is now widely accepted. At the same time, the fact that many journals publish only one or two focus group studies a year could also mean that the methodological competence among editors and reviewers to assess focus group studies is lacking.
The great variation in the number of focus groups that we registered was surprising, and was wider than authors of teaching materials and text books assume.
For example Stewart et al [ 13 ] claim: "Most focus group applications involve more than one group, but seldom more than three or four groups.
Overall, reporting of sample size and explanations for this size was poor. Where such explanations were given, our study confirms the dominant role of the concept of data saturation.
0コメント