Clear & Simple
Achieving Quality and Effectiveness in Health Communication
NIH Office of Communications and Public Liaison
For many Americans, instructions, directions, signs, advertisements, notices, applications, forms, and even bus schedules have become everyday challenges. While print materials still form the cornerstone of health and social services information, some materials may be of little or no use to people with limited-literacy skills. This guide, Clear & Simple, is designed to assist health communicators in developing audience-appropriate information and communicating effectively with people with low-literacy skills.
People with literacy problems are found among all ethnicities, races, and classes but there is a link between literacy and education and income levels. Many of the same populations at risk for limited health literacy also suffer from disparities in health status, illness (including heart disease, diabetes, obesity, HIV/AIDS, oral disease, cancer deaths, and low birth weight), and death.
In the United States, hundreds of languages are spoken or signed and in some cities, less than 60 percent of the population has English as a first language. The Hispanic/Latino and Asian American populations are growing in the United States at dramatic rates. By 2025, almost 40 percent of Americans and about half of all U. S. children are projected to be members of minority populations [Source: Pew].
Clear & Simple outlines a process—five standard steps—for developing health information materials for people with limited-literacy skills. The 2003 National Assessment of Adult Literacy found that about 14 percent of 18,500 adult Americans surveyed could not read, or understand text written in English and could only comprehend basic, simple text.
- Step 1: Define the Target Audience
- Step 2: Conduct Target Audience Research
- Step 3: Develop a Concept for the Product
Step 1: Define the Target Audience
A target audience is the group of people you want to reach with your message. People with limited-literacy skills compose a broad target audience, crossing all ethnic and class boundaries. There are some common characteristics among people with low-literacy skills, for example how they may interpret and process information. It is important to keep the following in mind as you develop materials:
- Some individuals think in concrete/immediate rather than abstract/futuristic terms;
- Some people lean toward literal interpretation of information;
- Not everyone shares sufficient language fluency to be able to comprehend and apply information derived from written materials;
- Some people may have difficulty processing information: reading a menu, interpreting a bus schedule, following medical instructions, or reading a prescription label.
The target audience may be defined by age, sex, marital status, educational level, occupation, income, religion, race, ethnicity, language, geographic location, lifestyle, health-related attitudes and behaviors, and many other characteristics. These may influence each step in the process of developing low-literacy materials, so understanding them is important.
- Case 1: Pretest the product with members of the target audience. A research institute begins a clinical research study including the participation of Spanish-speaking patients, who take a new drug. The institute develops state-of-the-art educational handouts as part of the trial. One product is a fact sheet and poster about the benefits of taking the medicine each day as instructed. A brightly colored wall poster greets patients with a cheerful salutation in Spanish, urging them to take a 10 mg. coated pill “Once Each Day.” An elderly woman in the waiting room spots the greeting on the poster and wonders if the doctors really mean for her to take eleven pills each day.
- Case 2: Develop culturally appropriate materials for reaching audiences at all literacy levels. The director of a state-funded clinic in a Western city is asked to increase outreach to local Native Indian communities. He tells his staff that the clinic can't develop materials for every community. All materials—photos, diagrams, and messages—will have to work for everyone. Recycling an old brochure with a few new design elements will be OK. A few months later the newly redesigned campaign products are released to the local community along with a photocopied cover letter. Tribal leaders immediately begin to contact the clinic and Congressional offices to complain about the inappropriateness of the brochure—especially its photos—and expenditures of state funds to produce materials not reflective of their culture.
Q: Should I develop a low-literacy product or a regular publication at a lower reading level?
A: General public audiences may not require special low-literacy-readers' aids and tools to help them comprehend information. The goal is to reach individuals with very poor or marginal written communication skills who may benefit from materials written in simpler language. The two audiences are distinct from one another.
Q: Why focus specifically on low-literacy readers?
A: Many ambitious and meritorious information campaigns are designed at a 9th to 10th grade reading level. Planners likely base the decision to target low-literacy-readers after researching public knowledge of a particular health topic. Despite the best efforts of health communicators, there may be little if any gain-in-knowledge among some audiences. Messages directed to the general public may not get through to all of the target audience. Demographic profiles can help communicators target messages at people with low-literacy skills and other groups missed by campaigns. Once you decide that you need to reach readers with limited-literacy skills, you can define that audience further and develop appropriate messages.
Q: Can I communicate effectively to people with average or good reading skills by using a low-literacy format?
A: Readers appreciate messages that are conveyed simply and clearly. Readers who want more detail can be directed to sources of in-depth information. Testing your product with the audience is the only way to evaluate the effectiveness and appropriateness of materials and is discussed in greater detail later in the guide.
The National Institute of Healthís Making Health Communication Programs Work: A Planner's Guide (1989) provides further details on how to define target audiences.
Step 2: Conduct Target Audience Research
It is important to understand relevant physical, behavioral, demographic, and psychographic characteristics of your audience. Research can help you define the specific ethnic, cultural, and lifestyle preferences of your audience. This information plays a key role in the development of culturally relevant materials. It is also critical to the goal of reaching audiences at all literacy levels. Research can tell you:
- What the target audience already knows about your topic;
- What rumors, myths, and misinformation may exist about the topic;
- How audience members feel about the topic; and
- What questions and information gaps you will need to address.
Target audience research includes review of existing data and in some cases, the development of new data. The first task is to explore existing sources of information. These include library databases; health statistics gathered by Federal, state, and local agencies, including health departments; information offered by nonprofit organizations that may have worked with your audience; and results of polls.
If critical information about your audience is not available in existing data, you may have to conduct new research to fill in information gaps. National statistics may not capture unique characteristics of your target audience, so whenever possible, supplement national data with local population data.
Information You Need About Your Audience
- Age, sex, income, educational level, career path, residence, and community or cultural identification;
- Relevant behaviors (e.g., teen risk behaviors, thrill seeking behavior, views toward or participation in violence)
- Related knowledge, attitudes, and practices (e.g., stigma, shame, discrimination)
- Patterns of use of related services (urban vs. rural, access to services, travel distance and time)
- Cultural habits, preferences, and sensitivities related to your topic;
- Barriers to behavior change (e.g., access to public transportation, quality of care, depression)
- Effective motivators (e.g., benefits of change, fear of consequences, incentives, or social support)
- Surveys that measure the respondent's knowledge, attitudes, and practices on a specific topic are conducted by telephone, mail, online, or through face-to-face interviews with members of the target audience.
- Advantages: Surveys provide highly targeted, directly relevant information and can provide estimates representative of the total population.
- Disadvantages: Surveys require time, statistical expertise, and ample resources to accomplish. They also require a mechanism for locating and reaching large numbers of your target audience. Mail surveys may be inappropriate for readers with low-literacy skills.
- Advantages: Focus groups require fewer audience representatives and are faster than other research methods. They help explain why members of an audience believe or act as they do and allow for in-depth discussion of issues. Sometimes, they can be less expensive than other approaches.
- Disadvantages: Focus group findings are qualitative, not quantitative, meaning they require expertise in conducting, reporting, and applying results appropriately. They also require you to locate and motivate members of your target audience to participate. Some findings may not extend to the total population.
- Advantages: Interviews may yield more in-depth information than focus groups.
- Disadvantages: Individual interviews are both more time- and labor-intensive than focus groups.
Q: I don’t have the time or budget to do new audience research. Do I have to apply these methods in order to develop an effective low-literacy product?
A: The best approach is to make audience research a part of your product-development process for audiences of all literacy levels. Then the time and budget for research will be built-in automatically. What are the alternatives to formal research? You may be able to seek input from target audience members who serve on an advisory board or from individuals who have close working contacts with the target audience. If you rely on indirect information sources, remember to pretest the product with members of the audience themselves. It is important to get direct audience involvement at some point to test the comprehensibility and appropriateness your materials. Pretesting should supplement earlier findings and confirm that the product is on target.
Q: How can I get information specific to my product if I do audience research before I develop my concept?
A: Audience research can take place at one or more stages as the developers of product development. You can test key ideas you may have in mind at the beginning of the planning process. Audience research can also help you decide if a key message that you have always used in your products is still effective or if it will work with a new audience. You may wish to see if a simply crafted message conveys a complex concept clearly. Just as the story at the beginning of the guide showed, audiences can play a key role during early planning in determining whether spokespersons are credible.
For more information about research approaches, see Making Health Communication Programs Work: A Planner's Guide (1989) and Family Planning Print Materials for Low-Literate Audiences: A Guide.
Step 3: Develop a Concept for the Product
You are now ready to use information from your audience research to outline the objectives, style, format, and approach of the product that will carry your message. Many writers prepare a formal concept statement at this stage for those involved in the productís development. Discussing the concept with individuals or groups who understand the needs of your target audience is an easy, inexpensive way to double-check the appropriateness of your intended approach. It is, however, no substitute for pretesting materials later with your target audience. Researchers Leonard and Cecilia Doak pioneered the field of health and low-literacy. In their ground-breaking work, Teaching Patients with Low Literacy Skills, (J.B. Lippincott, Philadelphia, Pa., 1985), they suggest five principles to use in developing the concept for a low-literacy publication:
- Define the behavioral objective(s) of the material.
Some products are purely informational; others are designed to move target audience members to new behaviors: new ways of thinking and improved understanding and awareness. Concrete action objectives help guide planning decisions and avoid unnecessary details in content. For example, a new diagnostic factsheet designed for women with low-literacy skills might tell readers that getting a test is important for good health. Another product may focus more on behavioral objectives: improving diet, losing weight, reducing stress, and increasing exercise. Another factsheet would provide readers with a toll-free telephone number to call or contact information for a new clinic in their neighborhood.
- Determine the key information points the reader needs to achieve the behavioral objective(s).
What will move the reader to take the desired action?
Information from audience research can help inform planning and may answer a number of important questions, such as how and when a product will be used. Will the product be used by the reader alone? By a professional trainer? Or during an appointment with a health or social services professional?
The time in the learning process when the reader receives information is also important. Is the product designed to address an illness, diagnosis, treatment intervention? Or will it be used during recovery and rehabilitation? Will the reader likely have already received related information?
The process of concept development is also affected by whether a product or products stand alone or whether they are part of a series of materials. Additionally important is whether materials are component products within a broader communications plan, such as an awareness campaign or recruitment effort.
Another consideration is the careful selection of resources that might be useful and relevant for the reader. It may be appropriate to include a resource list of key organizations and Web sites where the reader can obtain additional information.
- Select the most appropriate presentation method(s)
The 21st century offers a wide range of outlets through which health professionals can convey important messages. New and emerging formats include social media tools, mobile applications, streaming media, videoconferencing, and many other Internet-based options. These supplement traditional approaches, such as brochures, posters, and toolkits. It is not uncommon for a single well-written and well-produced printed item to serve the needs of the target audience—provided it is culturally appropriate. So at the most basic level, it is important to determine early on the appropriate medium for conveying a message to a target audience.
If you choose print, you must still decide between a 1-page factsheet, a 10-page booklet, or a series of short factsheets delivered at different points in the learning process. Budget and target audience information help shape decisions about the length and scope of the product. Design questions include photo and graphics selection, layout and color, packaging and paper stock. Some questions to consider:
- How will the product be distributed: in person, on a rack in a retail or office environment, in the mail, in a clinical setting, or through another method, such as on a mobile device or tablet?
- Is the material intended for one-time or long-term use?
You will also have to look at the setting in which you want audience members to receive their information. Offices and waiting rooms can provide communicators with an opportunity to develop a slightly more robust product than if the audience will access information on a mobile device. It may not be possible to meet information goals if you focus solely on a bus advertisement or in pharmacies, when readers’ time is limited. If your target audience is at-risk teens, then careful planning is required. Addressing their unique communication needs will have important implications for the tone, structure, and design of products.
The term "reading level" refers to the number of years of education required for understanding a written passage. To account for a probable decline in reading skills over time, some experts suggest aiming for a level that is 2-5 grades lower than the highest average grade your target audience members have achieved. Others recommend that a 3rd to 5th grade level is often appropriate for low-literacy readers.
How do you estimate reading level? A number of readability formulas are often used to assess the reading level of materials. Reading levels and readability formulas are useful aids in targeting publications to an audience. Using one or more of them is a simple process that can be accomplished manually or with a computer software program. Each method takes only a few minutes. Among the better known formulas is the SMOG Readability Calculator.
Readability formulas only measure the difficulty of the vocabulary used and average sentence length, not the reader's level of comprehension. Computer-based programs analyze a document's grammar, style, word usage, and punctuation and then they assign a reading level. Using such formulas in product development is not a guarantee of well-written, understandable content. Creators acknowledge that using the formulas as writing guides can have negative consequences. If sentences and words are too short, the result may be a choppy product that leaves out familiar terms solely because they are polysyllabic. In addition, some formulas may not be applicable to readers with low-literacy skills. Some other means of assessing the effectiveness of the material is almost always required. Most importantly, make sure you write for your audience.
Provide readers with the most important points first and last. Studies show that low-literacy audiences remember these best.
Literacy experts also suggest presenting information in chunks of text with a clear, ordered format, one idea at a time. You can outline these as steps (Step 1, Step 2, Step 3) or chronologically (first, second, next, last). Another approach is to use a main heading and subheadings, depending on how you want the person to access and comprehend the information.
The goal is to give the reader action steps to improve information retention and immediately move the individual into desired behaviors. Examples might include having them call a clinic or send off a request to receive more information.
SMOG Readability Calculator
Count the number of words with three or more syllables in a 30-sentence section of your draft. Using this chart, look up the approximate grade level. The SMOG formula can predict the grade level difficulty of a passage within 1.5 grades in 68 percent of passages.
SMOG Conversion Table
Grade Level (+1.5 Grades)
|1 - 6||5|
|7 - 12||6|
|13 - 20||7|
|21 - 30||8|
|31 - 42||9|
|43 - 56||10|
|57 - 72||11|
|73 - 90||12|
|91 - 110||13|
|111 - 132||14|
|133 - 156||15|
|157 - 182||16|
|183 - 210||17|
|211 - 240||18|
Q: I have to limit my publication to a few key points. How do I decide what the reader needs and what I can leave out?
A: Focus on your behavioral goals. Is each information point fundamental to the reader understanding, accepting, and taking the desired action? Does each information point help to motivate a desired action? One approach is to separate need-to-know from want-to-know and want-to-tell. Some if not all of the information in the second two categories can probably be safely eliminated. Tough decisions always remain. Pretesting can help determine what information the audience really needs.
Q: Writing at the correct reading level for the audience is important. What should I use to help me write at an appropriate level?
A: Good writers may not trust their usual instincts to tell them what will work for people with limited reading skills. Look at the checklist in the next section. It outlines the fundamental principles of writing and designing a low-literacy publication. The checklist can help you address your readers' needs and provides a basic framework for writers. Readability formulas have both value and limitations. A writer may set out to write simply, using an action-oriented approach. It is a good idea to then put down the draft for a while and return to it with a critical eye. Rewriting to achieve appropriate flow and comprehensibility and applying readability formulas can help. Readability formulas and software can be useful in approximating a reading level or, in the case of some software products, identifying specific problems that inhibit readability.
You may test your product at one level and find you have to simplify some of the text, taking care not to detract from the message and goals of the product. Pretesting may also show that one or two concepts are still not well understood.
For more information, see Teaching Patients with Low Literacy Skills, Cecilia Conrath Doak, M.P.H. and Leonard G. Doak, BSEE, PE(J.B. Lippincott, Philadelphia, Pa., 1985) and Making Health Communication Programs Work: A Planner's Guide (1989).
Step 4: Develop Content and Visual Design Features
Now that you have a product concept, you can begin outlining and writing the first draft. What you include will draw heavily from your important early—and continuing—research. In developing a low-literacy product, you will need to tailor content, layout, and visuals (i.e., photographs, illustrations, and diagrams or charts) to the needs of a reader with poor reading and communication skills. What does this mean in concrete terms? There are several key principles to use in creating effective materials for this audience. You can refer to the list as you draft and review your work:
- The material is interactive and allows for audience involvement.
- The material presents "how-to" information.
- Writing reflects peer language whenever appropriate to increase personal identification and improve readability.
- Words are familiar to the reader. New words are defined clearly.
- Sentences are simple, specific, direct, and written in the active voice.
- Each idea is clear and logically sequenced (from the reader’s perspective).
- There are a limited number of concepts in each piece.
- The writer uses concrete examples rather than abstract concepts.
- Text highlights and summarizes important points.
- The material uses advance organizers or headers.
- Headers are simple and are close to text.
- Layout balances white space with words and illustrations.
- Text features both upper and lower case letters.
- Underlining or bold formatting—not caps—provides emphasis.
- The font selection (design, size) are easy-to-read. Opt for a 12 point or larger size.
- Visuals are relevant to text, meaningful to the audience, and logically located.
- Illustrations and photographs are simple and free from clutter and distraction.
- Visuals use age-appropriate images selected with the age of the target audience in mind.
- Illustrations show familiar images that reflect cultural context.
- Visuals (graphics, photos) have captions written with active verb constructions. Each visual is illustrative and is directly related to one message.
- Visual elements (e.g., photographs without background detail, shaded line drawings, and simple line drawings), are shown in pretesting with the audience to be appropriate and conducive to information retention.
- Cues (e.g., circles or arrows) point out key information.
- The color palette is appealing to audience members during pretesting.
- Readability analysis has been carried out to determine reading level.
Q: Do readers with low-literacy skills need to know technical terms, especially if the words raise the reading level of a publication?
A. There is no absolute answer to this question
Q: How can I keep material simple as it goes through expert and/or organizational review?
A: The simplicity of quality, effective low-literacy products may startle some reviewers, especially those accustomed to scientific or technical publications. They may be unfamiliar with low-literacy approaches and may be concerned that a product written at a low reading level may reflect poorly on the expertise of an agency or organization. There is ample data and scientific information in support of the development of low-literacy health materials. This can and should be shared with reviewers. Involve reviewers at the concept development stage so that they are not surprised when they receive the draft of the product. Work personally with reviewers. Make sure that all simple explanations are accurate. Do not distort the scientific or technical facts as you pare away details. If a suggested change is inappropriate, discuss both of your concerns and work cooperatively toward a solution. Test reviewer-inserted concepts, specifically during prepublication evaluations. If the reviewer's idea does not work with the audience, you will have a firm basis for change.
Q: Are pictorial signs, symbols, and charts more effective than words for readers with low-literacy skills?
A: Not necessarily. Some experts suggest that "universal" symbols, such as a stop sign, an arrow, or a big black "X," usually test well with this audience. If a pictorial representation is open to interpretation, however, it can fail to communicate with any audience. Likewise, while a simple chart may work well, a large matrix or visually busy schema are likely to confuse. For example, functionally illiterate individuals may even have trouble using a bus schedule.
Q: I know that low-literacy products should focus only on a few key concepts. How do I handle a complex topic with 8 or 10 important messages when I can only afford to do one low-literacy publication?
A. A strong grouping of main and sub-points is a workable solution to this problem. When individual sections are organized effectively and when each can stand alone, readers can approach the text at their own pace.
Q: My budget doesn’t allow for illustrations and I cannot use color because my products will need to be photocopied. Can I still design an effective low-literacy product?
A: Although color is a powerful communication tool, strong format, good use of white space, and alternative highlighting devices can help a black-and-white product convey your message. In addition, low-literacy products do not always need illustrations to break up text. Boxes, lines, and white space can keep a design from being too copy-dense. Pictures are valuable if they illustrate an action or a key point. They do not have to be expensive to produce. In fact, simple line drawings may be preferable to detailed pictures for low-literacy audiences. Even off-the-shelf graphics can be effective if the selection fits the message and tests well with the audience.
Q: I cannot afford to do separate low-literacy publications for all of our organization's publications. Is there an effective way to adapt higher level reading materials for low-literacy populations?
A: Some professionals who work with low-literacy audiences are adept at selecting and highlighting key concepts from products written at a high reading level and using them to create separate products for low-literacy audiences. Formatting features, such as underlining, circles, stars, or arrows can help professionals adapt existing materials into new ones for target audiences. This approach is far from ideal, but it does meet audience needs better than using materials produced at high reading levels. Writers can also meet the needs of both audiences by using headlines and subheads to carry key message points, in logical order. Readers can choose to skip the details that the accompanying text provides.
Using Technical Terms
Technical terms can raise the reading level of a publication. But will readers understand them?
A person with diabetes may know technical words such as 'insulin,' 'glucose,' and perhaps even 'retinopathy,' because these are either a daily part of the patient’s life or because their doctor or nurse may have warned them to watch for a sign or symptom. Or they may have heard these terms from a friend, coworker, or family member.
When text includes a technical term, it is probably a good idea to offer a simple explanation next to it. Including a glossary can help, but it can also contribute to the reader's difficulty getting through text. Readers may not know that unfamiliar words are defined separately at the end of the materials. Others may not want to navigate between sections. There is also the risk that a glossary can tempt the writer to include a longer list of technical terms, perhaps with less clarity than if fewer items were accompanied by more clear explanations and spread throughout the information. You may want to apply a “need-to-know" concept to the decision about which technical terms to include. For example, if you want to inform patients that they are to consume a contrast agent prior to an imaging test, referring to it as a flavored milkshake might be less than truthful. It could also be interpreted as “talking down” to the reader. It may not be precise enough. But referring to it as a barium contrast agent is probably too technical and may not be fully understood.
Offering the technical term followed by an explanation or in a series of sentences will probably meet your needs and those of the audience. You might explain that contrast agents help coat the body’s organs to make structures and functions more visible. They also help create views of different organs and tissue types to aid with diagnosis. Continue by saying that "contrast agents include a number of chemicals, including iodine, barium, and barium sulphate. Barium sulphate is a chalky, liquid chemical with a metallic taste. It comes in milkshake form to disguise the taste of the chemical."
For some words, you may discover during pretesting that audience members understand what they need to do prior to undergoing the test, such as not eating for a certain period. Other terms may point to a need to do further rewriting.
Step 5: Pretest and Revise Draft Materials
“Developing low-literacy products represents a major change from business as usual. In some respects, it's like learning how to write, design, and test materials all over again.”
Communicators may find that audiences don’t understand a product perfectly. The decision to include a passage or illustration is subjective, no matter which principles are being used in the development of a product. That is why testing a material with the intended audience is important. Audience understanding and acceptance is critical to ensure that materials are culturally relevant and responsive to the needs and concerns of the target audience.
Pretesting is a qualitative measure of audience response to a product. A number of goals are important. Pretesting is essential for measuring comprehension. It is also important to measure audience attraction to the product, its acceptability to them, and the audience’s personal involvement with the material. Although funds may not be available for extensive pretesting, some pretesting is essential. The focus of this section is pretesting low-literacy materials.
What To Test For
Pretesting products can help communicators learn whether or not test respondents understand any behaviors the material recommends for readers. To some audience members, writing can be unclear, confusing, or hard to believe. Test questions can also address the suitability of words and the meaning or relationship of visuals to text. It is also instructive to learn about the meaning each respondent attaches to key words, symbols, abbreviations, and visuals. Next are a few examples of questions designed for pretesting a fact sheet related to eating a healthy diet:
- "What does the educational piece mean when it says to eat balanced meals? How do you do that?"
- "Which vegetables have lots of fiber?"
- "Looking at this picture, how will you cut down on fat in your soups or stocks when cooking?"
Pretesting can yield important information about the kind of feelings draft materials generate and whether or not audience members are enthusiastic or turned off by text. Communicators can ask audience members if the individuals depicted in the materials are appropriate representations for their community or group and if they connect with them. Next are two examples of questions concerning the attractiveness of a product being pretested:
- “Is there anything you don't like about the people or pictures in this material?”
- “Is there anything you don't like about the color and the layout of the material?"
There are many well-intentioned health communicators who attempt to launch products but end up offending their intended audience. Material may not be compatible with local culture or may not be realistic. The choice of a celebrity spokesperson can be risky; not all readers may recognize an actor, sports personality, or singer. Text may not be suitable for the sex, age, or ethnic group for whom a product is being developed, especially if the audience includes seniors or teens. Details, such as hairstyles and, clothing may be inappropriate for the product and intended audience. Here are a few examples of questions for pretesting a diet product’s attractiveness to respondents:
- "Do you think your family would be willing to cook their food this way?"
- “Was the information presented in a way that is interesting to you?”
Organizing a Pretest
Pretesting with audiences that include people with low-literacy skills can present communicators with unique logistical challenges. First, there is the issue of what to test and at which stage of product development. Experts recommend testing a rough draft of copy with graphic concepts explained and a few potential illustrations included. Another option is to pretest a typeset, early layout of the product with rough graphics in place. You can also pretest at both of stages of product development.
Some researchers believe it is best to test a product in the same type of environment where the reader will use the material. For example, if a patient will read the factsheet in a noisy, busy clinic, take care to expose test readers to the same distractions by pretesting in a similar setting.
You can organize a pretest in a number of potential environments and settings:
- Clinic or hospital waiting rooms
- Physicians’ offices
- Residential settings (homes, age-restricted communities, extended care centers, independent living facilities)
- Community facilities (churches, recreation centers, senior centers)
- Adult Basic Education and English language classrooms
- Government facilities (Social Security offices, WIC centers)
- Nonprofit offices (job training centers, wellness and day program sites)
Ideally, at least 25 to 50 members of the intended audience should review each product. If time and budgets are limited, it is far better to test material with 5 to 10 people than not to test it at all.
|Type of Interview||Advantages||Disadvantages|
Individual interviews, 10-20 minutes each
Provide "cleanest" results; good for short materials; less chance of bias among and between respondents.
Scheduling may not allow for speedy completion.
Group interviews, 8 to 10 people, 30-60 minutes each
Better for longer products (e.g., booklets, kits). Group discussion may yield valuable information not resulting from straight questioning
Requires advance organization and a trained group facilitator. Sessions must be paced to maintain respondent attention.
Engaging fewer participants may work if the material is very simple, short, or requires little in the way of attention for comprehension. A smaller group of participants may also work if the product is directed at a single, homogeneous target group. Conversely, your efforts may require more participants if the material is complex, lengthy, demanding, or culturally diverse.
Determining Reading Levels for Pretesting
“We knew the importance of developing our products as low-literacy materials. But during pretesting we found that better educated information seekers wouldn’t pick up our product if it was clearly aimed at a basic level.”
This poses an important question—and a lesson as well. Are you pretesting materials with the right readers?
Engaging participants who have the same characteristics as the low-literacy audience you are trying to reach is critical for achieving valid results. Recruiting participants through groups or settings that include people with limited-literacy skills is a logical starting point. But the only way to make sure your pretest volunteers read at the same level as your intended audience is to test their reading skills.
A number of tests are used to measure reading levels and comprehension. One or more can be integrated into the pretest interview to avoid offending or causing discomfort to potential participants and respondents. For example, in one pretest, interviewers introduced key language into the latter part of their pretesting process.
"Thank you for helping with the questions on the treatment booklet. We need your help with one last part: a review of technical words that appear in the booklet. You will assign a score to each word listed, ranging from very easy to very difficult. The exercise will help us learn how well our readers understood the booklet. This will take only a few minutes."
Caution: This approach must be used in such a way as to spare participants the pressure or embarrassment they might feel for fear of “failing” a reading test.
The Wide Range Achievement Test
The Wide Range Achievement Test (WRAT) is an efficient way to determine reading levels and takes only a short time to administer. The WRAT focuses on word recognition because at the most basic level, if a person does not recognize a word, comprehension is impossible. The WRAT test involves listening to the participant read from a prepared list of words, arranged in increasing order of difficulty. The reader looks at each written word and says it out loud. Pronouncing a word correctly shows that the reader recognizes the word. The test is over after the reader mispronounces ten words. The test administrator notes the level at which the last mispronunciation occurred, and the "stop" level equates to a grade level of reading skills. Planners can compare this level with the reading level of their intended audience to see if pretest readers are a good representative match.
The Cloze Technique
The Cloze technique is used to measure the reader's ability to comprehend a written passage. Because it requires readers to process information, it may take up to 30 minutes to administer. In a Cloze test, text appears with every fifth word omitted. The reader tries to fill in the blanks. This task demonstrates how well the respondents understand the text. Their ability to identify the correct word also reflects familiarity with sentence structure.
You can integrate either a WRAT or a Cloze test into the pretesting process by purchasing off-the-shelf tests or developing and scoring a test yourself based on the materials you are pretesting. The resources section at the end of this guide provides contact information for obtaining testing instruments.
Conducting the Interview: Special Considerations
Effective low-literacy products are designed to meet audience needs. Effective pretesting must therefore be tailored to participants with limited-reading skills.
Here are a few suggestions for pretesting with a low-literacy audience:
- Give only one product to each individual or group to focus respondents’ attention on your materials—even if you are testing a series of publications.
- Distance yourself from the product and assure participants that you want their honest assessment. Otherwise, audiences may steer away from critical discussion. Some respondents may not be comfortable explaining negative responses or may not understand the material or pretesting process.
- Make sure participants understand that it is the materials that are being tested, not them. Explain this clearly when you introduce the materials and the process. Respondents need reassurance.
- Pay attention to audience interests. If necessary, change the pace to keep reader attention levels up. You may need to split up a group to allow for more one-to-one interaction and switch the focus of discussion.
- Select people to recruit and interview pretest participants who are culturally sensitive and who have good social skills. Unless potential participants feel at ease with the interviewing staff, they may not agree to take part.
Avoiding Waste: Using Results to Revise
Pretest results raise issues and point out problems, but the next steps are up to you. Simple or comprehensive revisions can address any number of problems that pretesting uncovers. It may not be necessary to scrap the product and begin all over again. If one or only a few respondents raise a concern, revisions may not be necessary—depending on the nature of comments. Any remark that shows lack of understanding of a key concept should receive careful attention. If one person fails to grasp a point, others may have the same problem but may not air it. Idiosyncratic comments about a product's appeal or personal relevance are less of a concern. Percentages are more complicated. If you test the material with 10 people, each response needs to be considered seriously. With 50 participants, a percentage can be relied on more comfortably.
You should start all over when the majority of responses indicate fundamental problems and whenever results show that:
- Readers cannot identify key behavioral actions the material is designed to convey;
- Readers are completely lost;
- Readers find the format unappealing or off target (e.g., brochure vs. web content);
- Readers believe the medium interferes with the message (e.g., a poster or infographic is too complicated to understand);
- Readers say the format is altogether unappealing or off target; or
- Readers report that the product is culturally inappropriate and lacks relevance.
The results of pretesting, however, are not a blueprint for revisions.
"At first, we had to educate my organization about the role pretesting plays in developing low-literacy products," one writer says. "Now that everyone sees its value, we make time for pretesting in every low-literacy product we develop."
Q: Should respondents' suggestions for revising a product be followed?
A: Pretest respondents are not experts in design. They have knowledge and expertise in what they can understand and accept about a product. Professionals must devise effective ways to address respondent concerns. Professional writers often prefer to attend pretest group sessions or read questionnaires themselves rather than relying on the results report alone. Sometimes they contribute to questionnaires as well to draw attention to any wording or format choices they want the audience to validate.
Q: My timeframe is very tight. How can I fit pretesting into the production schedule?A: Pretesting need not be an elaborate, time-consuming research project and depends on:
- How quickly you work,
- How elaborate your internal review processes are,
- How large your test group sampling is, and
- The schedules of any sites or organizations that are involved.
You should include time for pretesting during initial product development. Arranging with a group or site to develop a sampling of test audience respondents can begin during concept development. You may want to begin to draft the questionnaire while your product is going through internal review. Any needed modifications can be made later.
Q: How can I get access to people from the target audience for pretesting?
A: Many agencies coordinate efforts with literacy organizations, directors of Adult Basic Education and English as a Second Language programs, local nonprofits, community groups, and medical facilities. You also can cooperate with groups that represent your particular target audience. If your budget allows, offering incentives, such as small cash payments or reduced waiting time for appointments, can help motivate individual participation and garner support. Overworked and underfunded literacy programs also are more able to provide timely assistance if your agency can compensate them for their participation.
Q: Professional intermediaries distribute our pamphlets to their clients. Should they be part of pretesting?
A: Professional intermediaries and advisory groups can be an important source of insight for your product’s success. You should weigh the responses of test audience members against professional views in cases where opinions vary or conflict. It is important to not rely on professional review alone, no matter how close professionals are to clients. They are not low-literacy readers themselves.
For more information, see Making Health Communication Programs Work: A Planner's Guide (1989).
For More Information
- 2003 National Assessment of Adult Literacy, U.S. Department of Education, National Center for Education Statistics .
- Making Health Communication Programs Work: A Planner's Guide, U.S. Department of Health and Human Services, National Cancer Institute, Bethesda, Maryland, 1989.
- Zimmerman M, Newton N, Frumin L, Wittet, S. Developing Health and Family Planning Print Materials for Low-Literate Audiences: A Guide, Program for Appropriate Technology in Health, Washington, D.C., 1989.
- Leonard and Cecilia Doak. Teaching Patients with Low Literacy Skills, J.B. Lippincott, Philadelphia, Pa., 1985.
- Vernon, John A., PhD, et al., Health Literacy: Implications for National Health Policy (Report), October 2007.
- King Jr., Talmadge E., M.D., et al., Medical Management of Vulnerable & Underserved Patients: Policy, Principles, and Practice, McGraw-Hill Medical Publishing, New York, New York, 2006.