What is
required for someone to become competent in using the ADOS?
To obtain essential competence, ADOS users
should:
Have prior education, training and
experience that includes extensive exposure to Autism and PDD.
Take a WPS in-person clinical training
workshop or use the WPS Training Video/DVD and accompanying materials.
Upon completion of the in-person workshop you will receive a certificate
of completion that is equivalent to 12 contact hours of
professional continuing education credit. Upon completion of the
Training Video/DVD Continuing Education (CE) Questionnaire and
Evaluation Form (W-365CE) you will receive a certificate of completion
that is equivalent to 18 contact hours of professional continuing
education credit.
Practice using the ADOS on cases that are
not part of formal evaluations and bring themselves to complete
familiarity with the assessment activities and complete confidence that
they can apply the coding categories accurately. Examiners with
considerable prior experience in making formal observations and
administering individual assessments may reach this level in as few as 8
practice sessions (2 per module). Others may take considerably more
practice.
If a user is to be involved in formal
research directed at producing articles for publication in peer-review
journals, he or she must additionally take a research training workshop
followed by exercises that establish item coding accuracy to a specific
criterion.
After I take the ADOS training, can my results be used to diagnose
autism?
The ADOS is used in several kinds of
practice. In each, the test user's credentials need to match the purpose
of the use. For example, a medical diagnosis requires a license, such as
that of a medical doctor or clinical psychologist; a special education
qualification requires a locally recognized license or credential, such
as a school psychologist; and so on. The ADOS can also be used for
purposes other than making a diagnosis. For example, occupational
therapists and speech and hearing professionals often use the ADOS for
treatment planning and program evaluation. Some speech and hearing
specialists also use the ADOS for early childhood screening and program
qualification, as mandated by the state where they practice. It may be
helpful to check with your local mental health licensing agency with
additional questions along these lines.
What is a clinical training workshop?
A clinical training workshop takes two
days and uses a lecture format to introduce people to the basic
principles of administering and scoring all four ADOS modules. The
Training Video/DVD Continuing Education (CE) Questionnaire and
Evaluation Form (W-365CE) also provides the equivalent content. Either of
these provides one of the essential steps towards competence in using
the ADOS in everyday clinical practice. Working with WPS, Dr. Lord and
several of her most experienced colleagues provide these Clinical
Workshops in various parts of the country throughout the year. For more
information on ADOS workshops
click here
What is a research training workshop?
A research training workshop is only
available to those who have completed the in person clinical training
workshop. Research training focuses on bringing attendees' scoring
skills to a high level of inter-rater agreement with the scoring
practices of skilled examiners. The heightened level of accuracy is
needed to ensure that item-level coding is comparable at different labs
in different parts of the world where research is being conducted. These
courses are usually conducted at the university laboratory of one of the
test authors. WPS is not involved in these courses. For information on
research training:
United States click here
www.umaccweb.com.
United Kingdom contact:
a.s.le-couteur@ncl.ac.uk or
anthony.bailey@psychiatry.oxford.ac.uk
In other parts of the world contact either group above to make
arrangements.
Who may attend a clinical training
workshop?
Typically, attendees are individuals who
will be qualified to use the ADOS--that is individuals who, prior to the
workshop, have education, training and experience in using individually
administered test batteries and who have a background and experience in
the treatment of autism. Some examples of clinicians who have a
professional mandate to treat autism and who make use of the ADOS in
their daily work are clinical and school psychologists, psychiatrists,
occupational therapists, and speech and hearing professionals. In some
cases, schools and clinics will send additional staff to a workshop with
a view that a better understanding of the instrument will help their
teachers, counselors, special educators, or other program staff make
better use of reports based on ADOS results. Note that in all cases,
however, simple attendance at the workshop is not sufficient to ensure
competent use of the ADOS.
Once I have obtained clinical training, can I train someone else?
No. The WPS materials are designed to
provide training to individual users only. Completion does not provide
the additional materials and experiences required for the skills needed
to train others. The Training Video/DVD Continuing Education (CE)
Questionnaire and Evaluation Form (W-365CE) can be used to train
multiple clinical users at a single site.
Further training through the specialized research training course
offered by the research community is required before others may be
trained. (See next question).
Once I have research training, can I
train someone else?
Only in limited circumstances. A
researcher who has taken the ADOS clinical workshop and the ADOS
research workshop and achieved reliability with research workshop team
can, back in their home lab, train people that they work with on a daily
basis. A letter with these responsibilities fully specified is given to
each research trainee at the time the training workshop is completed.
Note that even the original trainee at a research site is not seen as
qualified to train people away from their own location until they
undergo further practice and supervision in actual training. For
example, the individuals who conduct the WPS in-person clinical
workshops have worked under supervision as training workshop leaders in
Dr. Lord's research labs.
What is the youngest age at which the ADOS may be used?
For valid interpretation of the
Algorithm scores, the child should have a non-verbal developmental age
of 12 months or higher and must be able to walk independently. The
latter requirement does tie the decision a bit more to chronological
markers than would sole attention to developmental age in the most rigid
sense. Current research indicates that the ADOS algorithm is not
sufficiently specific for kids with nonverbal age equivalents under 12
months (and should be interpreted with more than usual caution through
18 months age equivalents).
We stress, however, based on data so far reviewed, that the individual
items are still reliable across raters and time even for developmentally
younger children. The results also appear to be valid in terms of
discriminating autism from non-autism when obtained scores are not near
the cut-off scores. It is just that when scores occur near the cut-offs
that the algorithm recommendations premised on results from older
children may not apply as accurately. So it is not that people cannot
use the information they get when they use the ADOS in the first 12
months of development, but that they have to be wary of interpreting the
algorithm in the same way as they would with older children.
Any professionally developed test with current norms can be used to
obtain estimates of non-verbal developmental age equivalencies. Some
examples might be the
Mullen Scales of Early Learning, the Bayley Scales
of Infant Development , the Vineland Adaptive Behavior Scales or the
Kent Inventory of Developmental Skills (KIDS). We often use more than
one such test in cases where making an appropriate judgment appears
difficult. As with any testing, the decision should be a professional
judgment based on multiple sources of evidence, never a rule based on
rigid application of a single test score.
All of these concerns apply only to Module 1, of course. After Module 1,
the determinations are based on expressive language level that is
determined during the administration of the ADOS. We recommended that
people could use the expressive language level from the Vineland as an
estimate to determine where to start, but in the end, how well the child
talks during the actual session is the final basis for the decision.
Is coding and scoring the ADOS
subjective?
While not properly categorized as an "objective test" in a classical test sense, the ADOS is anything but
"subjective."
The ADOS is, as its name specifies, a schedule of "observations." This
stands in contrast to a norm-referenced objective test. For an
objective test, questions are designed to elicit specific responses
that are easily and immediately categorized. Behaviors observed in
social interaction are not discrete events and cannot be categorized
so easily. There is a long history of scientific observation of
behavior that has developed quite separately from the history of
classical test development. The observation categories used in the ADOS have been developed over several decades and have been tested and
proven to produce accurate inter-rater agreement on the classes of
behavior that are fundamental to the diagnosis of PDD and ASDs.
For all assessments, behavioral observations or classical tests,
discrete individual test items and coding categories have far lower
rates of reliability than do the total scores that are used to inform
final interpretation and diagnostic decision making. For example,
individual items on personality and objective behavior checklists
often have internal consistency estimates in the range .30 to .40. It
is only when these are combined into scales that reliabilities become
adequate for score interpretation. A parallel circumstance exists for
observation schedules like the ADOS. The final results for ADOS
algorithm scores, as reported in the manual and scientific literature,
demonstrate a high degree of reliability and accuracy. However,
compared to the user of an objective test, a heavier burden does fall
on the ADOS user who must work hard to insure that the codings of
individual behaviors are as accurate as possible
What does obtaining reliability mean?
When researchers refer to "obtaining
reliability," they mean that a new researcher has reached a proven level
of inter-rater reliability when his or her item-level codings are
compared to those of other more experienced researchers.
What happens when a child just misses the autism criterion on the ADOS?
Sometimes a child with very high functioning
autism can miss autism criteria on the ADOS, just because it is a short
observation. This is more likely to happen in Module 1, with children
who have had a lot of good specialized education (e.g., "practiced"
going to a birthday party). On the other hand, when this happens, the
examiner is usually quite aware of it since these children will almost
always have some high codings and scores, just not enough of them to
reach the criterion for autism. So it is not that the examiner did not
see any autism, but that the particular thresholds for the codes were
not met.
The situation can also occur when a child's developmental skills fall "between" two modules. A user may decide to give the lower one, even
though the child has enough language to do the next one up, out of
concern that the social questions might be difficult. This can cause
kids to miss threshold--for example, reach the criteria for ASD but not
autism, or meet criteria in one domain, but not another. Even so, such a
child wouldn't "not look autistic."
In all such cases, an examiner must use clinical judgment. Apart from
the ADOS scores, did the child show any signs of autism during the
session? How does this fit with the specific abnormalities another
mental health team member may be reporting from the
Autism Diagnostic Interview-Revised (ADI-R) for example
(which is retrospective and relies completely on the parents' report).
In cases where parents', teachers' or other clinicians' reports seem to
disagree with observations made during an ADOS session the examiner need
to find out: are people are actually seeing the same things but
interpreting them differently; is it that the child's behavior changed
radically; or are other parties reporting things that really were not
present during your session?
This is also the place to consider a central caveat for all testing: no
clinical decision should rest on the results of a single test score. A
diagnosis, whether it is medical or school program based, is the
informed judgment of a skilled and qualified professional, based on all
the accumulated evidence from multiple sources.
Are there recommendations on using the ADOS with children who are from
different cultural backgrounds, are bilingual, etc?
The ADOS has been used in quite a few
Western cultures and in Korea and India and worked quite well, at least
based on informal feedback. Currently, however, we are not aware of
results from formal studies completed outside the UK and US.
In everyday clinical use, the critical factor would be to have someone
from the child's home culture available able to consider the ratings in
the context of what would be considered normal in that culture--for
example, some gestures, eye contact, or imaginative play may have
different meanings in different cultures. In addition users must always
consider the general effect of the testing experience (e.g., whether
conducted at home or in a clinic) and who the examiner is (similar to or
different from the child's ethnicity) on the behavior. If a clinic
serves many individuals from a specific cultural background it may be
helpful for the ADOS users there to administer the ADOS with a control
group of non-PDD kids from the same background. This may be particularly
useful in situations involving non-Western cultures where such
differences may be more pronounced. The control group could probably be
pretty small since the variation in ADOS scores for typical kids is
small and the differences due to culture should become obvious fairly
readily.
Dr. Lord notes: I have given Module 1's in many parts of the world on
all continents and even when I did not understand the language, with the
parent transferring simple things, I have been able to score reliably
with other trained raters. However, my ratings of Module 3 and 4, even
when I do understand bits and pieces of the language (Spanish and bits
of French and German), have been far off -- I really could not pick up
odd intonation at all, and had quite different impressions of gestures
and facial expressions than native speakers. In fact, even comparing UK
and US samples, in the UK, when a high functioning child had odd
intonations the British coders would frequently say that the child had
an American accent, which I would not have ever said. So there are some
subtle cultural differences in the higher modules that probably do
require people from within the culture or possibly very experienced in
the culture.
If a child is on medication, is it recommended that the ADOS be
administered on-meds or off-meds?
In general, we have suggested children not
be taken off medication. The kinds of behaviors targeted by the ADOS
should still be present even if a child is medicated. It may depend on
what the examiner is using the ADOS for, however, and clinical judgment
should be used.
The ADOS seems to penalize some highly verbal children for not pointing.
Are there suggestions for ways in which to elicit pointing?
We make a point of standing up during the
balloon and bubbles tasks and placing the mechanical bunny or car as far
away as possible, so that it makes sense for the child to point. Another
strategy is to play dumb a bit when a child says "look" and wait a few
seconds before responding, but I do not think we do this systematically.
We have not found it to be a consistent problem with typically
developing children in Module 1 or 2, but it may become more of an issue
with older kids with autism or developmental delay. Even so, it probably
would make sense for a user to present even more deliberate presses if
this situation arises.
Are there tapes available to observe the ADOS being given to low
functioning kids who are not autistic?
At present there are no workshop or
commercially available tapes of ADOS sessions conducted with normally
developing children. WPS is working with the authors to see if one or
more tapes of this kind can be developed. Depending on the source of
such tapes, they may be used in the WPS Clinical Workshops or they may
become included with the WPS Video Training materials. However, there is
no present timetable for when such tapes, of either type, may become
available.
Are complex motor tics (e.g., licking finger than touching his ear and
alternating between right and left) coded as complex mannerisms?
In early research on the ADOS there used to
be a separate score for tics, but they were sufficiently uncommon that
our researchers could not get good coding reliability. For this reason
they are not are not formally coded or scored, although we would
encourage people to note them on the protocol.
What is known about the ADOS and differential diagnosis of other
disorders such as ADHD or Bipolar?
The ADOS Manual lists the different
diagnoses of the children in the control groups in the description of
subjects for each Module. These were heterogeneous groups that differed
in diagnosis across age and module (so that the nonspectrum children for
Module 1 were mostly children with non-autism developmental delays
and/or language impairments, whereas for Module 3 children were not
mentally retarded, but more various behavior disorders, and so on).
There were children with ADHD in the groups; bipolar was not a common
diagnosis at that time at University of Chicago, so it is less likely
any child had that diagnosis, but there were children who would get that
diagnosis now. In all events however, the original research did not
include a sufficient number of any other diagnosis besides autism/ASD to
really address the issues raised in this question. These questions do
need to be investigated, but it is likely to take quite some work and
time before useful conclusions can be drawn.
Has the ADOS been used as an outcome
measure?
The ADOS has been used for an outcome
measure in a couple of different situations although, in some ways, it
may not suit all outcome evaluations. The ADOS is intended to pick up
autistic characteristics across a lot of situations and so it may not be
terribly sensitive to subtle changes. For example, other tests directed
more narrowly at behavior in specific areas (e.g., language syntax
scores from the SICD, high activity scores from the Conners) may be more
sensitive to change than are behaviors associated with actual diagnostic
criteria as seen on the ADOS.
Comparisons over time have been made in some clinical trials. For
example, the ADOS did document small improvements in all groups in our
secretin study, including placebo. And in our clinic we routinely
compare results when we see a child on follow up. You do need to be very
careful to make sure that any given comparison is based on identical
materials: individual items can be compared across different modules in
cases where the items are identical in content; scores based on multiple
items, however, can only be compared within modules.
Also, in our longitudinal study, the ADOS was pretty highly correlated
with more general measures of social adaptation such as the Vineland and
even children's functioning in a less restricted environment. The data
on changes in test-retest scores are in the manual, so if the assessment
is repeated it in a short period of time results can be compared with
that report (as social scores did get a little more normal and
restricted behaviors got a little worse with repeat testing).
How often can the ADOS be administered?
Recent research seems to suggest that there are not significant practice
effects on the ADOS. Although children who are given the ADOS repeatedly
(say, every month or so) do become more familiar with particular tasks
(for example, they may get better at following a birthday party
routine), their codes are generally not affected and the diagnosis as a
whole does not seem to be impacted. Because the scoring of the ADOS
emphasizes the child's spontaneous initiations and responses to the
examiner's behavior, as opposed to scoring of the tasks themselves, it
makes sense that becoming more familiar with a task should not change
the child's codes. Similar observations have been made when the ADOS is
used repeatedly with adults with autism spectrum disorders; they may
become more familiar with the tasks, but they are consistently
classified on the algorithm as having an autism spectrum disorder.
There are test-retest data of the ADOS in the manual that are relevant
to repeated administration. In general, minor changes in scored
responses have been noted, with repetitive behaviors, on the average,
scored as slightly more abnormal on repeated administration within a few
months, and social behaviors improving slightly. Even though there are
minor scoring changes, as discussed above the child's codes are
generally not affected and the diagnosis as a whole does not seem to be
impacted. In addition, research developing the ADOS for use with
toddlers has used monthly repeated administrations and has not found
significant practice effects. The ADOS has also been used in several
drug treatment trials where it has been administered repeatedly over
short intervals (two to three times within a few months), with similar
results.
While ADOS scores don't change much on repeated administration, ideally
it is better if children and adults do not learn the tasks. There should
always be good clinical reasons for repeating the ADOS, especially if it
is administered more than twice within a brief interval. If you know
that you are going to repeat the ADOS during a short period of time, you
may want to decide in advance to use different toys and materials from
the kit each time when possible (e.g., use one book one time and another
the next).
Can the ADOS be used with children with
visual and hearing impairments, such as children who are blind or deaf?
A standard ADOS administration and scoring is not recommended with
children who are blind or deaf. However, ADOS materials and codes could
be used as part of an informal observation with these children, for
example, using verbal parts in a structured observation. The algorithm
should not be used, because many of the scores would not be appropriate.
In the case of a child with visual and hearing impairments, professional
judgment would need to be used to determine the extent of the
impairments and their likely impact on ADOS scores.
Are ADOS classifications accurate when
there are comorbid disorders?
The ADOS is quite good at identifying individuals with autism and autism
spectrum disorders (ASDs), but simply meeting cutoffs on the ADOS does
not necessarily mean that a person has autism or an ASD. Overall ADOS
totals and scores on individual items may be elevated for various
reasons. Individuals with other disorders, such as mental retardation
and/or severe behavior problems, may have elevated scores on the ADOS
due to comorbid conditions that affect social-communication behaviors
that we see in ASD. To provide an accurate diagnosis, your evaluation
must include more than just the ADOS. A diagnostic evaluation should
include several components, including a thorough developmental and
medical history, a current developmental or cognitive profile,
information from parents and/or others who know the child well, and an
assessment of adaptive functioning.
Should ADOS numerical results and
associated cutoff points from the instrument be included in clinical
evaluation reports?
We recommend that ADOS algorithm scores not be used in evaluation
reports. Only the final, categorical ADOS classification has been
validated. Diagnostic interpretation should focus on that classification
and information from other relevant sources. The numerical scores help
in reaching a classification but have no separately established
psychometric properties. Providing these scores or cutoff points in a
report can lead to misunderstanding and leave you open to professional
criticism. For these reasons we also recommend that the ADOS algorithm
sheet not be included in reports. Clinicians should discuss ADOS results
by providing a detailed description of the client's social-communicative
behavior during the ADOS, along with the resulting ADOS classification.
Can an extra observer be used during
administration of the ADOS? Should the observer interact with the child?
The ADOS is intended to be administered by
one person. While some examiners may have another professional present
to watch for subtle behavior cues, it is important that the examinee
never be confused about who should be attended to. Anyone present beyond
the examiner should serve solely as an observer.
Related to this, it is important that an examiner not become dependent
on having another observer present. The ADOS has its primary focus on
interactive social behavior between the examiner and examinee. While an
observer may catch an occasional additional behavior, the examiner is in
the position to be, and must be, the primary observer and judge. A
competent examiner must be proficient enough to manage the materials and
flow of the session while also collecting a good record of what occurs.
Can the ADOS be given in a home or should
it be given in a clinical setting?
It is more difficult to administer the ADOS
in a home because of the distractions there and the need to have the
testing space in the control, to some degree, of the examiner. It is
also important for the child to be as comfortable as possible. Often
children with autism are more comfortable in a structured environment
than an unstructured one (especially with a stranger in the midst of the
familiar environment). If the ADOS will be administered in a home, then
the examiner should, before he or she arrives, discuss the kind of space
needed (e.g., somewhere the examiner can blow bubbles and use Play-Doh;
somewhere without siblings, noisy appliances, or a television; somewhere
the child will sit at a table without necessarily expecting to eat). If
these needs can be met and the examiner feels comfortable, then the ADOS
can be administered in a home setting.
May I translate the ADOS items for my research in a language other than
English?
May I create a "personal translation" to use with my clients whose first
language is not English?
If I am fluent in another language, can I use English forms and
translate as I go?
It is the firm policy of Western
Psychological Services (WPS) and the ADOS authors that all ADOS
administrations must make use of a written translation that has
undergone a formal review and approval process through WPS or one of our
licensed international publishing partners. The authors take this
position owing to their concerns about proper standards of patient care.
Experience indicates translation can significantly affect ADOS scores,
and thus the reliability and validity of the test results. Only properly
reviewed translations can provide results that correspond with those
published in the manual and research literature. WPS echoes those
concerns, not only in exercising our responsibilities and obligations
under international copyright law, but also in protecting the integrity
of our proprietary materials across cultures.
To help you meet professional standards when
conducting assessments with the ADOS, WPS is continually working to
broaden the availability of the ADOS across language groups and
cultures. Authorized translated materials are available from many of our
international publishing partners, and other translations approved for
use in qualified scientific research can be secured through WPS Rights
and Permissions.
There are currently authorized ADOS
translations available or soon to be available for clinical and/or
research use in the following languages:
All other uses in translation must have
the prior, written consent of WPS.
If you have any further questions about available translated editions
please contact Ms. Susan Weinberg in the WPS Rights and Permissions
Department at (800) 648-8857 or
rights@wpspublish.com
How can research use of the ADOS be arranged?
WPS provides a research discount on all of
its test publications for graduate students and for researchers involved
in formal, funded research directed at publication in peer review
journals.
WPS offers scholarly discounts for education and research
involving WPS test materials. You may apply if you are using a WPS
product in a research study or if you are planning to use WPS products
to train students. For more information please see the
Scholarly Discounts for Education and Research
page.
Selected, authorized
translations in other languages may be available directly through WPS
Rights & Permissions for use in funded, qualified research studies by
properly trained, scholarly investigators. Contact
rights@wpspublish.com for details.
