THE TEAM DIAGNOSTIC SURVEY (TDS™): HOW DOES YOUR TEAM MEASURE UP?

“If you can’t measure it, you can’t manage it.”
– Peter Drucker

How do you feel about psychometric assessments in general?  Consider yourself on the continuum below:

In our training workshops in the 6 Conditions of Team Effectiveness and Team Diagnostic Survey (TDS™) we often invite participants to stand along this continuum and debate the pros and cons of using psychometric tools.

In my own career as an Organizational Psychologist and Team Coach over the past 20 years, I have moved from being quite skeptical about psychometric assessments–i.e. closer to the 0 end of the continuum–to seeing their value and place in organizational development.  I’m probably an 8 now on the continuum after years of using many individual and team-level psychometric assessments.

Assessments can help individuals, leaders and teams develop a common language, gain new insights, compare themselves against a benchmark, gain consensus on areas to shift and more.  And since teams are complex systems, having a measurement model to support you is a great addition to the practitioner’s toolkit.

As Drucker notes in the quote at the beginning of this post, the act of measuring things helps us to focus our energy and manage change.  In summary, when coupled with artful facilitation, assessments can be the catalyst  that gets teams moving or accelerating along a positive trajectory.

Considerations for Selecting a Team-Level Psychometric Assessment

The first thing to look for when selecting a team assessment is that it measures something about the team!  Lots of team assessments are just using individual measures and having a conversation about them.  A team is an entity unto itself and not something that is captured by merely talking about or aggregating individual level assessments.

Secondly, when selecting a psychometric assessment it is important to dig below the surface to see whether there is a strong underlying scientific foundation.  Developing a psychometrically sound instrument is no small undertaking. Unfortunately many instruments widely used in practice in today’s organizations are based on questionable science at best and have not subjected themselves to scrutiny in peer-reviewed journal articles.  If we are going to use a psychometric assessment, we should have a sense of its efficacy.

So what basic criteria should we look for when evaluating whether an assessment is worth deploying?  Here are a few considerations:

1. Conceptual Framework:  Does the instrument have a sound underlying conceptual framework for what is being measured and how change occurs (i.e. a theory of change)?  Popular lists of 3, 5 or 10 things that we think influence team effectiveness fall short of this standard.  Is there any research or literature to support the conceptual model or aspects of it?
   
   
2. Reliability:  Do the items in the measure reliably measure the constructs.  The scale should be internally consistent (i.e. if i rate one item high, I should likely rate the next item high as well).  Measures of a scale’s reliability is most often reported in statistics on a scale from 0 to 1.  Anything above a .7 (Cronbach’s Alpha) is considered a reliable scale from an academic perspective. 
   
   Similar measures are usually reported for Test-Retest reliability.  In other words if I apply the measure twice, do I get the same result?  Imagine measuring your weight twice and getting two dramatically different results?  That’s not a reliable scale.
   
   
3. Validity: This one is of most crucial considerations.  Does the measure(s) actually capture what it says it is measuring? Furthermore, does what we think we are measuring actually exist?  This concerns whether the variability in the data set can be explained by the number of factors in the conceptual model or by a more elegant (i.e. more or less factors) configuration.
   
   Finally does the measurement model meaningfully predict the outcome of interest (e.g. team effectiveness)?  You can have a reliable and valid measure but if the amount of variability in the outcome measures that is explained by the measure is low, then the measure isn’t so useful.

The above points may serve as an initial screen when considering an assessment’s credibility.  The gold standard is that the assessment’s development and measures as well as its validation studies on laboratory and/or real world samples are submitted for public scrutiny in a peer-review scientific journal.  Such scrutiny is essential so that everyone and their brother doesn’t just slap together some items and call it a valid assessment.

The Team Diagnostic Survey (TDS™)

The Team Diagnostic Survey (TDS™) developed by Drs. Ruth Wageman, J. Richard Hackman and Erin Lehman is one of a few team assessments available on the market today that meet the standards of a scientifically sound assessment based on the criteria outlined above.  The measurement model, called the 6 Conditions for Team Effectiveness Framework (a.k.a. 6 Team Conditions), begins with clearly specifying the outcome measure of interest (i.e. Team Effectiveness) which is a balance of 3 components:

1. Task Performance – Do the team’s outputs meet or exceed the customer’s expectations?
2. Quality of Group Process – Does the team become more agile and capable as it works together over time?
3. Member Satisfaction – Does the team contribute to the learning, growth and satisfaction of its members?

It’s important for a measurement model to clearly define the outcome(s) of interest.  Does the team assessment you are looking at clearly define team effectiveness?  There are many ways one could define team effectiveness (e.g. task performance, learning agility, innovation, sustainability, engagement, etc…). What is important is whether one is explicitly provided.

The 6 Team Conditions framework is a process model (Inputs → Throughputs → Outputs).  Six structural drivers or design conditions have a large influence on the emergent behaviors and dynamics in teams (Key Task Processes) which ultimately impact the 3 team effectiveness outcomes mentioned above.  These are depicted and described below:

There are 6 structural drivers or design conditions in the model divided into 2 subsets:

1. The 3 Essentials: Real Team, Compelling Purpose, Right People.  These are foundational and our research shows that when these are not present in a team, the whole ecosystem of the team is undermined.
   • Real Team – Real Team is one that is Bounded, meaning that members know who is and is not on the team. It is Stable, meaning that members stay together long enough to learn how to work together. It is Interdependent for a common purpose, meaning the team must interact and share resources to accomplish their purposes.
   • Compelling Purpose – Compelling Purpose means that the purpose is Challenging (a stretch, but not impossible), Clear (they know what it would look like to accomplish it) and Consequential (it has meaningful impact on the lives and work of others).
   • Right People – Right People means that the team has both the right Skills to do the work effectively—including teamwork skills; and Diversity that brings a range of perspectives needed to perform creatively and well.
     
     
2.  The 3 Enablers: Sound Structure, Supportive Context, Team Coaching. These 3 conditions accelerate a team that has the 3 Essentials in good shape.
   • Sound Structure – Sound Structure consists of three subdimensions: Task Design means the team performs work that makes sense to be done by a team and that allows members to use their judgment and experience to complete it; Team Size means the team is neither too big to coordinate nor too small to have the resources they need to do the work; Team Norms means the team has clear ground rules and practices (i.e. ways of working) for how members are expected to work together.
   • Supportive Context – Supportive Context means the organization’s structures and systems promote rather than prevent teamwork and includes 4 components: 1) Rewards and Recognition means the organization pays for and celebrates excellent team (not just individual) performance. 2) Information means teams get the data they need in a form they can use. 3) Education/Consultation means that when teams encounter novel problems or the need to improve their teamwork they can get training and technical consultation. 4) Material Resources means that meeting space, technology resources, people, time—whatever the team needs—is made readily available.
   • Team Coaching – Team Coaching means someone is available to the team—a leader, a member, an external consultant—who pays attention to the team’s process and intervenes at helpful moments. There are two aspects to the provision of team coaching: Availability means often or readily present for the team. Helpfulness means the person doing the coaching knows how and when to intervene.

The 6 Conditions channel their influence through 3 intermediary elements or throughputs called the 3 Key Task Processes.  These are observable indicators in a team and exert the most proximal or immediate influence on the 3 Team Effectiveness outcomes.

1. Effort – Is the team working in ways that builds shared commitment to the work and to the team?
2. Strategy – Is the team inventing approaches that are uniquely suited to the work the team is doing?
3. Knowledge & Skill – Is the team utilizing it’s members’ talents and skills well?

How Do the 6 Conditions & TDS Measure Up?

The Team Diagnostic Survey was originally published in 2005 in the following peer review journal article:

Wageman, R., Hackman, J.R., & Lehman, E. (2005). Team Diagnostic Survey: Development of an Instrument.  The Journal of Applied Behavioral Science. 41, 4, 373-398.

More than 3,000 teams have been assessed to date using the TDSTM and a number of studies have been conducted to apply the framework to real world teams.  Here are three noteworthy examples:

1. In a study of 127 top leadership teams around the world,  Wageman, R., et al.,  (2008) found that the TDS 6 conditions predicted more than 50% of team effectiveness. It was found that only 24% of leadership teams were high performing, but those that were had distinctly better designs on all 6 conditions.  Source: Wageman, R., Nunes, D.A., Burruss, J.A. & Hackman, J.R. (2008). Senior Leadership Teams: What it Takes to Make Them Great.  Harvard Business Review Press.
   
   
2. In a study of 64 intelligence analysis teams, Hackman  (2011) found that the TDS explained 74% of the variance in team effectiveness outcomes. Intelligence teams with better designs on the 6 conditions delivered superb analysis, grew in capability over time and fostered the development of members’ abilities. Source: Hackman, J.R. (2011). Collaborative Intelligence: Using Teams to Solve Hard Problems. Berrett-Koheler Publishers.
   
   
3. A validation study of the TDSTM including a randomized field experiment on the effects of a TDS feedback intervention on team effectiveness in 97 teams (Eisele, 2013) found significant differences between treatment & non- treatment groups.  Source: Eisele, P. (2013). Validation of the Team Diagnostic Survey and a Field Experiment to Examine the Effects of an Intervention to Increase Team Effectiveness. Group Facilitation, 12, 53-70.

In short, the 6 Conditions and TDSTM  is a highly predictive measurement model for understanding the drivers of team effectiveness and that has practical utility in guiding team coaching interventions.

Learn More About the 6 Conditions & TDS™

The TDS™ was made available commercially in 2017 and now is being deployed by more than 700 certified practitioners around the globe.  It has been translated into a number of languages (e.g. English, French, Dutch, Portuguese, Spanish, Persian, Arabic, Chinese, Italian and more).   The assessment generates a 27 page customized report that includes data on the main measurement model as well as on a range of supplemental quantitative and qualitative measures that illuminate the findings of the main model:

The report provides team leaders, teams and team coaches with clear areas for helping a team accelerate its effectiveness.  Here are some additional resources to learn more:

1. Sample Report & Case Study: To review a sample report and a case study go to the Team Diagnostic Survey page.
2. Podcast on the Team Diagnostic Survey: Listen to a podcast episode, 6 Conditions | The Science & Art of Great Teams | Episode #008 | The Team Diagnostic Survey, by Dr. Ruth Wageman and myself (Krister Lowe).
3. Become a Licensed Practitioner or Enterprise Customer: Finally if you are interested in learning more about getting certified to use the Team Diagnostic Survey or to bring the TDSTM into your organization go to Team Practitioner Development or Enterprise Services.

Conclusion

When combined with artful facilitation, a validated team effectiveness assessment can be a powerful catalyst for accelerating a team’s effectiveness.  It can help a team develop a common language, get a baseline for where the team is currently and can point to areas that need shifting in order to keep the team headed in the right direction.

The Team Diagnostic Survey ranks among the highest calibre instruments available today for use by team leaders, teams and team coaches for fostering team effectiveness.  For more information go to 6 Team Condtions.