Lyssn Clinical Quality Metrics

Metrics range from general counseling and patient-centered communication to fidelity to evidence-based counseling, including Motivational Interviewing and Cognitive Behavioral Therapy. This document describes Lyssn’s metrics and when applicable provides guidance on clinical benchmarks. 

For each metric, the metric name is in bold, the (scale range) in parentheses, and the dataset [variable name] in brackets. These variable names are how each metric is defined in the data download feature.  When applicable, clinical benchmarks are in italics. 

Example: Empathy (rated on a scale of 1-5) and variable name is [sessionempathy]

GENERAL SESSION DATA

Talk Time [percenttherapisttalktime] - the percentage of counselor versus client talk time during the session

MOTIVATIONAL INTERVIEWING 

Metrics below were derived or adapted from the Motivational Interviewing Skills Code (MISC; versions 2.1 and 2.5) and the Motivational Interviewing Treatment Integrity (MITI; version 4.2.1)

MI Proficiency Score (range 0-12) [miproficiencyscore] - a total score summarizing MI proficiency across 6 MI fidelity metrics: empathy, collaboration, % reflections, % questions, % MI adherent, and ratio of reflections to questions (all individually defined below).  Each of the 6 metrics is scored 0, 1, or 2 on whether the score met criteria for basic or advanced competency, using cutoffs from the MITI v3.1.1 manual (p. 27).  MI Proficiency is a single value summarizing the quality of MI delivered, where higher scores indicate better MI performance.

Empathy (range 1-5) [sessionempathy] - how well the provider demonstrates understanding of the client’s thoughts, feelings, and behaviors.

• High empathy is evident when the counselor shows an active interest in understanding what the client is saying.
• Low empathy is characterized by counselor indifference or active dismissal of the client’s perspective and experiences.

Empathy High Points [highpoints_empathy] - Lyssn also provides localized instances of empathic statements in the transcript view. These empathy “high points”  represent exemplary instances of the therapist demonstrating understanding to their clients. 

Collaboration (range 1-5) [mispirit] - how skillfully the counselor fosters power-sharing that allows client’s ideas to influence the direction of the session.

• High collaboration is evident when the counselor works cooperatively with the client toward the goals of the interview. The counselor does not rely on dominance, expertise or authority to achieve progress.
• Low collaboration is characterized by a lack of counselor/client mutual understanding during the session. The counselor actively assumes the expert role for the majority of the interaction with the client.

Collaboration High Points [highpoints_collaboration] - Lyssn also provides localized instances of collaboration statements in the transcript view. These collaborative “high points”  represent exemplary instances of the therapist demonstrating collaborative behaviors.

Evocation (range 1-5) [sessionevocation] - how effectively the provider elicits the client's own motivations for change rather than imposing their own

• High evocation is evident when the counselor draws out the client's perspectives and focuses on eliciting the client's own desire and reasons for changing, helping clients articulate their motivation for change.
• Low evocation is characterized by the counselor showing little interest in exploring the client's reasons for change and instead providing information, education, or logical arguments for change.

Evocation High Points [highpoints_evocation] - Lyssn also provides localized instances of evocation statements in the transcript view. These evocative “high points”  represent exemplary instances of the therapist effectively evoking client change language. 

Client Self-Exploration (range 1-5) [clientselfexploration] - the depth to which the client explores personally relevant material during their high point in the session

• High self-exploration occurs when the client actively engages in exploring values, problems, feelings, and perceptions, potentially experiencing shifts in self-understanding or discovering new personal meanings.
• Low self-exploration is characterized by the client avoiding discussion of personally relevant material, responding only minimally to counselor prompts, or discussing personal matters in a mechanical or emotionally detached manner.

Client Self-Exploration High Points [highpoints_clientselfexploration] - Lyssn also provides localized instances of client self-exploration statements in the transcript view. These self-exploration “high points”  represent potential high clinical impact statements by the client.  

Clinical Benchmark: For Empathy and Collaboration a rating of  “3.5” is considered Basic and a “4” is considered Advanced. 

MI Counselor Behaviors

Counselor behaviors are listed below. Certain behaviors are indicative of high-quality MI whereas others should be avoided. See below for these MI adherent and MI non-adherent counselor behaviors. 

Advise [advise] - the counselor gives advice, makes a suggestion, or points out a possible problem or warning with client’s intentions

Affirm [affirm] - the counselor says something positive, compliments the client, or reinforces clients strengths or efforts

 

Confront [nconfront] - the counselor directly disagrees, argues, corrects, shames, blames, seeks to persuade, criticizes, judges, labels, moralizes, ridicules, or questions the client's honesty

Emphasize Control [emphasize_control] - the counselor acknowledges, honors, or emphasizes the client's freedom of choice, autonomy, personal responsibility

Facilitate [facilitate] - simple utterances that function as keep-going acknowledgments 

Giving Information [giving information] - the counselor explains something, educates, provides feedback, or discloses personal information

No Code [no_code] - not codeable behaviors such as poor audio or laughter

Question Open [question_open] - a question that allows a wide range of possible responses and can invite client’s perspective or self-exploration

Question Closed [question_closed] - a question that implies a short answer (yes, no, specific factor or number)

Reflection Complex [reflection complex] - the counselor demonstrates significant understanding of what the client stated, typically above and beyond what the client stated

Reflection Simple [reflection_simple] - the counselor adds little or no meaning or emphasis on what the client has said, merely convey basic understanding

Structure [structure] - the counselor describes what’s going to happen directly to the client in this or subsequent sessions

Note: There are no specific ranges or expected counts  for each of the above behaviors but rather a high quality MI session would have higher counts of MI adherent behaviors versus MI nonadherent behaviors (defined below). 

% MI-adherent and nonadherent behaviors are calculated taking the total number of each behavior divided by all behaviors in the session. 

MI-adherent behaviors [percentmiadherent] 

affirm, emphasize control, open questions, reflections (simple & complex)

Clinical benchmark: 90% for Basic and 100% for Advanced 

MI-nonadherent behaviors [perminonadherent]

advice, confront

Giving information and closed questions are not included in either MI-adherent or MI-nonadherent. In a high-quality MI session we would expect fewer giving information statements and fewer closed questions. Yet, on their own these do not constitute MI-nonadherent behaviors.

MI summary metrics are common ways of assessing MI adherence by summarizing question and reflection behaviors. For all summary metrics, higher scores equal better performance. 

% open questions [percentopenquestions]- the percentage of open questions versus closed questions

Clinical benchmark: 50% for Basic and 70% for Advanced 

% complex reflections [percentcomplexreflections] - the percentage of complex reflections versus simple reflections

Clinical benchmark: 40% for Basic and 50% for Advanced 

.

Reflection to question ratio [reflectiontoquestionratio] - the number of reflections divided by the number of questions

Clinical benchmark: 1.0 for Basic and 2.0 for Advanced

COGNITIVE BEHAVIORAL THERAPY (CBT) 

Lyssn’s CBT fidelity ratings come from the Cognitive Therapy Rating Scale (CTRS). CTRS ratings are rated once for a session on a 0-6 scale with higher scores indicating greater fidelity. 

CBT Total Score [cbt-competency] - Summed total score of the below behaviors

Clinical benchmark: a total score of 40+ demonstrates competence.

Agenda [cbt-agenda] - How well does the counselor structure and set the session agenda with the client

Feedback [cbt-feedback] - How well does the counselor elicit feedback from to assess client understanding

Understanding [cbt-understanding] - How well does the counselor use listening skills and empathy to understand the client’s viewpoint

Interpersonal Effectiveness [cbt-interpersonal] - How well does the counselor show warmth, concern, confidence, genuineness, and professionalism

Collaboration [cbt-collaboration] - How well does the counselor engage with and encourage the client to take an active role in the session

Pacing and Efficient Use of Time [cbt-pacingandtime]- How efficient is the counselor in timing and staying focused on treatment relevant topics

Guided Discovery [cbt-guideddiscovery] - How well does the counselor assist client in exploring problems and developing new perspectives from the client’s point of view

Focusing on Key Cognitions or Behaviors [cbt-keycognitionbehavior] - How well does the counselor focus on key thoughts, assumptions, and behaviors that are relevant to the client’s target problem

Strategy for Change [cbt-strategyforchange] - How appropriate and coherent strategies are incorporated CBT techniques

Application of Cognitive-Behavioral Techniques [cbt-applicationoftechnique] - How skillful is the application of specific CBT techniques

Homework [cbt-homework] - How skillfully does the counselor review previous homework and discuss upcoming homework assignment 

Note: A score of “4” is a benchmark for basic clinical proficiency in each CBT skill. 

TOPICS OF THERAPY 

Lyssn Topics identifies topical session themes and gives users the ability to identify parts of the transcript where specific topics were discussed. For example, clients can quickly navigate to a section of the transcript where the therapist and client discussed the client's mood or cultural identities. These topics can be used to guide note writing or quickly find topical areas of discussion for review. 

Psychosocial Topics

Culture, Race, & Ethnicity [topic.cltr] - racial, ethnic, indigenous heritage, cultural concerns, sexual orientation, gender or gender identity

Education [topic.edu] - academic performance, school, learning disorders or difficulties

Hobbies & Activities [topic.fun] - hobbies, sports, leisure activities

Primary Relationship or Family [topic.prtr] - family or a primary relationship (partner)

Parenting & Parent-Child [topic.chld] - parenting, children, issues related to custody, and cps reporting

Religion [topic.reli] - religion and spirituality 

Identity & Self-Development [topic.self] - identity development, self-esteem, confidence, values

Sex [topic.sex] - sex, sex acts, masturbation, intimacy, sexual concerns, porn

Social [topic.soc] - non-romantic primary relationship (e.g., friend), social isolation, shyness

Violence [topic.hate] - harassment, threats, bullying, violence, assault and other violent crimes

Work [topic.work] - job concerns or career discussions

Diagnosis or Presenting Concern Topics

Adjustment [topic.ajst] - adjustment concerns, grief and loss

Anxiety [topic.anx] - anxiety symptoms, stress, panic, phobias, perfectionism , obsessions or compulsions

Health & Medical [topic.hlth] - physical health, sexual health, disability status, medication management

Mood or Emotional State [topic.mood] - mood, emotional state or feelings, symptoms of depression, mood instability, bipolar symptoms, emotion dysregulation, and anger management

Suicide [topic.si] - self-injurious thoughts or behaviors, self-harm, suicidality, suicide attempt

Trauma [topic.trma] discussions of trauma, physical or sexual abuse, domestic violence, symptoms of PTSD, harassment, bullying, violence

Alcohol [topic.etoh] - alcohol use or dependence 

Drug - Other [topic.drug] - unknown drug, polysubstance, medication misuse

Drug - Marijuana [topic.mj] - thc and cannabinoid derivatives

Drug - Stimulants [topic.stim] - cocaine / crack use, methamphetamine, amphetamines

Drug - Opiates [topic.opi] - opiates and opioids

Drug - Tobacco [topic.cigg] - tobacco, vaping, nicotine

Other Psychological Pathology [topic.psyc] - autism spectrum, eating disorder/body dysmorphia, psychotic thoughts, violent thoughts or behaviors towards others, cognitive and neurocognitive disorders 

Therapy or Session Specific Topics

Assessment [topic.asmt] - structured psychological assessments (such as the Outcome Questionnaire 45, BDI, PHQ-9, GAD-7, CCAPS, etc.),

Case Management [topic.case] - topics related to coordination of care and external services outside of one-on-one therapy (housing, legal, child custody) 

Scheduling [topic.cal] - scheduling next appointment 

Therapy Meta [topic.meta] - treatment goals, orientation, interventions, psychoeducation, homework assignments

LYSSN CRISIS 

Lyssn Crisis seeks to provide feedback on counselor suicide risk assessment. For client’s that endorse suicidal concerns, Lyssn Crisis identifies the core components of a thorough risk assessment. Similar to the empirical measures on which Lyssn Crisis is derived (e.g., Lake et al., 2019), these codes focus on suicide concerns and less on self-harm. Lyssn Crisis can be used to identify (1) counselor risk assessment behaviors and (2) client explicit endorsement of suicidal concerns or client statements that might require follow up (e.g., passive SI). 

Counselor Codes

Current Suicidal Ideation [risk_csi] - Assessing if the client is currently feeling suicidal

Past Suicidal Ideation [risk_si} - Assessing if the client has a history of suicidal ideation, experience suicidal thoughts in the past 

Past Attempt or Behavior [risk_pab] - Assessing if the client had attempted suicide or engaged in suicidal behaviors in the past 

Current Plan [risk_cpm] - Assessing if the client has developed a plan or thought out ways they would die by suicide

Means [risk_mn] - Assessing if the client has access to lethal means 

Intent [risk_int] - Assessing if the client has made preparations or holds a desire to die by suicide

Attempt in Progress [risk_atm] - Assessing if the client is actively engaging in suicidal behaviors either in the present moment or earlier the same day

Protective Factors [risk_prf] - Assessing if the client has reasons for living, protective factors 

Client Codes

Client General Risk Statement [risk_risk_gen] - Client statements that endorse one of three areas: 

(1) Passive suicidal statements; 

(2) Self-harm statements; or 

(3) Hopelessness

Intended to flag potential need for counselor follow-up regarding safety and suicidal thoughts 

Client Suicidal Risk Statement [risk_risk_si]  - Client endorses suicidal thoughts or actions.