Performance-Based Evaluation of Lawyer-Client Communication:
The Development of the Standardized Client Assessment
Clark D. Cunningham
W. Lee Burge Chair in Law & Ethics
Georgia State University College of Law
Director, National Institute for Teaching Ethics & Professionalism
Working Paper (as of 10 September 2013)

Abstract

From 2004 -2006 an international and interdisciplinary team undertook an ambitious project to change the way lawyer-client communication skills are taught and assessed.  Medical education has been transformed by a new methodology for assessing competence in patient communication: the use of intensively-trained lay persons who present standardized patient scenarios to medical candidates and then assess the candidates’ performance. The Glasgow Graduate School of Law in Scotland was the site for a series of pilot projects testing whether a similar methodology using standardized clients would be more valid, reliable and cost-effective than the approach, widely used by many law schools, of having client roles played by students with assessment based on law teacher review of the interview videotape. The results strongly indicated that assessment by standardized clients was sufficiently valid and reliable to be used for a high-stakes examination in legal education. This methodology has since been adapted for use in England, the United States, Australia, Hong Kong and Japan.

The Effective Lawyer-Client Communication (ELCC) project was initiated in 1998 and has at various times included participants from Australia, England, India, Israel,  Scotland, South Africa, and the United States and from a wide variety of disciplines. 
The long-term goal of the project is to determine whether international and interdisciplinary collaboration on the issue of lawyer-client communication can actually change basic institutional practices and beliefs in the legal profession. The ELCC project is currently based at the Georgia State University (GSU) College of Law in Atlanta. For more information, see the ELCC web site: http://law.gsu.edu/Communication/ 

The ELCC project made significant progress when it found partners at two innovative law schools – the Glasgow Graduate School of Law (GGSL) in Scotland and the University of New Hampshire (UNH) Law School in the US – that were making  unusually strong connections between the academic enterprise and the world of practice.

GGSL was created by the collaboration of the University of Glasgow and the University of Strathclyde to offer a university-based post-graduate year-long program – then called the Diploma in Legal Practice – that would meet new requirements established in the 1990s by the Law Society of Scotland to provide a transition between the undergraduate law degree and the two-year work-based apprenticeship leading to licensure. GGSL fortuitously hired as its founding academic director Paul Maharg, who held a PhD and a diploma in education as well as a law degree. Influenced in particular by the work of the American philosopher and educationalist John Dewey – who famously created the “Laboratory School” at the University of Chicago to integrate educational research, theory and practice – Maharg developed a curriculum built not around classroom instruction but engagement with complex and realistic legal transactions. Students acted in many various roles as lawyers, and learned from a teaching staff – drawn largely from the top tier of the Glasgow legal profession – that functioned as mentors and guides rather than lecturers.

Due to its close connection to the profession, GGSL recognized that client interviewing was an essential component of effective practice and set as a graduation requirement that all students pass a performance-based assessment of their ability to conduct an initial client meeting.  Because completion of the Diploma was a pre-requisite for licensure, Scotland thus became a jurisdiction where competence in lawyer-client communication was formally assessed as part of the licensing process.  Having decided from a practitioner perspective that a high-stakes assessment of interviewing competence was necessary, GGSL then drew upon its scholarly resources to explore ways to make that assessment valid, reliable and cost-effective.  Through participation at an international conference on innovative approaches to lawyer licensing hosted by GSU in Atlanta, Maharg became aware of the ELCC project, which facilitated his observation of the standardized patient methodology for assessing doctor-patient skills.

Over the past 30 years, medical education has given increasing emphasis to the use of “standardized patients” both for teaching and licensure. Standardized patients are individuals trained to perform a previously-scripted role in an initial clinical examination – always responding in the same manner to the same question, with the same physical complaints, and body language throughout the interview.  They are also trained to fill out a written evaluation form after completion of the simulated examination. The medical profession has concluded that this methodology has made the assessment of clinical skills much more reliable as well as providing an excellent opportunity for students to practice communication and examination skills in a controlled setting prior to examining real patients with real conditions.  In the United States most medical schools  have some standardized patient experiences for students, either as an educational or as an assessment program, and effective in 2004 all medical students must pass a multiple-station examination using standardized patients to qualify for a medical license.

Impressed by the potential for adapting this methodology to the assessment required by GGSL, Maharg proposed a sophisticated research project to ELCC and several other sponsors to test whether a comparable “standardized client” assessment could be developed for legal education. Funding was secured and a team was assembled that included a physician-scholar from the Dundee Medical School in Scotland who was an expert on standardized patient assessment and a statistician with expertise in decision science and policy evaluation.  

Interviewing scenarios based on real cases were developed and a group of standardized clients — primarily retired school teachers — were rigorously trained not only to portray clients in a realistic and consistent manner but also to complete an assessment instrument that focused on aspects of lawyer competence appropriately evaluated from a layperson perspective (such as whether client understood what the lawyer was saying).  The final stage of the project produced compelling evidence that this new methodology was not only very cost-effective but sufficiently valid and reliable to be used for high-stakes assessment. For a complete account of this project,
see Karen Barton, Clark D. Cunningham, Gregory Todd Jones & Paul Maharg, Valuing What Clients Think: Standardized Clients and the Assessment of Communicative Competence, 13 Clinical Law Review 1 (2006), available at http://www.teachinglegalethics.org/valuing-what-clients-think 

According to Maharg, this project – “stemming from another discipline and deeply embedded in research”– changed law teaching at GGSL in “significant and radical” ways. Paul Maharg, Transforming Legal Education: Learning and Teaching Law in the Early Twenty-first Century 67 (Ashgate 2007).     .

During this same time period, UNH was working closely with the supreme court and bar leaders of its state to develop a pilot program for a “two year bar exam,” that would transform the second and third years of law school into a progression of simulated and real-life practice experiences that would be continuously evaluated by outside examiners appointed by the profession.  If these special bar examiners were satisfied upon review of the student’s two year progress that the student was “client-ready,” the student was granted a license to practice law upon graduation without the need to take the usual multi-day written bar admission examination.

Both UNH and the state supreme court were interested in supplementing the bar examiners’ review of a student’s two-year portfolio of work with a supplemental objective, performance-based assessment.   Learning of the successful GGSL experiment, UNH – supported by the ELCC project – engaged Maharg and his GGSL colleague Karen Barton to come to New Hampshire and develop a comparable standardized client program for use there. After completion of that consultation, UNH has added a standardized client assessment as a mandatory requirement for completion of the “two-year bar exam.”  New Hampshire has therefore now joined Scotland as a jurisdiction where competence in lawyer-client communication is formally assessed as part of a lawyer licensing process. For more information on the UNH program, see Clark D. Cunningham, Should American Law Schools Continue to Graduate Lawyers Whom Clients Consider Worthless?, 70 Maryland Law Review 499, 506–512 (2011), available at http://www.teachinglegalethics.org/lawyers-clients-consider-worthless 

The energy generated by these collaborations between practice concerns and scholarly resources has now been transmitted to England and Wales in a highly significant development.  In 2010 the Solicitors Regulation Authority (SRA) made radical changes to the Qualified Lawyers Transfer Test (QLTT), a comprehensive assessment that must be passed by lawyers from other jurisdictions and barristers who wish to be qualified as a solicitor in England and Wales. Drawing from the successful experiment at GGSL, the SRA decided to add an Objective Structured Clinical Examination (OSCE – the same phrase used by medical education) that includes three standardized client assessments. Inasmuch as one in four persons currently being admitted to practice as a solicitor in England and Wales is now entering the profession by passing the QLTT, the impact of adding a mandatory performance-based assessment of client communication may have far-reaching effects both in how candidates prepare for entry to the solicitor’s profession and in how licensed solicitors consider client communication as an essential component of effective practice.  For more information, see Eileen Fry et al., The Qualified Lawyers Transfer Scheme: Innovative Assessment Methodology and Practice in a High Stakes Professional Exam, 46 Law Teacher: the International Journal of Legal Education 132 (2012).