What is “the best” way to assess emergent bilinguals?

9 March 2017

Last month we published The Assessment of Emergent Bilinguals by Kate Mahoney. In this post, Kate explains how she came to dedicate her research to this topic and introduces us to her decision-making framework, PUMI (Purpose, Use, Method, Instrument), that can be used to better inform assessment decisions for bilingual children.

Since my first days as a teacher, I wanted to answer questions about how language and culture impact learning and schooling. I found myself teaching in Puerto Rican communities in New York, Navajo communities in New Mexico, Mexican communities in the Southwest, and in bilingual communities in Belize. Each experience drove an awakening clarity: assessment was an incredibly powerful influence on schooling and success, and language and culture strongly influenced assessment. In 1999, my then-advisor Dr. Jeff MacSwan at Arizona State University (ASU) suggested I adopt the study of tests and the testing process – within the context of bilingual learners – as a research topic. Admittedly, I was reluctant to begin a formal study involving psychometrics, language assessment and related methodologies, but I needed a multidisciplinary approach to answer questions. I was reluctant because the topic of testing seemed so frustrating and unfair, and seemed to privilege some students over others, based primarily on the relationship between culture and language. It was this reluctance that led me to begin my study of assessment, and from multiple disciplines. At the same time, I began teaching graduate courses in assessment for the multilingual programs at ASU. I’ve continued to teach this course throughout my career and today teach and conduct research at the State University of New York at Fredonia.

As I think back over the past 15-plus years of researching this topic, I’m continually struck by its complexity, and how difficult it can be for classroom teachers to learn about and stay abreast of the evolving methodologies. There is so much more to assessment than simply establishing a rubric and giving the test. Because of the complexity and multidisciplinary nature of assessment, it was difficult to deliver a course on assessment in a connected way to university students. That’s why I developed PUMI (Purpose, Use, Method, Instrument) for my first class on the subject back in 1999. I didn’t call it PUMI back then, but my students and I always discussed assessments within this framework, and it became an important way to make decisions and select appropriate assessments, while also understanding the complexities of emergent-bilingual assessment.

This book about the assessment of emergent bilingual learners is the culmination of teaching a university course for the past 18 years. I use the PUMI framework across the whole book; it’s a decision-making process teachers can use to make better assessment-related decisions. Also included are more in-depth topics in assessment that warrant full attention, such as validity as a theory, the history of the assessment of bilingual children, as well as testing accommodations and accountability topics.

Over the years, many people have approached me to ask about “the best” assessment or test for assessing Spanish or assessing math with emergent bilinguals. The answer is definitely not prepackaged, and not easy for that matter either. To begin to understand the answer to these types of questions, one must ask PUMI questions, and in that order. So, my response to questions about the best assessment is always first, what is the purpose “P” of the assessment and how will you use “U” the results. After considering the purpose and use, then we can begin to consider the best assessment method “M” and instrument “I”. Selecting an appropriate assessment for emergent bilinguals is not an easy task, but PUMI can guide us toward better assessment for this unique group of students.

For more information about this book, please see our website. If you liked this, you might also be interested in Foundations of Bilingual Education and Bilingualism (6th Edition) by Colin Baker and Wayne E. Wright.


Publication of Multilingual Matters’ First Open Access Book: A Milestone for Pronunciation Assessment

8 December 2016

This month we are publishing our first open access title, Second Language Pronunciation Assessment, edited by Talia Isaacs and Pavel Trofimovich. In this post, Talia tell us about her experiences of researching the once unfashionable topic of pronunciation as well as the importance of open access publishing.

When I started doing dissertation research on pronunciation assessment in 2004 during my Master’s degree, this topic was drastically out of fashion. Pronunciation in language teaching had had its heyday earlier in the 20th century, culminating, from an assessment perspective, with the publication of Lado’s seminal book, Language Testing, in 1961, which is viewed as signifying the birth of the language testing field. There had been little sustained research on the topic in the years since. In my early days as a postgraduate student, when I stated that I was researching pronunciation assessment to members of the language assessment community, I remember thinking that reactions from some, however polite, were similar to how some passersby might respond when looking at an odd relic in a museum through protective glass—that this topic may have had some use or merit once in a misguided way but is now decidedly passé. Pronunciation carried much baggage in assessment circles and within language teaching and applied linguistics more generally as a symbol of the decontextualized drills targeting linguistic forms that had been left behind during the Communicative era.

I could not have foreseen, at the time, that pronunciation would gradually begin to embed itself in some of the discourse and have growing visibility in scholarly fora (e.g. Language Testing Research Colloquium), at least on the periphery. Spurred by trends in researching pronunciation in other areas within applied linguistics, where developments happened earlier (e.g. SLA, sociolinguistics), modern work on pronunciation shifted to a focus on intelligibility and listeners’ evaluative judgments of speech and was at the heart of developments in automated scoring of speaking. Assessing pronunciation was, thus, informally rebranded and was able to establish some contemporary relevance.

Second Language Pronunciation AssessmentFast forward to 2017, with the publication of Second Language Pronunciation Assessment. This book breaks new ground in at least two respects. First, it is the first edited collection ever published on the topic of pronunciation assessment. Although the volume is far from comprehensive, it begins to establish a common understanding of key issues and bridges different disciplinary areas where there has historically been little conversation.

Second, it is Multilingual Matters’ first “gold” open access book. Tommi Grover shared his thoughts on open access in a recent blog post, and it is exciting that our book is at the forefront of what we believe will be a growing trend in monograph publishing in time.

When we first learned that our external research funding could pay for open access costs for a monograph to a maximum amount pre-specified by the funder as part of a post-grant open access scheme, we broached this with Tommi and Laura Longworth. It is a luxury to have a world-renowned applied linguistics publisher practically on my doorstep in Bristol, UK, where I currently reside. Tommi mused about some of the pros and cons and the likely logistical challenges aloud over coffee, and I am sure that the flies on the wall were intrigued. It was clear that pursuing open access entailed a degree of risk for the publisher, as the open access maximum payment from the funder alone would not cover the full production costs and overheads. We were delighted that, ultimately, the Multilingual Matters team decided to treat our book as an experiment and go for the open access option to see how it would work.

Thus, with the publication of our book, a scenario that once seemed hypothetical has now become a reality, and our contributors were also very happy about this development. The push for open access within the academy is to ensure that publically-funded research outputs are also publically available free of charge where possible. From the readers’ and authors’ perspective, it is generally preferable to be able to access the official versions with professional typesetting than to have author-approved unofficial versions with different page numbers floating around. We believe that, through the availability of our publication for free download, it will reach a much wider readership than it would have had the access costs been levelled onto the consumer. The print version has also been sensibly discounted for those who still wish to purchase a softback copy. In this way, it will hopefully reach the interdisciplinary audiences of researchers and educational and assessment stakeholders that we feel would benefit from knowing about this book and inform further research and practice.

My co-author, Pavel Trofimovich, and I could not have envisioned a more positive experience working with the Multilingual Matters team from start to finish. As Pavel wrote in an email, reflecting on the publication process, “I cannot think of any publisher who [is] so professional, hands-on, and also human in their interaction with colleagues.” We are extremely grateful for the tremendous help and advice in navigating all aspects of the publication of our book, including dealing with the unexpected. This process has been enriching and the production tremendously efficient. We would highly recommend that any prospective authors in applied linguistics, new or experienced, consider Multilingual Matters as a venue for publishing their book. If you have internal or external funds available or could budget for open access costs for a monograph into a grant application, it might be worthwhile pre-empting a conversation with Tommi about open access. This is an option that the team is clearly open to and which may, in time, revolutionize the publication of monographs, as it already has with academic journal articles.

Talia Isaacs, UCL Institute of Education, University College London, UK
Pavel Trofimovich, Department of Education, Concordia University, Canada

For further information about the book, please see our website. For more information about open access please read Tommi’s blog post or contact him directly at tommi@multilingual-matters.com.

To download the open access ebook please go to the following link: https://zenodo.org/record/165465.


%d bloggers like this: