Citation: Burns, D., Brown, M., McNamara, G. and O’Hara, J. (2017) Summary of Assessment and Diversity in Irish Schools. Dublin: EQI – The Centre for Evaluation Quality and Inspection
Download: The content contained on this page can be downloaded at the following link.
This summary of the literature review of assessment in Ireland includes, firstly, the assessment system in Ireland at secondary level and the main issues addressed in assessment literature in Ireland relevant to culturally responsive assessment. Secondly, the summary includes the literature survey of culturally responsive assessment in the United States.
This study adopts a definition for “Culture” that is described as a “cognitive” approach to culture because the project concerns the elements of culture related to “thinking, teaching, learning and making meaning” (Citing Fetterman, Trumbull and Rothstein-Fisch, 2008, p. 3). The project is concerned with cultural dimensions to ways of knowing, epistemology, world-view, beliefs and values. The term “culturally responsive” is interpreted to mean sensitive to, respectful of, taking cognisance of, cultural variations in ways of thinking/knowing, learning, making meaning, values and beliefs.
Assessment at Secondary Level
Students commence 1st Year of secondary after an eight-year cycle of primary schooling, so students are about 12 years of age when entering secondary which consists of three years of Junior cycle, an optional transition year and two years of Senior cycle.
At the end of Junior cycle, students are awarded a Certificate which is equal to level 2 of the European Qualifications Framework. Currently, the Junior Certificate is awarded based on national examinations that are set and administered by the State Examinations Commission. Assessment in Junior cycle is currently undergoing change which is being phased in gradually commencing with one subject in 2016/2017 and will be introduced in all subjects in 2022. The reform includes two classroom-based assessments which will be reported on the Certificate separately from the results of the State examinations.
At the end of the Senior cycle, students are awarded a Leaving Certificate which is considered to be level 3 in the European Qualifications Framework. Proposals for reform of assessment in the Leaving Certificate were published in 2004 but have not been implemented. The established Leaving Certificate (followed by the vast majority of students), for which most students study seven subjects, is assessed externally by the State Examinations Commission, mainly with final written examinations.
In both Senior and Junior cycle, Teachers use informal and formal assessment and reports of formal assessments are distributed to parents. In 3rd and 6th Year students typically sit “Mock” examinations in preparation for the State examinations. Ireland participates in the international standardised assessments of PISA and TIMSS.
Media coverage of educational issues in Ireland is extensive. Media interest in the written examination papers each year is intense and proposals for reform of national assessment are debated in the media.
Issues relevant to Culturally responsive Assessment
The aspiration to respect diversity is enshrined in Irish educational legislation since 1998. Ireland has two significant reports: one published in 2006 by the National Council for Curriculum and Assessment on guidelines for schools for intercultural education, including a chapter on assessment. In 2010 the Department of Education and Skills published a strategy for intercultural education from 2010 to 2015. These publications were prompted by the sudden changing demographic in Ireland since the increase in significant immigration that commenced in the late 1990s but has slowed since 2008. The 2006 document emphasises that all students be enriched by diversity and that minority students be valued with equal respect and opportunity. The report acknowledges the ways that cultural or language factors can give rise to errors in assessment and makes recommendations for assessment to minimise potential errors.
Research on the experience of migrant students in Ireland over the past 15 or so years points to: a school system unprepared for the influx, inadequate systems for recording data, difficulties in accessing schools and gaining appropriate placements in year levels and programmes. The research also suggests that migrant students have high motivation and educational aspirations.
Research in Ireland suggests there are limitations of the national assessment of the State examination in the Leaving Certificate, in the claims that it promotes rote learning and measures a narrow range of lower-order skills. Literature on school-based assessment encourages a range of assessment methods that are alternatives to the traditional written test, such as performance-based, peer and self-assessment, portfolio and e-portfolio assessments.
Baird, J-A, Hopfenbeck, T. N., Elwood, J., Caro, D. & Ahmed, A. (2014). Predictability in the Irish Leaving Certificate. Oxford University Centre for Educational Assessment and Queen’s University Belfast. http://oucea.education.ox.ac.uk
Brown, M. (2012). Deconstructing Evaluation in Education - the case of Ireland. Ph.D Thesis, Dublin City University, Ireland
Brown, M., McNamara, G., & O’Hara, J. (2016). Quality and the rise of value-added in education: The case of Ireland. Policy Futures in Education, 14(6), 810-829
Bryan, A. (2010). Corporate multiculturalism, diversity management and positive interculturalism in Irish schools and society. Irish Educational Studies, 29 (3), 253 – 69. doi: 10.1080/03323315.2010.498566.
Burns, D. (2016). An investigation of the intellectual skill implicit in the aims and assessment of selected subjects in the Leaving Certificate in Ireland. 2005-2010. PhD Thesis. Dublin: Trinity College
Byrne, D., McGinnity, F., Smyth, E. & Darmody, M. (2010). Immigration and school composition in Ireland. Irish Educational Studies, 29 (3), 271 – 88. doi: 10.1080/03323315.2010.498567.
Central Statistics Office. (2012). Census 2011: Profile 7: Religion, Ethnicity and Irish Travellers: Ethnic and Cultural Background in Ireland. Dublin: Government Publications.
Clerkin, A., Perkins, R & Cunningham, R. (2016). TIMSS 22015 in Ireland: Mathematics and Science in Primary and Post-Primary Schools. Dublin: Educational Research Centre
Collins, B. & O’Leary, M. (2010). Integrating assessment with teaching and learning in the visual arts: A study in one classroom. Oideas, 52, 53-61
Commission on the Points System. (1998) Consultative Process — Background Document. Dublin.
Connolly, S. (2011). An exploration into the process and impact of introducing the Assessment for Learning strategies of sharing learning intentions and success criteria with Junior Infants. M.Ed. Thesis. Dublin: St. Patrick’s College, Drumcondra
Conference of Religious of Ireland (CORI). (1998). Inequality in Education: The Role of Assessment and Certification. Dublin: CORI.
Crean, R. (2012). How to Exceed Your Expectations in the Leaving Cert. eBook. Penguin Ireland.
Cullinane, A. (2012). A critical evaluation of the Leaving Certificate biology examination papers and implications for the future. Presentation at Educational Studies Association of Ireland Conference. Cork. 31st March 2012
Darcy, B. (2011) The Impact of Assessment for Learning on Pupil Writing. M.Ed. Thesis. Dublin: St. Patrick’s College, Drumcondra
Department of Education and Skills. www.education.gov.ie
Department of Education and Skills. (2015). Framework for Junior cycle. http://www.juniorcycle.ie/NCCA_JuniorCycle/media/NCCA/Documents/Framework-for-Junior-Cycle-2015-2.pdf
Department of Education and Skills. (2013). Supporting a better transition from second level to Higher Education: Key directions and next steps.
Department of Education and Skills. (2010). Intercultural Education Strategy, 2010 – 2015. http://www.education.ie/en/Schools-Colleges/Information/Intercultural-Education-Strategy/Intercultural-Education-Strategy.html#sthash.m6ty0EoB.dpuf
Douglas, G., McLinden, M., Robertson, C. Travers, J. & Smith, E. (2015). Including Pupils with Special Educational Needs and Disability in National Assessment: Comparison of Three Country Case Studies through an Inclusive Assessment Framework. International Journal of Disability, Development and Education, 63(1), 98-121.
Drudy, S. & Lynch, K. School and Society in Ireland. Dublin. Gill & Macmillan.
Dunphy, E. (2010). Assessing early learning through formative assessment: key issues and considerations. Irish Educational Studies, 29:1, 41-56
Ennis, E. (2006). Implementing a Learning Story Approach to Assessment with Junior Infants. M Ed. Thesis. Dublin: St. Patrick’s College, Drumcondra
Flynn, M. (1998) Changing Views, Changing Assessments: Intelligence, Inequality and the Need to ‘Even the Score’. In Á. Hyland (Ed.). Innovations in Assessment in Irish Education. (pp. 173 – 183). Cork: Multiple Intelligences, Curriculum and Assessment Research Project, Education Department, University College, Cork.
Forde, T. J. (1999). An analysis of factors which influence take-up of second-level senior science in Ireland. Ph.D. Thesis. Dublin: Trinity College
Granville, G. (2004). Politics and Partnership in Curriculum Planning in Ireland. In C. Sugrue (Ed.). Curriculum and Ideology: Irish Experiences and International Perspectives. (pp 67 – 99). Dublin: The Liffey Press.
Hall, K. (2000). A Conceptual Evaluation of Primary Assessment Policy and the Education Policy Process in the Republic of Ireland. Journal of Comparative and International Education, 30(1), 85-101
Hall, K. & Kavanagh, V. (2002). Primary Assessment in the Republic of Ireland: Conflict and Consensus. Educational Management & Administration, 30(3), 261- 274
Harrison, K., O’Hara, J. & McNamara, G. (2015). Re-Thinking Assessment: Self- and Peer-Assessment as Drivers of Self-Direction in Learning. Eurasian Journal of Educational Research, 60, 74-88
Higher Education Authority and National Council for Curriculum and Assessment. (2011). From Transaction to Transition: Outcomes of the Conference on the Transition from Second to Third-level Education in Ireland. A Report to the Minister for Education and Skills, Ruairi Quinn, T.D. http://www.education.ie/en/Publications/Policy-Reports/Supporting-A-Better-Transition-From-Second-Level-To-Higher-Education-.pdf
Hyland, Á. (Ed.). (1998). Innovations in Assessment in Irish Education. Cork: Multiple Intelligences, Curriculum and Assessment Research Project, Education Department, University College, Cork.
Hyland, Á. (2011). Entry to Higher Education in Ireland in the 21st Century. Discussion Paper for the NCCA/HEA Seminar to be held on 21st September 2011. HEA/NCCA.
INTO (2008). Assessment in the Primary School: Discussion Document and Proceedings of the Consultative Conference on Education 2008. Irish National Teachers Organisation.
Irish Universities Association. (2011). Higher education selection and entry mechanisms: Issues and Options. Report of the IUA Working Group.
Kelly, O. (2005). The development, implementation and evaluation of alternative approaches to teaching and learning in the Chemistry laboratory. Ph.D. Thesis. Dublin: Dublin City University
Kilfeather, P., O’Leary, M. & Varley, J. (2006). Adapting science performance tasks developed in different countries for use in Irish primary schools. Irish Educational Studies, 25 (1), 3-33
Kitching, K. (2010). An excavation of the racialised politics of viability underpinning education policy in Ireland. Irish Educational Studies. 29 (3). 213 – 29. doi: 10.1080/03323315.2010.498278.
Lambert, S. (2008) Formative Assessment in Process Writing: Promoting Peer- and Self-Assessment for Learning. M Ed. Thesis. Dublin: St. Patrick’s College Drumcondra.
Lee, A. (2008). Introducing assessment for learning to children in an Irish special class: An exploration of the challenges involved. MSEN Thesis). Dublin: St. Patrick’s College
Little, D. (2005). The Common European Framework and the European Language Portfolio: involving learners and their judgements in the assessment process. Language Testing, 22 (3), 321-336
Looney, A. (2006). Assessment in the Republic of Ireland. Assessment in Education: Principles, Policy & Practice, 13 (3), 345-353
Looney , A. & Klenowski, V. (2008). Curriculum and assessment for the knowledge society: interrogating experiences in the Republic of Ireland and Queensland, Australia. The Curriculum Journal, 19 (3), 177-192
Lynch, K. (1999). Equality in Education. Dublin: Gill and MacMillan.
Lyons, Z. (20160. Articulating a deficit perspective: A survey of the attitudes of post-primary English language support teachers and co-ordinators. Irish Educational Studie,. 29 (3), 289 – 304. doi: 10.1080/03323315.2010.498568.
Lysaght, Z. & O’Leary, M. (2013). An instrument to audit teachers’ use of assessment for learning. Irish Educational Studies, 32 (2), 217-232
Lysaght, Z. (2012). Towards inclusive assessment. In T. Day & J. Travers (Eds.) Special and inclusive education: A research perspective (pp.245-260). Oxford: Peter Lang
Lysaght, Z. (2013). The professional gold standard: Adaptive expertise through assessment for learning. In F. Waldron, J. Smith, M. Fitzpatrick & T. Dooley (Eds.), Re-imagining initial teacher education: Perspectives on transformation (pp.155-176). Dublin: The Liffey Press
MacCóil, L. (2003). Irish — one of the languages of the world. In M. Cronin and C. Ó Cuilleanáin (Eds). The Languages of Ireland. (pp. 127-147). Dublin: Four Courts Press.
Macken, S., & O’Leary, M. (2010). Assessment for learning in Irish primary school physical education. In E. Enright, & D. Tindall (Eds.), Proceedings of the Fifth Physical Education, Physical Activity and Youth Support Forum: A shared vision for physical education, physical activity and youth sport (pp.62-67). Limerick: University of Limerick
Mac Ruairc, G. (2009). ‘Dip, dip, sky blue, who’s it? NOT YOU’: children’s experiences of standardised testing: a socio-cultural analysis. Irish Educational Studies, 28 (10), 47-66
McCrudden, E. (2009). Questioning for appropriate assessment and learning. M.Sc Thesis. Dublin: Dublin City University
McNamara, M. (2010). Exploring the Impact of Standardised Assessment in the Primary School Classroom. M.Ed Thesis. Galway. National University of Ireland, Galway
McSweeney, K. (2014). Assessment practices and their impact on home economics education in Ireland. PhD Thesis. University of Stirling, UK. http://dspace.stir.ac.uk/bitstream/1893/21804/1/Thesis%20Kathryn%20McSweeney.pdf.
Murphy, R. (2000). The validity of a portfolio approach to instruction and assessment in writing in the primary school. MA Thesis. Dublin: Dublin City University
National Council for Curriculum and Assessment (NCCA). http://www.ncca.ie/en/Curriculum_and_Assessment/Assessment/
National Council for Curriculum and Assessment (NCCA). (2009). Key Skills Framework. http://www.ncca.ie/en/Curriculum_and_Assessment/PostPrimary_Education/Senior_Cycle/Key_Skills_Framework/KS_Framework.pdf.
National Council for Curriculum and Assessment (NCCA). (2006). Intercultural Education in the Post-Primary School: Guidelines for Schools. www.ncca.ie/uploadedfiles/publications/interc%20Guide_Eng.pdf
National Council for Curriculum and Assessment (NCCA). (2005). Proposals for the Future Development of Senior Cycle Education in Ireland. www.ncca.ie/uploadedfiles/SCycle%20Advice_0405.pdf.
Ni Chroinin, D. & Cosgrave, C. (2013). Implementing formative assessment in primary physical education: teacher perspectives and experiences. Physical Education and Sport Pedagogy, 18(12), 219-233
Ní Raghallaigh, M. and Gilligan, R. (2010). Active survival in the lives of unaccompanied minors: coping strategies, resilience and the relevance of religion. Child and Family Social Work, 15 (2), 226- 237.
O’Callaghan, C. (2008). Revising Assessment Practices in English Writing. Med Thesis. Dublin: St. Patrick’s College, Drumcondra
O’Hara, J., McNamara, G. and Harrison, K. (2015). Culture changes, Irish evaluation and assessment traditions stay the same? Exploring peer- and self-assessment as a means of empowering ethnic minority students. In S. Hood, R. Hopson & H. Frierson (Eds.). Continuing the journey to reposition culture and cultural context in evaluation theory and practice (pp. 205-231). Charlotte, NC: Information Age Publishing.
O’Leary, M (2006). Towards a Balanced Assessment System for Irish Primary and Secondary Schools. Oideas, 52,7-24
O’Leary, M. (2008). Towards an agenda for professional development in assessment. Journal of In-Service Education, 34 (1), 109-114
Regan, E. M. (2005). Irish student and teacher perspectives on enrolments, subject choice, attitudes towards science and the promotion of Leaving Certificate Chemistry. Ph.D. Thesis. Limerick: University of Limerick.
Shiel, G., Kellaghan, T. & Moran, G. (2010). Standardised testing in lower secondary education. Dublin: Educational Research Centre and the National Council for Curriculum and Assessment. http://ncca.ie/en/Publications/Reports/Standardised_Testing_In_LowerSecondary_Education.pdf
Smyth, B. D. (2007). Studying Chemistry at second level in the Republic of Ireland, 1990 – 2002. D.Ed. Thesis. Open University (United Kingdom)
Smyth, E. (2014). Preparing for life after the Leaving Cert requires more than just rote learning. Independent Newspaper. 12th August 2014.
Smyth, E., Banks, J. & Calvert, E. (2011). From Leaving Certificate to Leaving School: A Longitudinal Study of Sixth Year Students. Dublin: The Liffey Press in association with the Economic and Social Research Institute
State Examinations Commission. www.examinations.ie
Wyatt-Smith, C. M and Looney, A. (2016). Professional standards and the assessment work of teachers. In D. Wyse, L. Hayward & J. Pandya (Eds.), The Sage handbook of curriculum, pedagogy and assessment (pp. 805-820). Sage
Aiding Culturally Responsive Assessment in Schools (ACRAS)
Summary: Conceptual Framework
This summary conceptual framework for the ASCRAS project is based on the outcomes of the literature surveys conducted by the four partner countries. The purpose of this paper is: to draw together the themes and concepts in the literature and the questionnaires in order to point some direction for the future of the project. As such, the paper has application and relevance for the general issue of culturally responsive assessment at secondary level. The paper is set out with definitions, seven themes that emerged from the literature and the questionnaire followed by the concepts that were included in each of the themes.
To date, most of the literature on culturally responsive issues addresses learning and pedagogy (Brown-Jeffy & Cooper, 2011). The focus on culturally responsive teaching and learning has arisen because of the documented inequality in educational achievement of minority cultural groups (for example, Bradshaw, O’Brennan & McNeely, 2008; Raines, Dever, Kamphaus & Roach, 2012). The need to address assessment that is culturally responsive is considered a legal requirement, as well as fairness, ethics and the desirability of diversity. While curriculum and assessment systems differ in the four project partner countries with different levels of centrality and uniformity, classroom assessment is a common feature.
The paper adopts an OECD (2016, p. 7) definition of culture which implies that each individual occupies a unique cultural position and that cultural affiliations are fluid, that an individual student in the classroom may have a sense of identity and belonging that is derived from a constellation of cultures. Another perspective on culture is described as a “cognitive” approach to culture because it concerns the elements of culture related to “thinking, teaching, learning and making meaning” (Citing Fetterman, Rothstein-Fisch & Trumbull, 2008, p. 3).
This project includes in the definition of assessment the design of assessment instruments, the process of conducting the assessment with students and the outcomes of the assessment. This project is focusing on the gamut of classroom assessment, so it includes formal and informal assessment, oral, written and other modes that are applied to classroom assessment.
B.3 Culturally responsive assessment
Following the two definitions above, culturally responsive assessment therefore is adopted to mean assessment design, processes and outcomes that take cognisance of cultural variations in ways of behaving/socialising and thinking.
C.0 Seven themes
To present the concepts discovered in the literature survey and the exploratory questionnaire of schools, seven overarching themes were distilled.
1. The intersection between culture and assessment
2. Assessment and fairness
3. Assessment and cultural dimensions
4. All students benefit from culturally responsive assessment
5. Possibility of culturally responsive assessment
6. Challenge for teachers
7. Supports needed by teachers for culturally responsive assessment
D.0 Concepts included in the seven themes
D.1 The intersection between culture and assessment
D.1.1 Multicultural validity
As validity is a foundational notion of assessment, it must be considered in relation to culture. For students who may be in process of learning the language of the test instrument, construct validity may be a serious validity concern if the “linguistic complexity unnecessarily interferes with pupils’ ability to illustrate their knowledge” in the situation where language factors are “unrelated to the measured construct” such as mathematics (DeBacker, Van Avermaet & Slembrouck, 2017). LaFrance, Kirkhart and Nichols (2015) explore the connection between validity and culture, with awareness that validity does not carry “the same meaning across cultural contexts” (p. 50). Drawing on earlier work of Kirkhart, the authors propose a definition for “multicultural validity” as “the accuracy and trustworthiness of understandings and actions across simple, intersecting dimensions of cultural difference” (p. 57). Multicultural validity embraces five elements, both qualitative and quantitative:
● Theory (rationales support the inferences and actions based on assessment) ● Methodology (design and measurement)
● Relationship (among all forms of life including interactions among people) ● Experience (congruence with the lived experience of participants)
● Consequence (justice of outcomes)
(LaFrance et al, 2015, pp. 58-60).
D.1.2 Assessment and cognitive culture
Section B.1 pointed out that the cognitive approach to culture is concerned with cultural dimensions to ways of learning and epistemology. While these dimensions are relevant to the intersection between culture and assessment, their impact on student performance in assessment is described in some detail in section D.3 below.
D.1.3 Assessment and majority/minority cultures
Students with a migration background usually have cultural affiliations and identities that are different from the dominant culture. Several studies report on the conflict experienced by teenagers between home values and school values (for example, Walsh, 2017).
Some research on the outcomes of international assessments such as PISA indicates a large performance difference between children with the assessment language as their first language and children who have the assessment language as their second language (Herzog-Punzenberger, 2017,
p. 8; Nusche, Shewbridge & Lamhauge Rasmussen 2009, p.7). Kim and Zabelina (2015) suggest that one of the reasons for the persistence in performance difference is that most standardised tests are “normed using the scores of majority group populations” and it “may be inappropriate to use the same assessments with individuals of various racial/ethnic minority groups without norming the instrument to reflect those groups” (p.130).
D.1.4 Language matters
Each of the four partner countries reports the presence of students with a migration background for whom the language of the classroom is not their first language. Students currently in the secondary system may have been born abroad or be the children of parents born abroad. The status, education level and location (urban/rural) of immigrant parents can be factors but the most significant factor in educational achievement for students is the family proximity to the language of the classroom (Engen & Kulbrandstad, 2004; Hvitstendal, 2006, 2008; Özerk, 2001; Randen, 2015; Stevenson & Willot, 2007).
Responses to linguistic diversity in schools vary. Lack of proficiency in the dominant language can lead to approaches of the “deficit” model where students are considered a “problem” or a “risk” (Luciak, 2010) needing compensatory education similar to education for students with special needs (Portera, 2008). At times, school policy has insisted that only the dominant language was to be spoken in the school environment, in corridors and playgrounds. This negative response can devalue student’s culture and language and “can have a negative impact on their overall achievement and motivation” (Herzog-Punzenberger, Le Pichon-Vorstman & Siarova, 2017, p. 24). Many researchers share criticism of this negative response and strongly support multilingualism in schools (for example, Conteh & Meier, 2014; DeBacker et al, 2017; ECML, 2007; Hancock, 2014; Luciak, 2010). Such works emphasise:
● Positivity to all languages
● Teachers need to use the languages of students as a resource
● Giving time to students’ first language helps students to learn a second language more
● Multilingualism gives students the ability to use language resources to negotiate many
different social and academic situations confidently
● All mainstream teachers have to become competent in diverse classrooms ● The best place for second language learners is in the mainstream classroom
● This practice leads to increase in learner autonomy and to meta language awareness.
The European Centre for Modern Languages (ECML, 2007) considers that students need to develop a linguistic repertoire in order to “have easier access to specific languages or cultures by using aptitudes acquired in relation to/in another language or culture” (p. 9). As Herzog-Punzenberger et al, (2017) indicate, “multilingualism is becoming more a way of life than a problem to be solved” (p. 8). Many comments from school personnel in the open-ended comments of the questionnaire indicate that school personnel may not be in alignment with the principles espoused above by the ECML and other authors of linguistic diversity.
D.1.5 Homogeneity or diversity in the class group
Devine (2005) cites Lynch and Lodge (2002) in the claim that teachers tend to “homogenise” students as a group, “overshadowing differences that may exist” (p. 53). Whether a country has a long or a short history of immigration, empirical research has documented the frequent negative discourse of school personnel about migrant students (for example, Hancock, 2017; Lyons, 2010). Opposed to this position of school personnel, is the oft-stated desirability of diversity as a benefit for all students.
D.2 Assessment and fairness
D.2.1 Meaning of “fairness” in assessment
Fairness has become a central concept in the impetus for culturally responsive assessment, mainly as a result of the research on the educational achievement gap for students of different racial/ethnic minorities (Warikoo & Carter, 2009). The American Psychological Association’s 2004 document, Code of Fair Testing Practices in Education and also the 2014 edition of Standards for Educational and Psychological Testing (AERA, APA, NCME) emphasise the obligation to design tests that are fair to all test takers.The literature reveals considerable agreement that fairness in assessment means:
● Recognising diversity of culture
● Acknowledging the knowledge and skill that students bring with them to school ● Being free from bias toward a particular group
● Respecting and articulating construct validity.
In addressing this complex issue of fairness, Gipps and Stobart (2004) cite Linn, 1992 in concluding:
An important approach to offering fairness is to use, within any assessment programme, a range of assessment tasks involving a variety of contexts; a range of modes within the assessment; and a range of response format and style. This broadening of approach, though not always possible, is most likely to offer pupils alternative opportunities to demonstrate achievement if they are disadvantaged by any one particular assessment in the programme (p. 33).
D.2.2 Absence of fairness
Literature documents the hazards that occur if a deficit model rather than a model of respect for difference is dominant in a school/classroom/assessment. As well as inequality in educational achievement and early school leaving the absence of fairness can lead to low levels of motivation, morale, engagement and collaboration (Brown-Jeffy & Cooper, 2011, p. 75).
D.3 Assessment and cultural dimensions
D.3.1 Background to cultural dimensions
Hofstede’s 1980 publication has been revised and expanded in 2001 and often further developed by other authors. Parrish and Linder-VanBerschot (2010) synthesised this work to posit a Framework for Cultural Differences to identify dimensions of cultural difference most likely to impact situations of teaching, learning and assessment. However, a caveat must be noted. As mentioned in the OECD quote in section B.1, “cultural affiliations are dynamic and fluid.” Recognising that culture interacts with human nature and personality, Parrish and Linder-VanBerschot (2010) maintain that the Framework dimensions do not describe “either/or conditions but spectrums along which both cultures and individuals vary” (p. 5). While it is useful to describe cultural differences along these dimensions, “within any culture individuals will differ in how strongly they display these tendencies” (p. 5). In addition, individual students with a migration background will differ in the level of participation in a minority culture and in the level of acculturation to a dominant culture.
D.3.2 Identified cultural dimensions
Parrish and Linder-VanBerschot (2010) identify eight dimensions in three groups (p. 7 – 9) which they claim represent values (p. 6):
● Social relationships: equality and authority, individualism and collectivism, nurture and
● Epistemological beliefs: stability seeking and uncertainty acceptance, logic argumentation
and being reasonable, causality and complex systems
● Temporal perceptions: clock time and event time, linear time and cyclical time. (pp. 7–9)
The Framework suggests ways that each of these dimensions is manifest in learning situations.
D.4 All students benefit from culturally responsive assessment
As well as the migration of peoples the call for cultural responsiveness is based on the trend towards globalisation which Portera (2008) defines as the “spread of the mass media in our daily lives, the growth of information technology, profound geo-political changes, and the establishment of new markets” (p. 481). The need for all young people to learn the normality of diversity and to develop the skills of living in harmonious diversity is urgent.
Researchers who support plurilingualism in classrooms maintain that all students benefit with increase in: language awareness (Hancock, 2017), well-being, self-efficacy and motivation (DeBacker et al, 2017), learner autonomy (Kirwan, 2017). DeBacker et al (2017) report that students who conducted a task in Physics in different languages “appeared to strengthen the concepts the pupils gained.” Questionnaire responses indicate the conviction of school personnel that cultural diversity can benefit all students in developing tolerance, respect and cohesion.
D.5 Possibility of culturally responsive assessment
D.5.1 Criteria for culturally responsive assessment
The APA’s 2004 Code of Fair Testing Practice in Education is designed for application to culturally responsive assessment as well as to a wider number of assessment situations. The Code provides 31 guidelines under four headings: A. Developing and selecting appropriate tests, B. Administering and scoring tests, C. Reporting and interpreting tests results, D. Informing tests takers.
Publications over the past twenty years can be synthesized with the Code to provide a way of considering criteria for culturally responsive classroom assessment. Bringing the concerns of multicultural validity (section D.1.1) and cultural dimensions (section D.3.2) together to consider the implications for classroom assessment for students with a migration background yields questions that can interrogate classroom assessment, in preparation, execution and outcomes. On this basis, the following questions are formulated:
● What is the purpose of the assessment? What construct is being assessed and has that
been communicated to students? What is the rationale for setting standards for the assessment?
● Are participants in assessment provided in advance with information on the coverage of the
test, the types of question formats, the directions, and appropriate test-taking strategies? ● Are participants in assessment informed in advance of the intended purpose and use of the
test, test protocol, standards of performance and all other information relevant to the use of test results?
● Does the design, process and language of the assessment take cognisance of
o Different ways of relating to the teacher (as expert authority or as an equal) o Different ways of student engagement with learning
o Different ways of students approach to knowledge o Different student expectations of assessment
o Different ways of student collaboration
o Different ways of students experiencing failure o The lived experience of participants
● How will the results of the assessment be interpreted? What are the benefits and
limitations of the tests results and their interpretation? Are test results being assigned greater precision than is warranted? What are the factors influencing interpretation of results?
● What are the appropriate uses of the assessment results and the potential misuses?
● Are decisions about participants in assessment made on the basis of multiples of appropriate
information, rather than on a single test score? ● Does the assessment fulfil its intended purpose?
D.5.2 Assessment modes with potential for cultural responsiveness
A body of literature that addresses the search for assessment that has multicultural validity suggests that performance-based assessment has the potential to be more “culturally fair” (Hood, 1998a; Kim & Zabelina, 2015) than traditional tests. The term “performance-based assessment” is used to distinguish assessment strategies that tend to be different from “traditional” strategies in that performance-based strategies aim to assess the use, application or demonstration of knowledge and skill as opposed to simply the recall of knowledge. Performance-based assessment includes a wide variety of tasks.
A study by O’Hara, McNamara and Harrison (2015) found that peer- and self-assessment have potential for assessment that is “more learner-centric, flexible, and culturally responsive” (p. 226). Hempel and Sue-Chan (2010) and Kim and Zabelina (2015) both recommend that including creativity assessment can address cultural bias. Creativity assessment is defined as “producing something that is novel and useful” (Kim & Zabelina, 2015, p. 136). Creative tasks can include a variety beyond the obvious ones such as composing an original piece in literature/music.
D.6 Challenge for teachers
D.6.1 The type of challenge for teachers
The issue of classroom assessment of students with a migration background has significant challenges for teachers as the outcome of results can be so significant for students. The challenges are both conceptual and methodological and “remain complex, multi-faceted, context-rich” (Thompson-Robinson, Hopson, & SenGupta, 2004, p. 3).
The context for such a significant challenge can be drawn from both an historical/political perspective and a professional perspective. Historically, the role of schools and teachers has been very important in establishing the nation-state (Herzog-Punzenberger, 2016, p. 1). With the current movement of peoples and the geo-political changes, it is the schools and teachers that are encountering at first hand the realities of multiculturalism and multilingualism in a new way. Such a challenge can also be considered from the perspective of the professionalism of teachers as teachers understand the role of teacher as a “reflective practitioner” (Schon, 1983). Being a reflective practitioner means “becoming aware of the limits of our knowledge, of how our own behaviour plays into organisational practices and why such practices might marginalise groups or exclude individuals” (Bolton, 2010, p. 14)
The American Evaluation Association (AEA) (2011) published a statement on cultural competence which was described as “a stance taken toward culture, not a discrete status or simple mastery of particular knowledge and skills.” The statement adds, “Cultural competence is not a state at which one arrives; rather, it is a process of learning, unlearning, and relearning.” This is a daunting task, requiring the professional teacher to reflect on practice in a deep manner. The task requires “awareness of self, reflection on one’s own cultural position, awareness of others’ positions, and the ability to interact genuinely and respectfully with others” (AEA).
All authors report the significance of self-awareness and awareness of one’s own enculturation. Bennett (2009) suggests that a gap between knowledge and competence “may be due in part to being unaware of one’s own culture and therefore not fully capable of assessing the cultural position
of others” (p. 123). Bennett further suggests that reflection on a cultural framework to explore one’s own position can begin the process of building intercultural competence (p. 126– 127).
The literature suggests that for teachers to be serious about being a culturally responsive assessor, teachers have to be engaged in an introspective “deliberate and self-exploring” journey (Hood, Hopson & Frierson, 2015, p. xv). Teachers need to be researchers of their own culture and professionalism.
D.6.2 Teachers as researchers of their students
Section B.1 notes that individual’s cultural affiliations are dynamic and fluid and the Framework for Cultural Difference (section D.3.2) notes the ways in which cultural dimensions can be manifest in learning situations. The literature strongly desires that, in order to design valid classroom assessment that is cognisant of the cultural affiliations and language repertoires of their students, teachers consider their students with their varied life and language experience as a resource for the classroom (Conteh, 2017; Moll, Amanti, Neff & Gongalez, 1992).
D.6.3 Demands on teachers
Teachers who are products of their own culture can find it difficult for several reasons to respond positively to the demands of the situation where they have students of diverse cultural backgrounds in the class. Comments on questionnaires indicated little conviction that it is the role of the classroom teacher to be culturally competent in a diverse classroom. Comments also demonstrated the competing priorities on teacher time and attention. Some comments indicated that teachers perceive a tension between high-stakes State level assessment and classroom assessment.
These comments from the questionnaires indicate a contrast in the current position of many school personnel with the vision portrayed in the literature and described in sections D.6.1 and D.6.2. Despite the system-level pressures of high-stakes assessments, the challenge for teachers is to engage in reflection on practice to be culturally competent in a diverse classroom: to reflect deeply on their own enculturation and practice, to research their own students, to affirm the diverse cultural affiliation and life experience of their own students, to be a facilitator of multilingualism in the classroom, to consider the diversity of experience and language in their classroom as a resource for enrichment for all students in the class. These are the requirements for teachers to become competent in designing, conducting and using the outcomes of valid culturally responsive assessment.
D.7 Supports needed by teachers
The depth of the challenge for teachers requires theoretical and practical supports. Indicated supports in the questionnaires range from a need for State level recognition of the needs, policy formulations at national and local levels, professional development, resources for classroom assessment developed centrally and available online, language support. All levels of leadership in the education system need to appreciate the vision for the culturally competent teacher in a multicultural classroom so that policy, resourcing and practice at all levels will support teachers to meet the daunting challenge.
The literature surveys and questionnaires from the four partner countries yielded sixteen concepts for the project. As these concepts provide implications for future practice, they provide signposts for the data collection and data analysis in this project and the outcome of the project: a toolkit for secondary teachers for culturally responsive assessment.
The literature is drawn from many different countries implying that the issue of culturally responsive assessment is relevant in many countries. These sixteen concepts are offered as central considerations wherever the issue of culturally responsive classroom assessment at secondary level is a concern.
American Educational Research Association (AERA), American Psychological Association (APA) and National Council on Measurement in Education (NCME). (2014). Standards for Educational and Psychological Testing 2nd ed.). Retrieved from http://www.apa.org/science/programs/testing/standards.aspx
American Evaluation Association (AEA) (2011). Statement on CulturalCompetence on Evaluation. Retrieved from http://www.eval.org/p/cm/ld/fid=92
American Psychological Association (APA) (2004). Code of Fair TestingPractices in Education. Retrieve d from http://www.apa.org/science/programs/testing/fair-code.aspx
Baker, E. L., O'Neil, H. F.Jr., & Linn, R. L. . (1993). Policy and Validity Prospects for Performance-Based Assessment. American Psychologist. American Psychological Association, 48(12), 1210-1218
Bennett, J. M. (2009). Cultivating Intercultural Competence. In D. K. Deardorff,
(Ed.). The Sage Handbook of Intercultural Competence (pp.121 -140).Sage.
Bradshaw, C. P., O'Brennan, L. M. & McNeely, C. A. (2008), Core competencies
and the prevention of school failure and early school leaving. New Directions for Child and Adolescent Development, 122, 19–32. doi: 10.1002/cd.226
Bolton, G. (2010). Reflective practice: Writing and Professional Development. (3rd ed.). Sage.
Brown-Jeffy, S. & Cooper, J. E. (2011). Toward a Conceptual Framework of Culturally Relevant Pedagogy: An Overview of the Conceptual and Theoretical Literature. Teacher Education Quarterly, 38 (1), 65-84. San Francisco: Caddo Gap Press. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.908.1643&rep=re p1&type=pdf
Conteh, J. (2017). Multilingualism. Conference presentation. Minority language pupils and the curriculum: closing the achievement gap. Marino Institute of Education, Dublin. 25 April 2017
Conteh, J. & Meier, G. (Eds.) (2014). The Multilingual Turn in Languages Education: Opportunities and Challenges. New Perspectives on Language and Education, 1. Multilingual Matters
Council of Europe. The Common European Framework of Reference for Languages: Learning, Teaching, Assessment (CEFR). Strasbourg, France: Council of Europe. Retrieved from
Council of Europe. (2016). Competences for democratic culture: Living together as equals in culturally diverse democratic societies. Strasbourg, France: Council of Europe. Retrieved from https://rm.coe.int/16806ccc07
DeBacker, F., Van Avermaet, P. & Slembrouck, S. (2017). Schools as laboratories for exploring multilingual assessment policies and practice.Language and Education, 31(3), 217-230. doi.org.dcu.idm.oclc.org/10.1080/09500782.2016.1261896
Devine, D. (2005). Welcome to the Celtic Tiger? Teacher responses to immigration and increasing ethnic diversity in Irish schools. International Studies in Sociology of Education, 15 (1). 49-70 http://www.tandfonline.com/toc/riss20/current
Engen, T. O. & Kulbrandstad, L. O. (2004). Individual Bilingualism. In T. O. Engen, &L. O. Kulbrandstad (Eds.),Interrogation, minority language and minority education. Chapter 2. Oslo: Gyldendal Academic Publishing House
European Centre for Modern Languages (ECML). Council of Europe. (2007). From Linguistic Diversity to Plurilingual Education: Guide for the development of language education policies in Europe. Strasbourg, France:Council of Europe. http://www.coe.int/lang
Frierson, H. T., Hood, S., Hughes, G. B., & Thomas, V. G. (2010). A guide to conducting culturally responsive evaluations. In J. Frechtling (Ed.),The 2010 user-friendly handbook for project evaluation. (pp. 75–96). National Science Foundation, Directorate for Education and Human Resources. Retrieved from https://www.purdue.edu/research/docs/pdf/2010NSFuser-friendlyhandbookforprojectevaluation.pdf
Gipps, C. & Stobart, G. (2004). Fairness in assessment. In C. Adams & K. Baker. Perspectives on Pupil Assessment. A paper presented to the GTC conference New Relationships: Teaching, Learning and Accountability, London, 29 November 2004. pp. 31-35. General Teaching Council for England. http://dera.ioe.ac.uk/14022/1/1104_Perspectives_on_Pupil_Assessment._N
Greene, J. C. (2015). Culture and Evaluation: From a Transcultural Belvedere. In S. Hood, R. Hopson, & H. Frierson (Eds.). Continuing the Journey to Reposition Culture and Cultural Context in Evaluation Theory and Practice. Evaluation and Society series. (pp. 91–107. Charlotte, NC. Information Age Publishing.
Hancock, A. (2014). Language education policy in multilingual Scotland: opportunities, imbalances and debates. Language Problems and Language Planning, 38(2), 167-191. doi: 10.1075/lplp.38.2.04han
Hancock, A. (2017). The challenges of multilingualism for schools. Conference presentation. Minority language pupils and the curriculum: closing the achievement gap. Marino Institute of Education, Dublin. 25 April 2017
Hempel, P.S. & Sue-Chan, C. (2010). Culture and the Assessment of Creativity. Management and Organization Review. 6 (3), 415-435. doi: 10.1111/j.1740-8784.2010.00189
Herzog-Punzenberger, B. (2016). Successful integration of migrant children in EU member states: Examples of good practice. NESET II ad hoc question http://nesetweb.eu/wp-content/uploads/2016/02/AHQ-1-2016-Successful-integration_2016.02.04.FINAL_.pdf
Herzog-Punzenberger, B. (2017). Overview of results for the OECD PISA: Study for the ACRAS member countries: Austria, Ireland, Norway, Turkey
Herzog-Punzenberger, B., Le Pichon-Vorstman, E., Siarova, H. (2017).Multilingual Education in the Light of Diversity: Lessons Learned. NESET II report, Luxembourg: Publications Office of the European Union. doi: 10.2766/71255
Hvitstendal, R. (2006). ”I have to learn something from this.” Reading as a way to knowledge for students from linguistic minorities. In E. Elstad,& A. Turmo (Eds.),Learning strategies. Spotlight on teachers' practices. Oslo: University Publisher
Hvistendahl, R. (2008). Sampling and mapping in the language minority education. In E. Selj & E. Ryen (Eds.), With linguistic minorities in the class: Language and academic challenges. Oslo: Cappelen Damm Academic
Hollins, E.R. (1993). Assessing Teacher Competence for Diverse Populations. Theory into Practice, 32 (2), 93-99. Taylor and Francis. doi: 10.1080/00405849309543581
Hollins, E. R. (2015). Culture in School Learning: Revealing the Deep Meaning(3rd ed.). Routledge. New York and London. http://routledgetextbooks.com/textbooks/9780415743457/
Hood, S. (1998a). Introduction and Overview: Assessment in the context of culture and pedagogy: A collaborative effort, a meaningful goal. The Journal of Negro Education, 67 (3), 184-186. doi: 10.2307/2668187
Hood, S. (1998b). Culturally-Responsive Performance-Based Assessment: Conceptual and Psychometric Considerations. The Journal of Negro Education, 67 (3), 187-196. doi: 10.2307/2668188
Hood, S. Hopson, R & Frierson, H. (2015). Introduction: This is where we continue to stand. In S. Hood, R. Hopson & H. Frierson (Eds.), Continuing the Journey to Reposition Culture and Cultural Context in Evaluation Theory and Practice.Evaluation and Society series. (pp.ix-xviii). Charlotte, NC: Information Age Publishing
Johnson, S. T., Thompson, S. D. & Wallace, M. B. (1998) How teachers and university faculty perceive the need for and importance of professional development in performance-based assessment. The Journal of Negro Education, 67 (3). pp. 197-210. doi: 10.2307/2668189
Karatas, K. & Oral, B. (2015). Teachers’ Perceptions on Culturally Responsiveness in Education.Journal of Ethnic and Cultural Studies, 2 (2), 47-57. http://www.ejecs.org/index.php/JECS
Kim, K. H. & Zabelina, D. (2015). Cultural Bias in Assessment: Can Creativity
Assessment Help? International Journal of Critical Pedagogy. 6 (2), 129-147. University of North Carolina. https://libjournal.uncg.edu/ijcp
Kirwan, D. (2017). Case study of a linguistically diverse primary school. Conference presentation. Minority language pupils and the curriculum: closing the achievement gap. Marino Institute of Education, Dublin. 25 April 2017
LaFrance, J., Kirkhart, K. E. & Nichols, R. (2015). Cultural Views of Validity: A
Conversation. In S. Hood, R. Hopson & H. Frierson (Eds.). Continuing the Journey to Reposition Culture and Cultural Context in Evaluation Theory and Practice. (pp. 49-72). Charlotte, NC: Information Age Publishing
Lee, C. D. (1998). Culturally responsive pedagogy and performance-based assessment. The Journal of Negro Education, 67(3), 268-279. doi: 10.2307/2668195
Lidz, C. S. (2001). Multicultural issues and dynamic assessment. In L.A. Suzuki, J. G. Ponterotto, & P.J. Meller (Eds.),Handbook of Multicultural Assessment. 2nd ed. (pp. 523-539). San Francisco: Jossey-Bass.
Luciak, M. (2010). Educating teachers for diversity: Meeting the challenge. On diversity in educational contexts. In: Centre for Educational Research and Innovation. In OECD/CERI, (pp. 41- 62). Paris: OECDhttp://www.oecd.org/edu/ceri/educatingteachersfordiversitymeetingth echallenge.htm
Lyons, Z. (2010). Articulating a deficit perspective: a survey of the attitudes of post-primary English language support teachers and coordinators. Irish Educational Studies, 29 (3), 289 – 304. doi: 10.1080/03323315.2010.498568
Messick, S. (1989). Validity. In R. L. Linn (Ed.). Educational Measurement, (3rd ed.). (pp. 13–103). New York: Macmillan
Messick, S. (1995). Test Validity and the Ethics of Assessment. Princeton, NJ. Educational Testing Service. doi: apa.org/journals/amp/35/11/1012.pdf
Moll, L. C., Amanti, C., Neff, D. & González, N. (1992). Funds of knowledge for teaching: Using a qualitative approach to connect homes and classrooms. Theory Into Practice, 31, 132–141
National Council for Curriculum and Assessment (NCCA). (2006). Intercultural Education in the Post-Primary School: Guidelines for Schools. Dublin: NCCA. www.ncca.ie/uploadedfiles/publications/interc%20Guide_Eng.pdf
Nusche, D. Shewbridge, C. & Lamhauge Rasmussen, C. (2009). OECD Reviews on Migrant Education. Austria. OECD 2009. Retrieved from http://www.oecd.org/austria/44192225.pdf Download 3.2.2017
O’Hara, J., McNamara, G. & Harrison, K. (2015). Culture Changes, Irish Evaluation and Assessment Traditions Stay the Same? Exploring Peer-and Self-assessment as a Means of Empowering Ethnic Minority Students. In S. Hood, R. Hopson & H. Frierson (Eds.). Continuing the Journey to Reposition Culture and Cultural Context in Evaluation Theory and Practice (pp. 205-231). Charlotte, NC: Information Age Publishing
Organisation for Economic Co-operation and Development (OECD). (2009). Toolkit for Teaching on Diversity.
Organisation for Economic Co-operations and Development (OECD). (2013). Synergies for better learning: An international perspective on evaluation and assessment. OECD reviews of evaluation and assessment in education. Paris: OECD.
Organisation for Economic Co-operation and Development (OECD). (2016). Global competency for an inclusive world. Paris: OECD. https://www.oecd.org/education/Global-competency-for-an-inclusive-world.pdf
Özerk, K. (2001). Teacher–Student Verbal Interaction and Questioning, Class Size and Bilingual Students’ Academic Performance. Scandinavian Journal of Educational Research, 45(4), 353–367. doi.org/10.1080/00313830120096761
Özerk, K. (2009). Learning of learning materials and developing language at school. Some methodological ideas based on proven work in Oslo Schools. Norwegian Pedagogical Journal, 93, 294-309
Padilla, A. M. (2001). Issues in culturally appropriate assessment. In L.A.Suzuki, J. G. Ponter otto, & P.J. Meller (Eds.). Handbook of Multicultural Assessment. (2nd ed.) (pp. 5-27). San Francisco: Jossey-Bass.
Parrish, P. & Linder-VanBerschot, J. A. (2010). Cultural dimensions of learning: Addressing the challenges of multicultural instruction. International Review of Research in Open and Distance Learning, 11 (2), 1-19. www.irrodl.org/index.php/irrodl/article/viewFile/809/1553
Qualls, A. L. (1998). Culturally Responsive Assessment: Development Strategies
and Validity Issues. The Journal of Negro Education, 67(3), 296-301. doi: 10.2307/2668197
Raines, T. C., Dever, B. V., Kamphaus, R. W. & Roach, A. T. (2012). Disproportionate Placement in Special Education. The Journal of Negro Education, 81(3), 283-296. doi:10.7709/jnegroeducation.81.3.0283
Randen, G. T. (2015). Assessment of minority language proficiency in primary
school. Norwegian as a Second Language, 30(1-2), 350 – 372
Rothstein-Fisch, C. & Trumbull, E. (2008). Managing Diverse Classrooms: How
to Build on Students’ Cultural Strengths. Alexandria, US: Association for Supervision & Curriculum Development (ASCD), 2008. ProQuest ebrary. Web. 2 October 2016. Ebook. https://books.google.ie/books?hl=en&lr=&id=WvZQBAAAQBAJ&oi=fnd&pg =PR9&dq=Managing+Diverse+Classrooms&ots=d6WwQ6OGC1&sig=VRyt FxPURSb7ZRdVbrDUqrzU-rc&redir_esc=y#v=onepage&q=Managing%20Diverse%20Classrooms&f=fa lse
Schon, D. A. (1983). The Reflective Practitioner: How Professionals Think in
Action. New York. Basic Books
Stevenson, J. & Willott, J. (2007). The aspiration and access to higher education
of teenage refugees in the UK. Compare: A Journal of Comparative and International Education, 37(5), 671-687. doi.org/10.1080/03057920701582624
Symonette, H. (2015). Culturally Responsive Evaluation as a Resource for
Helpful-Help. In S. Hood, R. Hopson, & H. Frierson (Eds.),Continuing the Journey to Reposition Culture and Cultural Context in Evaluation Theory and Practice. Evaluation and Society series. (pp. 109-129). Charlotte, NC. Information Age Publishing.
Thompson-Robinson, M., Hopson, R. K. & SenGupta, S. (Eds.). (2004). Editors’
Notes. New Directions for Evaluation, 102. (1-4). San Francisco, CA: Jossey-Bass. doi:10.1002/ev.111/epdf
Walsh, A. (2017). Make Minority a Priority: Insights from minority ethnic young
people. National Youth Council of Ireland. http://www.youth.ie/minority
Warikoo, N. & Carter, P. (2009). Cultural Explanations for Racial and Ethnic
Stratification in Academic Achievement: A Call for a New and Improved Theory. Review of Educational Research, 79 (1), 366-394. American Educational Research Association. doi: 10.3102/0034654308326162