ESL 6638 Assessment of the Adult ESL Student Online
(2 sem. cr.) Dates: June 28-August 9 "Learn the basic principles of testing and evaluation, and how assessment is related to curriculum. Learn how to conduct valid formal and informal assessment of ESL learners. Develop authentic assessment tools for academic and workplace settings. Discuss entrance and exit criteria for ESL services and how to assess student progress. Explore the politics of testing and assessment. Online course and login information at www.hamline.edu/gseonline. Optional orientation session at Hamline on June 28, 10:00 a.m.-12:00p.m. Drew Science 305. Target Audience: ESL teachers of adult learners.
Pyschometrics-"...field of study concerned with the theory and technique of educational and psychological measurement, which includes the measurement of knowledge, abilities, attitudes, and personality traits. The field is primarily concerned with the study of measurement instruments such as questionnaires and tests. It involves two major research tasks, namely: (i) the construction of instruments and procedures for measurement; and (ii) the development and refinement of theoretical approaches to measurement..." eugenics-"...is a social philosophy which advocates the improvement of human hereditary traits through various forms of intervention.[2] Throughout history, eugenics has been regarded by its various advocates as a social responsibility, an altruistic stance of a society, meant to create healthier and more intelligent people, to save resources, and lessen human suffering." Craniometry-"..is the technique of measuring the bones of the skull. It is distinct from phrenology, the study of personality and character, and physiognomy, the study of facial features. However, these fields have all claimed the ability to predict traits or intelligence."
*see Rubrics "scoring scale, is typically created which contains the essential criteria for the task and appropriate levels of performance for each criterion... holistic rubric-"assigns a level of performance by assessing performance across multiple criteria as a whole" analytic rubric articulates levels of performance for each criterion so the teacher can assess student performance on each criterion.
valid-"degree to which a certain inference from a test is appropriate and meaningful" (AERA, APA, & NCME, 1985).." reliable-consistent washback-effect a test has on teaching and learning What Is "Washback" and Why Do Some Language Researchers Avoid the Term? , from jalt.org modality-channel (aural/oral) of language use
indirect vs. direct tests
*see A Comparison of Direct & Indirect Assessments of Writing Skill, by Hunter M. Breland discrete-point vs. integrative testing
*see Methods of Assessment, btinternet.com "Discrete point tests: cannot be pragmatic. There is no ordinary discourse situation and no normal language use context where a learner might be asked to listen and distinguish between "ship" & "sheep" or perform active to passive transformations." What is a discrete point approach to testing? "Electronic quiz tools usually involve a discrete point approach to testing as opposed to an integrated or authentic approach, such as papers and projects. Discrete point tests are made up of test questions each of which is meant to measure one content point. Discrete point testing is associated with multiple choice and true/false formats, which have been criticized for testing only recognition knowledge and facilitating guessing and cheating. However, if they are used for an appropriate PURPOSE and if the test questions are well constructed, discrete point tests can be used for effective teaching and learning..." objective-vs. subjective scoring
*see
Objective and subjective evaluation "Objective tests are often constructed with selected-response item formats, such as multiple-choice, matching, and true-false. An advantage to including selected-response items in objectively scored tests is that the range of possible answers is limited to the options provided by the test writer—the test taker cannot supply alternative, acceptable responses.
*concurrent validity (wikipedia)-"is demonstrated where a test correlates well with a measure that has previously been validated. The two measures may be for the same construct, or for different, but presumably related, constructs.
For example, if a test measuring job satisfaction gives similar results to those gathered using a job satisfaction test which has been validated in past investigations the new measurement has concurrent validity. Alternately, a measure of job satisfaction might be correlated with work performance. Note that with concurrent validity, the two measures are taken at the same time. This is in contrast to predictive validity, where one measure occurs earlier and is meant to predict some later measure."
*inter-rater reliability (wikipedia)-"iss the degree of agreement among raters. It gives a score of how much homogeneity, or consensus, there is in the ratings given by judges. It is useful in refining the tools given to human judges, for example by determining if a particular scale is appropriate for measuring a particular variable. If various raters do not agree, either the scale is defective or the raters need to be re-trained."
CURRICULUM-BASED TESTS-"..measurement that uses "direct observation and recording of a student's performance in the local curriculum as a basis for gathering information to make instructional decisions" (Deno, 1987, p. 41)...provides a structured way to see how well a child performs on the materials the teachers is assigning the class.(from umn.edu/~devenz) ".. refer to instruments that assess one element/component of a skill at a time" (Hamline Univ) COMPETENCY-BASED TESTS-"" What is CBT & Characteristics of CBT "..A skill performed to a specific standard under specific conditions...Assessment of competency takes the participant’s knowledge and attitudes into account but requires actual performance of the competency as the primary source of evidence. .. DIAGNOSTIC TESTS -"" Test your English language ability
Online diagnostic tests
*internal consistency(wikipedia)-"is a measure based on the correlations between different items on the same test (or the same subscale on a larger test). It measures whether several items that propose to measure the same general construct produce similar scores. For example, if a respondent expressed agreement with the statements "I like to ride bicycles" and "I've enjoyed riding bicycles in the past", and disagreement with the statement "I hate bicycles", this would be indicative of good internal consistency of the test."
"..Victorian polymath: geographer, meteorologist, tropical explorer, founder of differential psychology, inventor of fingerprint identification, pioneer of statistical correlation and regression, convinced hereditarian, eugenicist, proto-geneticist, half-cousin of Charles Darwin and best-selling author..." Wikipedia "..created the statistical concept of correlation and widely promoted regression toward the mean. He was the first to apply statistical methods to the study of human differences and inheritance of intelligence, and introduced the use of questionnaires and surveys for collecting data on human communities, which he needed for genealogical and biographical works and for his anthropometric studies. He was a pioneer in eugenics, coining the very term itself and the phrase "nature versus nurture". As an investigator of the human mind, he founded psychometrics (the science of measuring mental faculties) and differential psychology. He devised a method for classifying fingerprints that proved useful in forensic science. As the initiator of scientific meteorology, he devised the first weather map, proposed a theory of anticyclones, and was the first to establish a complete record of short-term climatic phenomena on a European scale.[1] He also invented the Galton Whistle for testing differential hearing ability..."
"..Yet, his technical and theoretical contributions to the development of intelligence research as a scientific enterprise cannot be overstated. He was the first to offer a tenable psychometric definition of intelligence, and is therefore considered to be the father of classical test theory (Jensen, 1994). In a famous article, “General Intelligence’ Objectively Determined and Measured” (1904), Spearman proposed the idea that intelligent behavior is generated by a single, unitary quality within the human mind or brain. Spearman derived this theoretical entity, called the general factor, or simply g, through a new statistical technique that analyzed the correlations among a set of variables. This technique, called factor analysis, demonstrated that scores on all mental tests are positively correlated; this offered compelling evidence that all intelligent behavior is derived from one metaphorical pool of mental energy. Although proponents of multiple intelligence theory reject this interpretation, factor analysis remains one of the most important tools in 21 st century intelligence research..." Wikipedia "..FRS (September 10, 1863 - September 17, 1945) was an English psychologist known for work in statistics, as a pioneer of factor analysis, and for Spearman's rank correlation coefficient. He also did seminal work on models for human intelligence, including his theory that disparate cognitive test scores reflect a single general factor and coining the term g factor..."
... remains one of the most complex and intriguing figures in the history of intelligence testing. He was a pioneer of educational psychology in England and was one of the most respected and honored psychologists of his time. However, he had controversial ideas regarding the heritability of intelligence, and there is ample evidence that he used fraudulent data to support his views (Scarr, 1994)..." Wikipedia ".. (March 3, 1883 – October 10, 1971) was an English educational psychologist who contributed to [1] educational psychology and claimed to have developed the method of factor analysis in psychological testing,[2] although his mentor and predecessor as chair of the psychology department at University College London, Charles Spearman actually did so.[3] Burt is known for his studies on the effect of heredity on intelligence. Shortly after he died, his studies of inheritance and intelligence came into disrepute after evidence emerged indicating he had falsified research data,[2] in addition to having falsely claimed to have invented factor analysis. Some scholars have asserted that Burt did not commit intentional fraud.[citation needed].."
"... made significant contributions in many areas of psychology, including psychometrics, statistics, and the study of human intelligence. He developed methods for scaling psychological measures, assessing attitudes, and test theory, among many other influential contributions. He is best known for the development of new factor analytic techniques to determine the number and nature of latent constructs within a set of observed variables..." Wikipedia "..29 May 1887–30 September 1955) was a U.S. pioneer in the fields of psychometrics and psychophysics. He conceived the approach to measurement known as the law of comparative judgment, and is well known for his contributions to factor analysis..."
"..is most widely known for his contributions to intelligence. Wolf (1973) postulates that this is the result of his not being affiliation with a major university. Because Binet did not have any formalized graduate study in psychology, he did not hold a professorship with a prestigious institution where students and funds would be sure to perpetuate his work (Siegler, 1992). Additionally, his more progressive theories did not provide the practical utility that his intelligence scale would evoke..." Wikipedia "..(July 8, 1857 – October 18, 1911), French psychologist and inventor of the first usable intelligence test, the basis of today's IQ test.
Born in Nice, Binet was a French psychologist who published the first modern intelligence test, the Binet-Simon intelligence scale, in 1905. His principal goal was to identify students who needed special help in coping with the school curriculum. Along with his collaborator Théodore Simon, Binet published revisions of his intelligence scale in 1908 and 1911, the last appearing just before his untimely death. A further refinement of the Binet-Simon scale was published in 1916 by Lewis M. Terman, from Stanford University, who incorporated the German psychologist William Stern's proposal that an individual's intelligence level be measured as an intelligence quotient (I.Q.). Terman's test, which he named the Stanford-Binet Intelligence Scale formed the basis for one of the modern intelligence tests still commonly used today. They are all colloquially known as IQ tests..."
"..(April 29, 1871 - March 27, 1938), born Wilhelm Louis Stern, was a German psychologist and philosopher noted as a pioneer in the field of the psychology of personality and intelligence. He was the inventor of the concept of the intelligence quotient, or IQ, later used by Lewis Terman and other researchers in the development of the first IQ tests, based on the work of Alfred Binet. He was the father of the German writer and philosopher Günther Anders. In 1897, Stern invented the tone variator, allowing him to research human perception of sound in an unprecedented way..."
"..(August 14, 1866 – June 18, 1957) was a prominent American psychologist and eugenicist in the early 20th century. He is known especially for his 1912 work The Kallikak Family: A Study in the Heredity of Feeble-Mindedness, which he himself came to regard as deeply flawed, and for being the first to translate the Binet intelligence test into English in 1908 and distributing an estimated 22,000 copies of the translated test across the United States; he also introduced the term "moron" into the field. He was the leading advocate for the use of intelligence testing in societal institutions including hospitals, schools, the legal system and the military. He played a major role in the emerging field of clinical psychology, in 1911 helped to write the first U.S. law requiring that blind, deaf and mentally retarded children be provided special education within public school systems, and in 1914 became the first American psychologist to testify in court that subnormal intelligence should limit the criminal responsibility of defendants..."
"... (born 15 January 1877 in Johnson County, Indiana, died 21 December 1956 in Palo Alto, California) was a U.S psychologist, noted as a pioneer in cognitive psychology in the early 20th century at Stanford University. He is best known as the inventor of the Stanford-Binet IQ test. He was a prominent eugenicist and was a member of the Human Betterment Foundation."
"... is a body of theory describing the application of mathematical models to data from questionnaires and tests as a basis for measuring abilities, attitudes, or other variables. It is used for statistical analysis and development of assessments, often for high stakes tests such as the Graduate Record Examination. At its most basic level, it is based on the idea that the probability of getting an item correct is a function of a latent trait or ability. For example, a person with higher intelligence would be more likely to correctly respond to a given item on an intelligence test..."
Gardner's Multiple Intelligences and Sternberg's Triarchic Theory
"The theory of multiple intelligences: ...Howard Gardner viewed intelligence as 'the capacity to solve problems or to fashion products that are valued in one or more cultural setting' (Gardner & Hatch, 1989). He reviewed the literature using eight criteria or 'signs' of an intelligence:.." Wikipedia "..The theory suggests that, rather than relying on a uniform curriculum, schools should offer "individual-centered education", with curriculum tailored to the needs of each child.[1] (This includes working to help students develop the intelligences in which they are weaker.).."
"..Under No Child Left Behind, states are working to close the achievement gap and make sure all students, including those who are disadvantaged, achieve academic proficiency. Annual state and school district report cards inform parents and communities about state and school progress. Schools that do not make progress must provide supplemental services, such as free tutoring or after-school assistance; take corrective actions; and, if still not making adequate yearly progress after five years, make dramatic changes to the way the school is run..."
"..In spite of the information explosion, America hasn't heard the message. Perhaps our leading change agents don't know what their deceptive strategies and dumbed-down curricula will do to our children. Maybe they haven't noticed the similarities between their educational strategies and former Nazi tactics for molding young minds and teaching group conformity. Perhaps today's psychological manipulations are simply a modern expression of human "wisdom" without biblical guidelines..."
"..The SAT measures, above all else, class, sex, and race. (Fairtest, Roney). The SAT, like every similar test, is designed to divide people with razor sharp precision, to enumerate human value and to track people's futures under a veil of objective science. The SAT is a commodity itself, for sale to every student, school, and college in the world. It also commodifies people, attaching worth to individuals, but more pointedly drawing the lines of what can only be called class warfare. The impact of the SAT is to create the logic for a more deeply stratified society, divided primarily by issues of inherited income, sexism, and racism. The fraudulent claims of the SAT to promote a more equitable and meritorious society have been thoroughly revealed elsewhere. (Lemann, Fairtest, Roney). This analysis is a brief history of the SAT, unmasking the politics of the people who designed it, and those who promote the SAT and similar exams today ..."
"..Its central point is that intelligence is a better predictor of many factors including financial income, job performance, unwed pregnancy, and crime than parents' socioeconomic status or education level. Also, the book argued that those with high intelligence, which it called the "cognitive elite", are becoming separated from the general population of those with average and below-average intelligence and that this was a dangerous social trend..."
"..The fact is, even if such biased items are removed from the SAT, the unequal educational experience of the students taking the test -- especially in terms of class and race -- all but guarantees a persistent scoring gap between whites on the one hand, and blacks, American Indians or Latinos on the other.
"..Black students are well aware of the negative stereotypes held about them by members of the larger society. As such, when blacks who are highly motivated and value educational achievement take a standardized test and expect the results to be used to indicate cognitive ability, the fear of living down to the stereotype negatively impacts their performance. These students may rush through the test—so as to seem more confident than they truly are—or alternately take too much time, trying desperately not to make mistakes. The self-doubt engendered by the racist beliefs of the larger culture is added to the general anxiety that all test-takers feel, to produce, for black students, a unique disadvantage. ..."
"..Brigham had begun administering the SAT to test groups in 1926, and had concluded by 1933, when he met Chauncey, that his SAT reliably predicted academic success. Conant and Chauncey seized upon this conclusion, began administering the test to students who wished to win scholarships at Harvard, and convinced many of their Ivy League colleagues to follow suit.."
"...Education [The County Board of Education] shall provide schools of two kinds; those for white children and those for colored children. Texas "
DISCUSSION BOARD:
-Topic: Doubters and Believers
Date: Tue Jul 15 2008 23:27
Author: M, Sal
Subject: certificate to show! Re: #2 oppressive and humiliating "yes, students would like to see scores or some way to know they are doing well. I had one student (from Bulgaria) that asked me and my "supervisor" if we could give him a certificate. He wanted to go back to his home country to show his people (future place of employment, family, etc..) that he took an ESL course while working (was a intern as a vet in a local nearby pig farm) in America! It was the first time my "supervisor" and I ever been asked this, which we will try to do more often now!"
Date: Wed Jul 16 2008 15:10
Author: W, Todd
Subject: Re: certificate to show! Re: #2 oppressive and humiliating "
Several programs around the country, including Minneapolis, issue certificates to students based on reaching certain score/proficiency levels on the CASAS tests. My understanding is that this process has been well-received in Minneapolis by both students and teachers. Implementing this process also promoted a great deal of consideration of the CASAS compentencies and whether they constitute a reasonable/quality/useful/valuable content domain of instruction for an ESL program. My understanding is that that process was also well received and that many teachers who were skeptical of relying heavily on the CASAS competencies have found the compentencies, on closer examination, to be reasonable/useful.-Todd
Date: Fri Jul 11 2008 21:06
Author: C, Matthew
Subject: Re: #2 oppressive and humiliating "
When I worked at a intensive English language school, I often had students who craved more drill than we offered. Some would purchase Raymond Murphy's Grammar in Use texts on their own. The underground movement got so popular that we built an elective around the text. At the time, my Academic Director argued that at least that way we could insure that they had some communication based activities along with the grammar.
-Topic: History of Educational Testing
Forum: History of Educational Testing
Date: Sun Jul 06 2008 13:57
Author: H, Gregory
Subject: History of Testing "Hello Classmates,
I just read the Power Point about the History of testing and it scares me. It seems that our whole approach to achievement testing in the U.S.A. has imitated experiments and practices of people who we deemed were "credible experts". What´s up with the craniometry test to determine intelligence? It seems a little troubling to me that the "IQ" test came from a method Alfred Binet developed to find out the causes why French school children where not meeting curriculum standards. Apparently no one knows how that test can be equated to a true representation of intelligence.
After reading over the 8th grade proficiency exam from a Kansas school, I wonder if this should be more like aCommunityy College proficiency exam nowdays. The questions are almost all based on theory or rote knowledge.
This article gets me a bit riled up, which I believe is the point.
Are achievement tests some means of control? Are standardized tests some means of social domination?
I think we should try to find out a way to test achievement based upon real life situations and performance of tasks. It seems like almost all tests are based on theory or are out of context for the people taking the tests.
Date: Thu Jul 10 2008 14:36
Author: R, Joyce
Subject: Re: Zappardino and the History of Educational Testing "
I'd like to be candid and say that I never thought studying the history of assessment would be so intersting. (Sorry, Todd) The good news is that this is intriguing.
Historically, I was unaware of the newnesss of the study of the mind psychology" and so aligining Binet's efforts and the entree into scientic methods of measuring elements of the mind helped me to understand how the scientific and lay communities could be awed and motivated to accept answers about the unknown elements of the mind. Aren't we all tempted to take alittle test to find out more about ourselves?
The idea that testing is a socially imbedded activity is important for us to remember as instructors who interface cross-culturally. The cloak of science that covered those early researchers political agendas may well seem obvious to some cultures represented by our students i.e. they may discern that we have motives beyond their benefit in our obsession with formal testing.
Zappardino seems to refer to the "faith" we have/ had in the numbers yielded from testing. I couldn't help but think how often we say "numbers don't lie". We sort of exalt numbers/statistics and the interpretations. Maybe I'm carrying this too far, but there is a scripture that says, "God is not a man that he should lie." So for me, this "numbers don't lie" thing puts numeric results on a pretty high level.
I'm in the dark about how one changes the culture of standardized testing, but I don't want to be a gatekeeper that holds folks at bay from educational or other opportunity, helping them "settle" and come to terms with a lesser view of themselves and their potential. I want to remember the idea of testing being culturally bound, even my own created tests.
Culturally speaking, I have one small anecdote: When we moved to MN from CA our youngest daughter was given a kindergarten readiness test. She was asked to name five jor six things you can do with water. She replied, "Drink it, swim in it, wash dishes." The tester smiled condescendingly, "How about skate on it? When it snows, make snowmen?" later she told me that I might need to read more to my child to help her think in broader contexts. WHAT? Hey, we lived on the beach! We drank it, swam in it and washed up! - Joyce
-Topic: Other Module 2 Issues
Forum: Other Module 2 Issues
Date: Thu Jul 10 2008 00:11
Author: M, Sal
Subject: Good tip for assessment from O'Malley "
I'm still finishing the reading for Module 2, but I wanted to comment on one particular advice O'Malley gave. It was on "Designing Authentic Assessments" (Ch. 2 pp. 17-18). I live in a small college town (5,000 pop) here in Morris, MN and thought the tip on "conducting professional development..." would be very useful here. I plan to contact some professors at the local campus and ask for their advice on planning authentic assessment. Very informative so far in this reading assignment!
Date: Thu Jul 10 2008 22:40
Author: M, Sal
Subject: The importance of meeting the students' needs '
Yes, reading Chapter 1 "Assessment as Information Gathering" by Kathleen M. Bailey was very helpful in the part of meeting the students' needs (pp.5-6). We are sometimes given guidelines on what to teach. However, we also need to consider the students' needs, which we'll know once we know the information about them as quoted by Diane's (teacher) dialogue with Kathi (author giving advice). I see this as very true when I teach ESL to local migrant "dairy farmers" compared to other students with a different profession/job. What I've learned from teaching ESL as a volunteer in the last 4 years, the students seem more interested on learning terminology relating to their workplace. They "want" to learn English to improve their work situation. As supposed to a learner that does not work in the same occupation/field or even as a homemaker. "
Date: Fri Jul 11 2008 10:15
Author: M, Sal
Subject: "Massive Systemic ignorance and blind reliance on standardized est scores..."... "
Wow, the quote above I partly took from Bailey's response on Ellie's (teacher w/three titles in a California school) response to the "school system" she was part of just summarizes one of many stories out there! Reading this (Chapter #3-Conflicting Purposes of Assessment) really got my attention and brought some personal experiences (as a student in grade school and teacher in ESL as an adult). In "grade school", I was close to being sent back because of my "learning disabilities". My parents immigrated here from the Philippines and never taught me their native language (Tagalog). They were afraid of me and my sibling being "left out" in our English-speaking dominant "new home country" (USA). However, my parents would speak Tagalog to each other in the home and their "English" (spoken differently in the Philippies, like Britain somewhat) was a little different. Thus, going to school, I would have trouble-another factor was their "passive" cultural life-style. English grammar was never every my strong quality (math was) growing-up! Anyways, going back to being a "current ESL teacher" (volunteer), I can relate to some of my students (both adult and non-adult). When I tell people in the small rural-college town community (Morris, Minnesota) that some of my students have "college degrees", they get suprised! One of my students (grown-up adult) was a "migrant worker" from Mexico, who got a college degree as a veterinarian . He came to our class because he just wanted to learn English! Just reading the story of Ellie makes me wonder how many school systems has "mis judged" or labeled students with a "learning disability" due to their language being "Non-English"! Wow, great story!
Forum: Other Module 2 Issues
Date: Fri Jul 11 2008 17:43
Author: M, Sal
Subject: Contrasting Pairs of Concepts-examples?
"
Hi all,
I really haven't had that much experience with creating my own testing and scoring of ELL or ESL Students. I was wondering if anybody has and willing to share their experiences relating to the "three contrasting pairs of concepts" Bailey shares in Chapter 6?
Forum: Other Module 2 Issues
Date: Fri Jul 11 2008 18:48
Author: M, Jeannine
Subject: Re: Contrasting Pairs of Concepts-examples? " One of our English department faculty, Susan Bosher, has done a lot of work with multiple choice testing with ELL health careers students. She uses language simplification techniques. I have one of her journal articles on my other computer. I'll find it and send it as an attachment. In the past I've taken my exams to Susan and had her review the questions. It is extremely helpful. I've also found that the modifications I make based on her suggestions make the test much more understandable for all students- not just English learners. Writing exams, as you probably know, is really hard. JMH "
Author: F, Patricia
Subject: Re: Statement #4 How well do I know my students? "I also agree with the statement that we can't be everywhere at all times. Even a poorly designed test can give us some information. Say we hand the student a test during registration on the first day that asks them to write their grocery list from last week rather than their name, address, etc. That is neither reliable nor valid. If the student is unable to write one item down on the paper, however, we can take and educated guess that they may not be able to read the instructions or may not be able to write the answers.
Greg, here is a prime example of your statement on being mislead by a student's proficiency in one area. I had a 50 something man from Mexico that came to class one evening. He talked with all the instructors, trading stories and information for more than 15 minutes at a level I would have said was close to a native speaker of English. When we handed him the registration forms he informed us he could not read or write in English or Spanish. That was during my first semester of teaching and it was a lesson I will never forget.
Author: L, Anne
Subject: 1-4 thoughts
"
...2- I tend to go back and forth on this one. If the assessments are relevant and address the needs of the students, then I think assessments are not oppressive and humiliating. However, “throwing” someone into an assessment who has never tested before is oppressive and humiliating.
Week #3
I. The Four Skills (Listening, Speaking, Reading, & Writing) and Hierarchical Components of Language
A. Syntax and Lexicon
1. syntax (wikipedia)-"study of the principles and rules for constructing sentences in natural languages"
2. Lexicon (wikipedia)-"its vocabulary, including its words and expressions. More formally, it is a language's inventory of lexemes."
B. Morphology (wikipedia)-"field of linguistics that studies the internal structure of words. (Words as units in the lexicon are the subject matter of lexicology.) While words are generally accepted as being (with clitics) the smallest units of syntax, it is clear that in most (if not all) languages, words can be related to other words by rules. For example, English speakers recognize that the words dog, dogs, and dog-catcher are closely related. English speakers recognize these relations from their tacit knowledge of the rules of word-formation in English. They intuit that dog is to dogs as cat is to cats; similarly, dog is to dog-catcher as dish is to dishwasher. The rules understood by the speaker reflect specific patterns (or regularities) in the way words are formed from smaller units and how those smaller units interact in speech. In this way, morphology is the branch of linguistics that studies patterns of word-formation within and across languages, and attempts to formulate rules that model the knowledge of the speakers of those languages..."
C. Phonetics and Phonology
1. Phonetics (wikipedia)-"the study of the physical sounds of human speech. It is concerned with the physical properties of speech sounds (phones), and the processes of their physiological production, auditory reception, and neurophysiological perception."
2. Phonology- (wikipedia)-"is a subfield of linguistics which studies the sound system of a specific language or set of languages. Whereas phonetics is about the physical production and perception of the sounds of speech, phonology describes the way sounds function within a given language or across languages."
Formative: "Formative Evaluation is a bit more complex than summative evaluation. It is done with a small group of people to "test run" various aspects of instructional materials. For example, you might ask a friend to look over your web pages to see if they are graphically pleasing, if there are errors you've missed, if it has navigational problems. It's like having someone look over your shoulder during the development phase to help you catch things that you miss, but a fresh set of eye might not. At times, you might need to have this help from a target audience. For example, if you're designing learning materials for third graders, you should have a third grader as part of your Formative Evaluation."
Summative: "Summative evaluation provides information on the product's efficacy ( it's ability to do what it was designed to do). For example, did the learners learn what they were supposed to learn after using the instructional module. In a sense, it lets the learner know "how they did," but more importantly, by looking at how the learner's did, it helps you know whether the product teaches what it is supposed to teach."
Summative evaluation is typically quantitative, using numeric scores or letter grades to assess learner achievement.
"..Other research studies found that teachers’ perceptions
of content standards and the professional development
provided for them play a strong role in the successful
implementation of standards. Specific professional development
about standards-based curriculum and assessment
contributed to better student math achievement in
California, according to a self-report of 1,000 K-12 teachers
(Cohen & Hill, 2000). Case studies and descriptions of professional
development to support standards-based education
in K-12 document a wide range of activities used by
school districts to train teachers, including workshops and
conferences, institutes, observations of expert teachers,
mentoring, study groups, grade-level meetings with other
teachers, site-based inquiry seminars, training and support
to use new curricula and standards, and certification training
(Bye, 2004; Dutro, Fisk,..-pp 5 of 6
"...Research on K-12 standardsbased
education shows that student progress depends to
a great extent on teacher professional development and
commitment to the standards..."-pp 5 of 6
Resources: The Splendid ESOL Web, Pima College Adult Education National Institute for Literacy’s Adult Education Content
Standards Discussion List Mid-continent Research for Education and Learning Standards, reporting, outcomes, accountability and
teacher credentials - What’s online
Assignment:
Compare-
-CASAS
-EFF (standards) "By identifying four categories of generative skills, EFF broadened the range of skills adult literacy and basic skills basic programs are typically expected to cover. These skills include strong reading, writing, and math skills; they include the skills we need to communicate and work well with others; to solve problems and to keep up with change. These categories of skills (see side bar) include those we traditionally think of as interpersonal skills, and those decision-making and learning skills we traditionally talk about as "higher order" or critical thinking. Our goal in proposing this range of standards and in framing them as we did was to shift the focus of adult literacy and basic skills instruction and assessment away from a decontextualized skills-based curriculum toward a contextualized, practice-based curriculum that was better matched to and firmly grounded in learners' own purposes for returning to schools." Four Categories of EFF Skills: Communication Skills, Decision-Making Skills, Interpersonal Skills , and Lifelong Learning Skills
-Massachussetts, Curriculum Framework for ESOL ...Teachers and counselors need to meet with learners early on in the program in order to identify students' most urgent needs for using English...smaller immigrant groups may have a pressing need to learn how to communicate with doctors or shop owners in English;-pp 9 ...For the language classroom this implies that students will benefit from practice in listening, speaking, reading, and writing that takes into account the kinds of communication that happen in different contexts outside the classroom.."-pp 10 "..Not all students will master a skill before the class moves on, but they may master it later, as the teacher focuses on a different context or topic. This is an especially important concept as teachers try to address individual learner needs in programs that are multilevel and/or open entry.
Teachers should be aware that students try to make sense of a new language and construct rules of how English works, often based on how their home language works; this concept is referred to as “interlanguage”...-pp11 "As much as possible, classroom materials should reflect the diversity of our students and those of the range of groups who live in the US without oversimplifying and stereotyping.
As students learn about American culture, their own culture needs to be validated. They should understand that they don't have to lose the personal identity and values they came here with and change themselves into a new “American” person..-PP 13 "..Teachers should also encourage learners to take advantage of learning opportunities outside the classroom. Where possible, instructors should plan class activities or assign homework that give students practice in learning on their own. These could include “eavesdropping” on the conversations of native speakers, by talking to people at work or their children's school, by watching TV or listening to the radio, or by reading magazines and newspapers..."-pp 14
# 6 standards look like in more detail at the six proficiency levels. (pp 17-
1 & 2. "The Listening Strand and the Speaking Strand:
These two strands include the skills that focus on developing fluency and making and gaining meaning in oral communication...
3. "The Reading Strand:
This strand includes the skills necessary to interpret printed material, including charts, graphs, schedules, and environmental print. Those skills include symbol mastery, phonological awareness, decoding, word recognition, word analysis, and comprehension.
4. "The Writing Strand:
This strand includes the skills necessary for both physical and intellectual mastery of written communication. Arenas of competence include language structure and mechanics, organization, and fluency. Development of every writer’s unique and personal “voice” is also valued."
5. "The Intercultural Knowledge and Skills Strand:
..The focus in this strand is on developing an awareness of cultural differences and attitudes and in developing the skills and knowledge needed to function in a culturally diverse society..."
6. "The Navigating Systems Strand:
..This includes knowing what opportunities exist and acting in accordance with both their rights and responsibilities within a particular system. It includes knowing about both mainstream systems (e.g. housing) and resources that are available to them in advocating within those systems (e.g. tenants' rights groups). Some systems impose problems or barriers on students, some systems (e.g. libraries) are opportunities, and many systems are a combination of barrier and opportunity.
*needs base
*referred by Joyce R. (7/14/08) "..The system uses the Accuplacer placement tests and set cut scores for placement in transfer-credit-bearing reading, writing and math classes. As you might guess, the placement criterion for reading is an Accuplacer reading sub-test score and for math classes is a Accuplacer math sub-test score. For writing, however, the review committee found the Accuplacer reading subtest score to be a better predictor of success in writing classes than the Accuplacer language subtest score! The language subtest is a multiple-guess type format that focuses on "knowledge" of writing components. The Accuplacer test suite also includes a "written essay" sub-test that, somewhat amazingly is scored by the computer. My understanding is that while some faculty found the written essay to be a better predictor of success in writing classes, there was resistance to mandated use of it because it takes longer to administer and because some faculty are still skeptical of the idea that a "computer" could score a writing sample validly and reliably. (Note that the reliability and validity of the computer scoring is well-established. I haven't had a chance to learn about how the scoring algorythm works, e.g. sentence length, word use, etc., but am defintely curious to find out.)
Anyway, point is that a reading test being a better predictor of writing class performance than a "writing" test suggests that the tests are measuring "writing" pretty indirectly...."-Todd W. (7/17/08 on Blackboard)
*referred by Patricia "Patti" F. & Christina (7/14/08)
DISCUSSION BOARD:
-Topic: 3.1 Adult ESL Instruction Content Domain
Date: Mon Jul 14 2008 17:11
Author: M, Sal
Subject: Re: CASAS "
Yes, we use the CASAS here (Morris Literacy Project-Morris, Minnesota) through the state. I personally feel it's been an "ok" measuring tool. We give it (entry) in the beginning and ending (exit) to see their progression. However, their conversation skills better tells us how they progress.
Just a little note: From my years giving it to our students, the "migrant workers" from Mexico always get the month/date/year (e.g. 7/13/08) question wrong. In Mexico (day/month/year-13/7/08), the order is different. This shows one example how CASAS can be "non-culturally friendly".
Date: Mon Jul 14 2008 23:35
Author: M, Sal
Subject: Re: CASAS-Massachussets (don't like the name-lol)would be my choice because... "
yes...in addition I feel the "Massachusetts, Curriculum Framework" would be the most useful of the 3 (CASAS, EFF, ..). Besides the title "Curriculum" ("curriculum based tests") and it being named after a state (sport dominate and take MN players-"KG", "Big Pappy", "Moss", etc..) I'm not fond of-j/k!-this would be my choice! It seems to have more of a cultural focus (see #5 Strand-The Intercultural Knowledge and Skills Strand), which would help many of my students (mostly migrant workers) in Morris, MN. Also, it serves ("needs base analysis of students") the "needs of the students" (see #6 The Navigating Systems Strand), which many come with a motivation to learn English for a particular reason (e.g. medical issues as was well put with stories-quotes in this particular "set"). This makes the CASAS look bad, which the Minnesota Literacy Council should look into implementing!
Author: Wagner, Todd
Subject: Re: functioning within culture
"There are three "models" for ABE/ESL programming in the use: K-12 system based, community college based and workforce system based. Each has strengths and weaknesses. For example, three big strengths in Minnesota's model are: substantial state funding ($6 state funds for everyone $1 federal funds, which is among the highest ratios in the country) which results in part from every school district in the state receiving funds (easier to get legislative agreement as funds go to all districts); integration of the volunteer/community-based organization programming into the system, i.e. volunteered based programming generates funding (in most states the literacy-council based system typically operates independently of the state and federally funded system); and third the many "points of entry" to the system that the first two strengths yield.
One weakness of the Minnesota system, which we are addressing with a three-year transition initiative (which is yielding a great deal of success) is that we have not been as well coordinated with the higher education system as would be ideal. This has been particularly problematic in ESL programming as some colleges have established several "levels" of ESL programming. In those programs, students may use up much of their financial aid progressing through the ESL levels and then the developmental education classes. Students are often attracted to those programs as they like the idea that they are going to college. We are, in part, addresses this issue through our transition initiative by promoted location of ABE programming on as many campuses as possible.
Anyway, after that long digression, my sense is that in Minnesota the vast majority of ESL students taking paying for ESL classes at colleges are there to "go to college/tech college" and not just for ESL classes. Note that, in states where the community college system hosts ABE/ESL programming students may be attending for ESL, GED prep, adult diploma only and not planning to continue with college course work.
-Todd
Subject: Re: functioning within culture "
I hear that so much from my adults that they are losing thier culture and that the kids don't care, all they want to be is American. I just keep telling them that they can still keep and practice their family traditions at home and that maybe someday their kids will realize the importance of there heritage, so try to be patient:0) My ansestors came from Germany about 5 generations ago, but we still celebrate Dec. 6th and put our shoes out for St. Nick. This is something that I continue with my children. I think that explaining tradtions in culture is important to know the why behind it when you teach or talk about it. -Anne
"Subject: Re: Hard Choices "
Overwhelmed and mind boogling....I asked our ESL coordinator how students are assessed for admission. She listed a battery of tests: Michigan English language battery (MELAB), MELICET, listening testing, and essay testing. Our other campus uses COMPASS, CELSA and CELT. I'll have to look up each of these because I've never heard of them before! She is apparently the only reader of the essay which she grades holistically. Students take 2 English classes for non-Native speakers. Interestingly, the students who struggle most are often those who were never assessed because they were in the US school system for 8+ years. These students have 'fossilized speaking/ writing errors' that become apparent in college level work. I love that term- fossilized."-Jeanine M.
Subject: Re: Hard Choices
"
My take is that, really, no one has this figured out. As a field we swing between doing no testing and throwing every test we can think of at students as Jeannine describes. I think this occurs, in part, because tests and testing are a bit of a "black box" to most educators. We don't really understand them and so sometimes fear their use has a negative impact on programming and at other times believe they have an almost magical ability to place students, determine success, etc.
One example of this has occured in relation to our post-secondary transitions (ABE/ESL to college) initiative here in Minnesota. Staff from programs that have complained for years that the testing required, e.g. CASAS for NRS purposes tells us nothing about how students or their programs are doing are now busy teaching to the Accuplacer. And I now find myself pointing out that doing well on the Accuplacer does not necessarily mean that a student will do well in credit-level college classes. The Accuplacer is a predictor tests like the SAT or ACT and, in all three cases, they are only crude predictors. What the programs should ideally be focusing on is articulating instruction with the college courses that students will be taking to help ensure they are prepared to function successfully in class.
I think we all need to be as clear and intentional as possible when developing, selecting, using and interpreting results from tests. The steps required for the class project attempt to provide a model for doing so.-
Todd
Subject: Re: Hard Choices
"
Joyce,
Each ABE consortium in Minnesota received funding for use in implementing the transition initiative this year. There are 53 consortia in Minnesota and some are very large, e.g. the Arrowhead Economic Opportunity Agency consortium covers the area east and north of a line drawn from Duluth to Grand Rapids to International Falls. Most consortia include multiple, in some cases dozens, of program sites/centers and not all sites will be implementing the transition initiative or providing tranistion-related services.
Regarding articulation, however, we have many programs around the state that have already developed or are working on developing various kinds of "pre" programming, e.g. pre-Certified Nurse Assistant, pre-Welding, etc. We also have several programs that have implemented a college prep program called Mindquest Academy with two of those programs co-located at MCTC in Minneapolis and South Central in Mankato. One of the goals of our professional development program, which is called ATLAS and is hosted at Hamline University, for next year is to work on providing professional development opportunities related to articulation. My guess is that most of the public two-year colleges in Minnesota are either collaborating with an ABE program already or have at least been approached by one or more ABE consortia in the past year regarding collaborating.
Mindquest is at http://www.mindquestacademy.org/
ATLAS is at http://www.hamline.edu/gse/academics/centers/sltl/atlas/index.html
ATLAS' transition info is at http://www.hamline.edu/gse/academics/centers/sltl/atlas/transitions.html .
For some examples of "pre" programs http://mnabe.themlc.org/Promising_Practices.html.
-
Todd
Subject: Re: functioning within culture
"
This conversation on acculturation is very interesting. I work on an NIH grant working with Somali women and girls. The concerns about US culture, particularly how it affects their sons is a concern repeated often in our focus groups. Some women are willing to send their sons back to Somalia rather than to lose them to US culture. The other issue, of course, is language. Many children out pace their parents in English language skills, resulting in a power shift in the family structure and function. Have you ever read "The Middle of Everywhere" by Mary Pipher? She give a wonderful language-related example of a young man who tells his parents (who don't speak English well) that a black leather coat is part of his school uniform. The parents use all their money to buy that for him.
By the way, the belief that past generations of immigrants 'assimulated' into US society easier and more willingly than current immigrants is largely a myth, particularly in Minnesota. Many immigrant groups lived in urban enclaves or farming communities and self-segregated themselves by language and/ or religion. Look at the Catholic churches in St. Paul- St. Francis de Sales was German, St. James: Irish, St. Stan's: Czech and each resides within one mile of the other near W7th in St. Paul..."-Jeannine M.
Three Challenges in English Language Assessment, html format & pdf format by Carol A. Chapelle & Joan Jamieson "..Terms such as “placement,”
“proficiency,” and
“achievement” might be used
in response to a question about
what a test is supposed to be
used for. These terms may work
for the public, but teachers
need a more systematic,
thorough, and accurate way of
considering the purpose of an
English Language assessment
because the purpose of the
assessment is critical for
choosing or developing a good
test. We think of test purpose
as consisting of three
interrelated concepts:
• One way to look at test
purpose is to consider the
inference that test users
want to make on the basis
of test scores. For example,
we designed Longman
English Assessment to allow
test users to make an
inference about examinees’
English Language reading
comprehension, listening
comprehension, and
knowledge of written
structures. Making an
inference about these three
aspects of English ability
was one way of looking at
the test purpose.
• A second way is to
consider the use of the test
results. Typically, a test is
given because a decision
has to be made about
examinees, for example,
their readiness to study at a
university, their eligibility
for a job, or the grade that
hey should receive in a
course. Longman English
Assessment is not intended
for any of these relatively
high-stakes test uses;
instead, it is intended to
give learners an idea of
their level of English
ability and to offer level-
appropriate advice for
improving their English.
• The third aspect of test
purpose is any other effect
that the test is intended to
have, such as serving as a
motivating, interesting, and
informative experience for
learners."
Final Project
Design of an Assessment Instrument: "Real-Life Skills"
July 17th of 2008
CONTEXT
This assessment is designed for classroom settings to help student learn "daily-living skills (e.g. intro conversations, grocery shopping, and other day-to-day social activities in the community). The "Real-Life English workbooks" (see Steck-Vaughn Company or amazon.com) has different levels, which uses various community "real-life" settings. Students want to be better "active-participants" in their workplace, home, and community while they are residents of this small rural college community (10,000+ population in the Stevens County area we serve). Thus, teacher too want to be "better" in working with each individual students to achieve their goals to succeed-not only in America, but also when they go back to their "home country". This assessment will encourage all three stakeholders (students, teachers, and program) through these results in various ways of performance.
SETTING
-Students Profile:
Many of our students (Mexico, Brazil, and Eastern Europen bloc nations) in the Morris Literacy Project are "migrant famers" that work in the surrounding area farms doing various work (e.g. dairy, hog, and typical field work). All of our students have been adults (late teens to the elderly); however, we currently have a student is only a teen. They like to come in groups (Spanish, Portuguese, Russian, etc..) , so the language barrier hasn't been a challenge. However, we they don't come in groups, they somehow work cohesively together.
-Audience:
The results would be very helpful for teachers (current and future volunteers) on how to improve our approach in teaching this particular topic ("real-life" skills). This may also help our continued new volunteers (mostly new in-coming college students) more interested in our on-going program (Morris Literacy Project). Knowing "what works" and "what doesn't work" (our approach in teaching) may help retain future volunteer teachers and more importantly-students that come to check out how we do class.
PURPOSE:
The purpose of this assessment is to see how students are improving in the "four-different
area skills" (Listening, Speaking, Reading, & Writing). Thus, to help them feel more confident in applying them in their "real-life" settings (e.g. workplace, home, community, etc..). We hope this would be given as both a summative and fromative placement test to inform both student and teacher.
QUESTIONS:
Several questions will come up from this assessment-1: Are the student's skills helping them in their various workplace, home, and/or commmunity settings? Are we consistently retaining our number of volunteer teachers and students in a certain period of time? Do they demonstrate a degree of progress to move to the next level of books of "Real-Life"?
Week #4
4.1
Terms
Itemized Response Theory: Wikipedia
Chapter #1-Item Calibration & Ability Assessment, from creative-wisdom (multi-media)
-Guttman Scale, Wikipedia "..if they can be ranked in some order so that, for a rational respondent, the response pattern can be captured by a single index on that ordered scale. In other words, on a Guttman scale, items are arranged in an order so that an individual who agrees with a particular item also agrees with items of lower rank-order. For example, a series of items could be (1) "I am willing to be near ice cream"; (2) "I am willing to smell ice cream"; (3) "I am willing to eat ice cream"; and (4) "I love to eat ice cream". Agreement with any one item implies agreement with the lower-order items..."
=>Multiple Choice Tests: Test scoring and analysis
Christina Ballantyne (tlc.murdoch.edu.au) tentative student proficiency-portion of correct answers for each student tentative item difficulty-the pass rate for each item convergeance-"there will be some differences int he model.."
Chapter #2-Item Characteristic Curve-"the relationships between the probabilities and the skill levels"
Related Sites: Assessment Issues: "..The ICC is a plot of Probability that the Item will be answered correctly against Ability. The shape of the ICC reflects the influence of the three factors:
* Increasing the difficulty of an item causes the curve to shift right - as candidates need to be more able to have the same chance of passing.
* Increasing the discrimination of an item causes the gradient of the curve to increase. Candidates below a given ability are less likely to answer correctly, whilst candidates above a given ability are more likely to answer correctly.
* Increasing the chance raises the baseline of the curve.
This simple simulation allows the user to investigate the factors governing the shape of the Item Characteristic Curve. All three well known IRT models are represented (referred to as IRT1, IRT2 and IRT3) and Item Characteristic Curves can be super-imposed on one another to see how they relate.''
IRT Model (ony 1 parameter=item selection: A&B-2 Parameter; A, B, & G-3 Parameter=diagnosis parameter=2/3 parameters) "B parameter"/threshold parameter-item difficulty "A parameter"-item discrimination "G parameter"-item guessing Lord's Paradox-a phenomenon of the probability of giving to the correct answer to the blue is .55 and red item is more difficult intercept
Chapter #3-Item Information Function item information function-"mathematical way to know how much info ICC provides, the more precision you have in estimating a parameter->the more you know of the value of that parameter Item information function of the 2PL model, from metheval.uni-jena.de
Chapter #4-Test Information Function test information function-identifying information for entire exam alternate forms-having different items with different levels form balancing-swapping items (e.g. easy, difficult, and hard) between the item pool and the forms is needed
Chapter #5-Item Person Map odd ratio-ratio of non-desired events (Q)/divided by desired events (P) or Q/P logit-log (odd ratio)
easy if below zero (e.g. -.7) item person map-item (test question itself-easy, average, difficult) and person (item proficiency value) can be viewed simultaneously
Q: odd of 1/6 passing?
A: .20
Chapter #6-Misfit misfit-about the response pattern (e.g. European history question accidentally added in a "American History Exam"); when there is a huge discrepancy (model and data don't fit with each other) residuals-discrepancies Chi-square fit-a statistical approach for the most common test of goodnesss (correct & incorrect; categorized by the skill level
O-E 2 (squared)/ E [O-observed, E-expected]
#2 Approaches to detect misfit:
1-graphical method-model and data are superimposed
2-Chi-square test of goodness of fit-chi square divided by the degrees of freedom
Q: In regression we obtain a systematic pattern of residuals
A: F Residual Plots and Regression Assumptions, from stat.tamu.edu Degree of Freedom ""Degree of freedom" (df) is an "intimate stranger" to statistics students. Every quantitative-based research paper requires reporting of degrees of freedom associated with the test results such as "F(df1, df2)," yet very few people understand why it is essential to do so. Although the concept "degree of freedom" is taught in introductory statistics classes, many students learn the literal definition of this term rather than its deeper meaning. Failure to understand "degrees of freedom" has a side effect: Students and inexperienced researchers mis-interpret a "perfect-fitted" model or an "over-fitted" model as a good model. To rectify this situation, two approaches of illustrating df in terms of sample size and dimensionality are recommended. To enhance the pedagogy of this concept, a multimedia program written in Macromedia Flash (Macromedia, 2001) was developed. In the program, different aspects of the above concepts are visualized and interactive features (questions and answers) are added to encourage the learners to think about the deeper meanings of df."
df=n (# of obeservations)-r (# of necessary relationships)
"Here you will have the opportunity to learn the logic of CAT and see the calculations that go on behind the scenes. You can play with an actual CAT. We provide the items and the correct answers. You can try different scenarios and see what happens. You can pretend you are a high ability, average or low ability examinee. You can intentionally miss easy items. You can get items right that should be very hard for you..."
Samples: Computer Adaptive Test Prep," ..your resource for the computer adaptive exam software otherwise only available through full-service GRE® Test, GMAT® Exam, and Patent Bar Test Prep service providers." Computer-Adaptive Test (CAT) for GRE, from syvum.com "decides according to your response what question to present you next. It covers the same content and uses the same types of questions as the paper-based test..."
"..is a nonprofit international literacy organization based in Syracuse, NY, that was formed by the 2002 merger of Laubach Literacy International and Literacy Volunteers of America, Inc. ProLiteracy Worldwide is the oldest and largest nongovernmental literacy organization in the world. It achieves its mission, championing the life-changing benefits of literacy for adults and their families, by sponsoring educational programs that help adults and their families acquire the literacy practices and skills they need to function more effectively in their daily lives."
"..began in the 1990s, during the trend towards greater accountability for educational and employment programs. In 1993, through the Government Performance and Review Act (GPRA), all Federal agencies were required to develop strategic plans to demonstrate that they are reaching their goals..." Getting Started "..Below you will find the answers to many questions you may have about this website.."
-"Q. What online training does NRS offer?
A. We have designed online training courses for adult education program administrators that explain NRS requirements and provide guidance on how to improve the quality of NRS data collection. Courses are available on NRS-related topics including:"
-"Q. What is an NRS Webinar?
A. NRS Webinars are an opportunity for state directors of adult education and their staff to attend regular online training courses on topics pertinent to NRS including the revised NRS Implementation Guidelines and planning for a successful data system. These training courses are announced on the home page of the Web site and listed under the Calendar. Additionally, state directors are invited via e-mail several weeks prior to each event. All NRS Webinar materials are available on NRSWeb. " History "A collaborative effort was undertaken to develop the NRS during 1997-1998. After the passage of Workforce Investment Act (WIA), NRS became mandatory. Implementation of the NRS began in 1999 and continued to be refined over the next couple of years. In 2000, states began submitting their student and program outcomes to DAEL. Current NRS activities focus on perfecting the system to demonstrate program effectiveness and improve student outcomes." Q&A
-" Q. Where can I learn more about how the implementation of NRS works, such as how to improve the quality of my program data?
A. The NRSWeb site offers a variety of materials and resources to help program administrators and state staff learn about the NRS requirements and how to use data, improve data quality, monitor programs, and build a better data system. You can find these resources in two ways: 1) by topic through the main menu bar to find guides, training materials, webinars, and online training courses for specific subject matter, or 2) by visiting the Training & Activities page for specific types of resources. "
ONLINE DISCUSSION:
"My (Morris, MN) Experience with NRS Testing"
Experiecnce: "To be honest, I never really looked very closely on the results NRS gives to us. As a volunteer teacher, I never had an interest as long as we have been getting good numbers. I let my "boss/supervisor" deal with the results and he gives me his input. After taking this class, I'll probably start looking at the scores more closely now.
Program: "We use the CASAS, which we have trouble-challenges with the "pre-testing within the first 12 hours". Awhile back, when we met students for the first time, we would give them the CASAS test and they seem to be overwhelmed. Now, we decided to just wait after the first class visit or another better time (within the "first" 12 hours). That first day of class makes a huge impression on what this class will be like. Thus, this will impact their decision to return or not! Unfortunately, we sometimes never see (due to many factors-migrant work is "unpredictable") these students again and would never get the chance to give them the CASAS. At other situations, we do give them the CASAS and they would never come back (after 3-5+ more class days) and we will never have the chance to give the "post-test" to see how or if they ever improved to another level.
Teachers: "We (teachers) usually get our results in our regular regional meeting in Alexandria, Minnesota. I have been able to witness different reactions of various teachers from different sites (e.g. Melrose, Alexandria, etc..). The teachers share their perspective, which we all learn from each other. We get "lectured" at times to do "better" and "encouraged" to keep doing what we are doing too!
Students: "We really don't share the results from the NRS with our students, which I don't think they have an interest. However, they are more interested on how they do in the CASAS (pre and post testing)! They (us-teachers too!) get very excited when the move up in the "6 ESL levels"!"
Accountability: "Yes, the accountability or test results from NRS testing has motivated us teachers to do better for many personal reasons (e.g. keep the program going in this area for this county-wide community , job security for "paid-teachers", and want to see our local program and the region overall we represent to do well). It keeps us on our toes!
Comment on others' posts: "No, I don't think students are persisting long enough to take the post test for many reasons as I stated some above. For our area, we have many migrant workers that keep occupied with their work. Most of our students in the past have been dairy farm workers and they work long hours. They are sometimes too tired to drive to our class site after a long day (especially the long Minnesota winters). There are other "work labor" issues that I don't want to get into in this discussion! A positive example, we had a student (05'-06') from Bulgaria that worked 6 days a week. I was surprised he was able to attend our classes regularly because of his personal motivation (and other factors that I can go too long with) to learn English before going back to his own country (became part of the European Union (EU) the past year.
We can help our students achieve their goals by "improving", "improving, "improving", etc.. I don't think we'll ever reach perfection, but know that we can keep "changing for the better" (unless-"don't fixed it if it's not broken"-your particular program is doing well). Each student has their own personal goals, so I can't speak for all. In general in this area (rural farming community), many of our students are "migrant workers". Thus, they will be returning back home to use or not use what they learn. They come to learn ESL to just get enough to get through the time they are working here. Being in the "middle of no where", they come to "socialize" and "learn about the community" they'll be living in the time period they'll be working. However, we just got a new student (from Brazil) that is 14 years old and wants (parents encouraging him too) to learn ESL because he is going to become an American citizen. He doesn't want to be an "outcast" when he enters high school this fall.
Our GED students are the ones that we usually focus on helping get employment and other various goals."
Author: L, Danielle
Subject: Importance of NRS "
Even with what we view as negative consequences to having to test, I definitely believe we need national standards like NRS. Otherwise, each of us would just teach what we think is important without much data to back us up. As I mentioned elsewhere, I did teach without guidance for a semester, and, although I knew we were making progress, sometimes I felt like I was going around in circles. "
Author: H, Gregory
Subject: New kid on the block "
Hello Group 1 and all classmates,
I have to admit that I feel a little bit out of the loop with the current conversation. This is mostly new material for me. It is very interesting to hear of your experiences and opinions about NRS and these standardized testing programs. I have been living in El Salvador for the past 5 years and do not have any experience with what you are going through in the States.
In El Salvador, curriculum design is in vogue and the various English teaching institutions are left to themselves to set their own standards. We deal with some Cambridge tests in the bi-lingual high school where I teach. We also prepare more advanced ELL's to take the TOEFL test. We are starting to offer test preparation for the SAT test as well.
I would just like to make a few comments about my observations of the NRS functioning levels. There are 6 levels with the beginning level practically 0 proficiency in the target language. The next two levels are so close to each other, they are nearly indistinguishable. Low and High Intermediate in my humble opinion does not seem realistic to me. Why am I reacting in this way, I ask myself? To quote a descriptor phrase from the High Intermediate level functioning level under Listening and Speaking; "Can communicate basic survival needs with some help." At the High Intermediate level of a second language I thought this would be taken for granted. What I am getting at here is the arbitrariness of these functioning levels. Since I am not familiar with the assessment tests given to arrive at these levels I will shut my mouth. It just seems difficult to me to put human beings into these neat pre-packaged boxes with different bench mark code names.
"he National Institute for Literacy hosted this webcast on Tuesday, August 15, 2006 to discuss the results of the National Assessment of Adult Literacy (NAAL) 2003, specifically relating to Americans who tested in the Below Basic and Basic literacy levels....
The panelists included John Strucker, of the National Center for Adult Literacy and Learning, discussed basic skills; and Brian Bosworth, of the consulting firm FutureWorks, discussed implications for workforce programs.
WEBCAST NOTES:
-Sheila prose -continuous texts (paragraph) documents-non-continuous text (graphs, tables, stats, etc..) quantitative literacy-use numbers embedded in prose and documents
#3 major features:
1. language-bilingual?
2. supplemental assessment
3.oral reading assessment
3rd Reason:
67% of jobs created over the next decade required college degree
#4 literacy levels "..Next, although 13% of American adults spoke only Spanish or another language before starting school, these individuals accounted for 37% of the below basic population. .."
-Dr. John Stucker ".. I would be working with the other teachers and asking questions about how is our outreach? Are we really reaching these folks in our community? How is our teaching approaching going? How we are doing with the young people who are sometimes getting pushed out of high school?..
as Sheida pointed out, you see the highest percentage of people who report learning disabilities as well as other disabilities. And most learning disabilities involve reading even if they aren't labeled as reading disabilities. So these are people who have identified reading problems. Other studies we have done show that there is a higher percentage of their native US born people who have been in special education, have gotten extra help, whose teachers realize they needed extra help, and whose parents realized they needed it.."
And then the other thing about these folks is, again, it's remarkable how fast these ELLs have picked up English conversation. But don't let that fool you. If you actually get a chance to assess their reading skills in English, it turns out they are extremely idiomatic when they talk to you. If one person walked in and joined us with this panel, we think gosh! That person speaks great English but when they went to do embedded literacy tests, such as the NAAL, those tests were hard for them. Let's talk a little bit about the basic people. I think this is an extremely important population we don't want to neglect, because they are close to intermediate. And we know from other studies that intermediate is the point where you see big changes to the positive in income, civic participation, personal reading habits, all the things that embed literacy in our life and make it so meaningful and important...
...If we can identify their specific needs, their needs for fluency and that sort of literate, academic vocabulary, it may be that we can get them into the intermediate level faster and that opens up a lot of opportunities for them in terms of involvement in post-secondary and all the other things that we talked about..
-Brian Bosworth, Future-Works ".. Fourth point I want to make, most employers say they are as concerned or even more concerned about other workplace basic skills as they are about traditional measures of literacy and what do they mean by those other workplace skills? Technology skills or computer literacy is increasingly important to all employers. They identify career management and lifelong learning skills, and by this I mean the navigational skills associated with career management and lifelong learning. The employers identify, thinking critically, acting logically and solving problems as a critical basic skill. They identify finding and using information, knowing when to ask a question, knowing when you don't have the right information to answer that question and knowing where to go to get that information. They identify teamwork as a basic skill. The ability to work in groups with others cooperatively. They identify basic employability skills as well, sometimes called social skills, attendance, timeliness, work ethic, showing up sometimes is the most critical basic skill. Employers do identify math, reading, writing as important deficits in their workplace. But we find, through our survey work that employers are more likely to suggest some of these other basic workplace skills are as important or more important to them than the conventional measures of literacy.."
They would be contextualized in the language and the situations of the workplace. They would use work as a way to develop the workplace basic skills. They would be focused on a credential, a credential, which satisfies entrants to the next level of education and training....
Q&A:
-Brian '..NCSALL colleague, Steve Reeder out at Portland state working with other researchers developed a way that you can sort of do a synthetic NAALs on your county or community, based on local demographic data. And some of the other characteristics that occur both on the NAAL assessment, and the background questionnaire and that you can also gather about your local population. .."
-Sheila "...Sheida White: I think vocabulary is an area where ESL adults have particular difficulty in...because unlike native speakers of English who have the listening vocabulary, and so their problem is essentially decoding the word and then they can map their overall vocabulary to the text that is before them. Oftentimes, ESL adults do not have the overall vocabulary, so they have to learn that in addition to learning the...how to read the words and so it's a little bit more difficult for them to do that. So I would say vocabulary is a particular area where ESL adults can benefit from. .." National Assessment of Adult Literacy "By comparing results from 1992 and 2003, NAAL provides the first indicator in a decade of the nation's progress in adult literacy. NAAL also provides information on adults' literacy performance and related background characteristics to researchers, practitioners, policymakers, and the general public."
-Brian
Q: "We marketed these classes through various venues, but attendance has been too low to continue the classes. Any suggestions on what they can do?
A: Brian Bosworth: "Well, I think my quick suggestion and I hate to sound like a broken record is that offering the instruction in the workplace in partnership with the employer where most of these folks are for 40 hours a week, isn't a bad strategy in terms of securing participation and securing attendance." National Institute of Literacy, discussion list
-Literacy in War and Peace
Tom Sticht's Work in Adult Literacy Education
By Thomas Sticht (from nald.ca) "...In the winter of 1967 I got a call from a friend working in Monterey, California at a field unit of the Human Resources Research Office (HumRRO) of the George Washington University. He told me the military was joining President Johnson's War on Poverty by inducting 100, 000 new recruits a year who were low in reading ability. HumRRO thought it might be necessary to do more teaching by "show and tell" so that the New Standards Personnel (NSP, as they were called) could learn by listening rather than by reading. With my background working with blind students, HumRRO thought I would be a good person to do research on listening and reading abilities of the NSP..."
..In 1992, I was invited to London by what is now the Basic Skills Agency (BSA), to speak at its national conference. Princess Anne, the Patron of the BSA attended and sat through my presentation which emphasized the importance of getting "double duty dollars" through the intergenerational transfer of literacy from parents to their children. Newspapers reported that the Princess Royal found these ideas quite sensible. I worked with the BSA for several years on their national family literacy program.
In 1994 I was honored by being elected to the Reading Hall of Fame and in 1997 Timothy Shanahan and Susan Neuman, writing in the Reading Research Quarterly , selected the work colleagues and I did in the late 1960s and early 1970s at HumRRO as one of the thirteen most influential lines of literacy research since 1961. Reach Higher, America: Overcoming Crisis in the U.S. Workforce...
Related Sites: pdf format
Foreward "..Lee Kwan Yew, the first Prime Minister of Singapore and architect of its economic miracle over the past half century,
once reasoned that a healthy economy depended on four fundamental requirements: dependable electrical power,
clean potable water, world-class transportation, and an educated and trained workforce. In our current global
economy, we can no longer take these precepts for granted."
....., from national commission adult literacy
".. calls for a dramatically revamped service system with the
capacity to effectively serve 20 million adults annually by the year 2020..
Section A. FACING THE PRESENT
2. Our Treacherous Path:
...Americans should have been stunned when the National Assessment of Adult Literacy (NAAL),
released in 2005, revealed that a staggering 30 million American adults scored at “below
basic”—meaning they could perform no more than the most rudimentary literacy tasks.
Another 63 million adults could perform only simple, basic everyday literacy activities.
The NAAL findings are ominous because most good jobs require at least some education beyond
high school. The NAAL found that of the approximately 222 million adults aged 16 or older living
in households or prisons in the United States, some 93 million lack literacy at a level needed to
enroll in the postsecondary education or job training that current and future jobs require...
[International Scale of Top H.S. Graduates-US #11; Korea #1]
3. The New Landscape
[National Demographic of States with College Diplomas-Texas #1]
This is a tall order, but it is what the U.S. job market demands. The Bureau of Labor Statistics
forecasts that between 2004 and 2014, 24 of the 30 fastest growing occupations will require
workers with postsecondary education or training...
Lack of Teacher Certification and Benefits. (pp 13)
High-quality instruction is essential to
foster student retention and produce successful outcomes. But salaries for adult educators
are low, and many positions lack benefits. This limits the pool of potential instructors and leads
to high turnover.
While many adult education instructors are credentialed for K–12 teaching, few states require
that adult education teachers show mastery of the specialized knowledge and skills needed to
teach adults. Moreover, states have not established certification systems to ensure that all
instructors have this ability.Worse yet, the majority of adult educators nationwide are working
part-time, often as a second job. This is another clear sign that adult education is not considered
a serious professional enterprise.
-David J. Rosen "World Education, Boston, MA
International consultant in non-formal education to the government of the Philippines. Sponsored by the Asian Development Bank and the Royal Norwegian Government. Helped to develop national adult education system including elementary and secondary equivalency certificate alternatives. 2/95-3/95; 11/96-12/96; 6/97-7/97; 1/98-2/98; 7/98; and 2/2000.Project awarded UNESCO Literacy Project of the Year recognition in 2000. Quality Assurance Consultant on performance-based assessment, curriculum development, and teacher training, for school-based and out of school youth Integrated Production and Pest Management farmer field school project. July, 2004 - April 2005.
Consultant to the World Education Literacy Division on developing an intermediary role in Boston-based workforce development, 2004; conduct of scan of national adult literacy organizations for the National Institute for Literacy, April, 2005 - present; development of state Professional Development Distance Learning plan for the System for Adult Basic Education Support (SABES), November, 2005 - April, 2006.
ONLINE DISCUSSION:
Sal "After watching (webcast form) and then reading the transcript (document form) of this report, I'm remembered as a child of easily understanding picture stories (e.g. quantitative literacy-71 million/32 % performed) instead of just words (e.g. prose-60 million) & (e.g. documents-48 million). Despite their data results concluding the adults did more poorly in the quantitative (using numbers) section, I feel math or working with numbers is complicated for certain groups.
The high percentage of only one language (Spanish) population that did poorly wasn't a surprise from my experience working with many migrant workers from Mexico. Also, the same goes with the population that didn't finish high school because of my experience with GED students in our ABE program.
The "Adults Just Don't Know How Stupid They Are" discussion was somewhat interesting, which I thought giving an international scope brought some enlightenment. The data on the importance of "apprenticeships" shows the importance of early programs (e.g. "OJT"-On the Job Training; Internships, etc..) in high school age level at least to college age level. I feel there is that different level of "pride" on every individual that we already know everything that we need to know on various subjects or skills (e.g. driving).
I feel it's the responsibility of "all of us" (researchers and adults themselves) on determining who is right about whose literacy levels pose a "problem". Instead of "criticizing" each other, we need to "work together" as a global society in literacy. Ever heard of the quote, "What we got here is failure to communicate" (used in a rock song from one of my favorite 80's band-Guns N' Roses)? I feel as ESL teachers or potential ESL teachers, we have a BIG task-role in teaching literacy to the "world". Thus, bridging different nations of different cultures of many ethnic backgrounds. Literacy is one way to communicate to each other, which can solve many of our social problems (e.g. wars that are somewhat caused by "misunderstanding" of one another)
An example of "working together" that's in the process in the area I'm teaching ESL as a volunteer is with our local dairy farm. They employ over 250+ "Spanish-speaking" migrant workers (many from Mexico) and they took the initiative (after hearing from the community) to start a "Spanish language" program for their "English-speaking" American workers. The guy that teaches "Basic Spanish" to them just moved up here from the Twin Cities the beginning of this year. Ironically, his wife was an ELL/ESL student in our program. That is how we found about this! They plan to hire a teacher to teach "English" to their "Spanish-speaking" migrant workers down the road.
Lastly, I would like to comment on the "Ohio Portfolio System", which I feel is a very good idea. I've learned somewhat about doing portfolios for our ELL/ESL students in the past. However, it is sometimes hard to do one when we are challenged with "retention" rates of our students. This is another "on-going" subject to discuss about later! ""arbitrary" literacy skills"-"hand on learning experience ( Newgate school here in Minneapolis)-Todd W. ' Algebra skills may not be part of my everyday life but I think the ability to 'think' mathmatically enhances brain function and math is a language..."-Jeannie M. "...http://wdr.doleta.gov/SCANS/.... I believe SCANS was mentioned in one of our readings. At the workforce center, I had to make sure my students were achieving these skills. I developed a curriculum I called SCANS QUEST to help the supervisors and our student workers engage in tasks that supported SCANS.-Joyce "In 1990, the Secretary of Labor appointed a commission to determine the skills our young people need to succeed in the world of work. The commission's fundamental purpose was to encourage a high-performance economy characterized by high-skill, high-wage employment. Although the commission completed its work in 1992, its findings and recommendations continue to be a valuable source of information for individuals and organizations involved in education and workforce development."
4.4 CASAS
" They cover nine broad content areas:three-circles
* Basic Communication
* Community Resources
* Consumer Economics
* Health
* Employment
* Government and Law
* Math
* Learning and Thinking Skills
* Independent Living Competencies, pdf format (print!) "The CASAS Competencies identify more than 360 essential life skills that youth and adults need to be
functionally competent members of their community, their family, and the workforce..." Content Standards "Since its inception, CASAS has focused on teaching and assessing basic skills in contexts that are relevant and important to adult learners. CASAS has developed and continues to refine a highly formalized hierarchy of competencies, the application of basic skills that adults need to be fully functional and productive members of society." Quick Search=> (click here for login) "QuickSearch Online has many exciting features. A bold, colorful interface helps users to select from a wide variety of library searches. This year's edition allows "quick searches" for instructional materials by:
* Skill area: reading, math, listening, writing, speaking, critical thinking, and government and history
* A wide range of programs (including Civics, Family Literacy, Workplace)
* Specific CASAS assessment series, i.e. Life and Work, WLS, ECS
* Year of publication of instructional materials
QuickSearch Online can also search for materials that include informal classroom assessments.
The QuickSearch Online Tutorial is now available from the main menu, making it easier to learn how to use QuickSearch Online and to train new staff. Also, several reference tools appear in the Reports and Other Tools menu; for example, test competencies for every CASAS pre- and post-test. "
Tests Overview Life Skills "to help identify the basic skills in reading, math, and listening needed by individuals to function successfully in today's society. Using CASAS assessment instruments, agencies assess learners as they enter a program, place them into appropriate educational programs, assess learning gains, and certify attainment of learner and program goals. CASAS integrates valid and reliable assessment with a curriculum and instructional management system." Life & Work "s new and is different from other CASAS series in that it was developed based on the California ESL Model Standards, which are organized according to basic skills objectives, or "skills sets." All items in the Life and Work reading tests are correlated to reading objectives, as well as to life and work competencies. This correlation allows for analysis of the results with respect to reading skills sets, as well as in relation to language use contexts (competencies). Assessment items in the new series focus on short narrative passages at all levels, and follow immigrant families as they go about their life and work. Listening and Grammar tests for the Life and Work Series are currently in development." Workplace Speaking "s a new standardized speaking assessment, which assesses the speaking skills of adult ESL learners in a workplace context. It helps adult education programs determine to what extent an individual learner has the speaking skills to succeed on the job. Competencies assessed are clustered in three areas:
1. job skills and job information
2. social language
3. workplace safety and customer service" Functional Writing "provides teachers of ESL (English as a second language), ABE (adult basic education), and ASE (adult secondary education) at the adult or high school level with a means of assessing their students' writing skills in a functional, workplace, employability, and life skills context. It is adapted from the work of the CASAS Writing and High School Committees." ECS, WLS, Life and Work and Life Skills forms CASAS FAQ
ONLINE DISUCSSION: ..http://www.npr.org/templates/story/story.php?storyId=92611633 http://www.npr.org/templates/story/story.php?storyId=92663976.."-from "Elizabeth" W.
How Merit Pay Played Out In A Colo. School District by Larry Abramson "Teachers have long resisted this idea, but they are gradually warming up to it. Still, many questions surround the issue of merit pay, including whether teachers can be punished for the challenges facing students — especially if those students are low-income, learning disabled, or struggling to speak English..."
D.C. School District Proposes Merit Pay For Teachers "Forum: Other Module 4 Issues
Date: Wed Jul 23 2008 19:31
Author: Wagner, Todd
Subject: Re: Criterion referenced standardized tests?
Ken, I think the CASAS folks consider their assessments to be criterion referenced--they are designed to measure the CASAS competencies and the results are almost always evaluated against a criterion, e.g. did the student score above the cut score we've established to move up to the next level class? CASAS results are rarely used to compare students to each other and students individual scores are not typically interpreted in relationship to other student's scores.
The classic indicator of norm-referencing is reporting scores in percentile ranks. This approach is still very common, for example even in state testing systems that have established performance levels as criteria--in part because the public demand percentile ranks. (It's sort of like the case with NAAL performance levels. Parents don't really trust states to determine what constitutes a "proficient" or "advanced" level and have trouble identifying with results reported in that way. If they know the percentile rank, however, they at least know where their kids stands relative to other kids--critics would say even they still have no idea how those kids do overall--e.g. 90th percentile in Mississippi vs. Maine. (This reminds me that the National Assessment of Educational Progress NAEP and many/most state K-12 test, e.g. the MCAs in Minnesota have struggled with doing the same thing the NAAL has arguably done. They set high standards for literacy/achievement, the need for which they believe they can justify, and attempt to "push" eduational systems to empower students to meet those higher standards. They then run into a face validity problem with the public when large numbers of students, as with the NAAL, are identified as "below proficient." I was part of a standard, i.e. cut score setting, process for the first round of MCAs in Minnesota. The standard-setting process used was a modified Angoff procedure in which judges, in this case teaches, are shown items and asked to judge the probability that a "competent student" would get each right. Cut scores are then derived from those judgments across all judges and all items. When state agency staff and other policy advisors learned that the Angoff procedure had yielded a cut score the classified far more students as proficient than the most recent NAEP testing in Minnesota had, they threw out the Angoff results and simply chose a cut score that yielded the same proportion of proficient students on the state test as were identified by the NAEP test.
"
-Todd W. (professor/teacher)
Subject: Re: NRS - CASAS " I still feel assesssment exams are a 'necessary evil'. I wonder if the students described who wouldn't return if they knew they were being tested, don't actually have other issues, like test anxiety. In that respect, 'setting the stage' for testing (ie practice exams, talking about the exam, and giving feedback on the practice) might help diminish fears a bit. One of our faculty has started a support group and counseling for students with test anxiety. Students who enroll in her program show improved test taking skills and test scores. I suppose such a group isn't really feasible within most ESL programs- although involvement in an informal 'support group' might ultimately improve language skills as well."-Jeanne M. HOW BCSP SETS EXAMINATION PASSING SCORE "...The most commonly used procedure for setting passing scores on professional certification examinations is the Angoff procedure. Dr. Angoff was a psychometrician (a specialist in testing and measurement of knowledge, skills and abilities) who developed a method that is now widely accepted for certification examinations. Such examinations must have a clearly defined cut score or passing score.
Just like an instrument reading determines whether someone can enter a confined space without protection, candidates who meet or exceed the passing score on a professional examination pass the examination.
To establish the cut score, each member of a panel of experts must review independently every question on the examination and rate them. The rating for each question is "what portion of those candidates who just barely qualify for the examination will know the answer?" The ratings do not include those who are more than qualified through years of experience or advanced studies. In essence, the mean of all ratings of all panelists across all questions becomes the recommended passing score."
".. The primary project goal was to link overseas refugee camp training to refugee programs in the United States to facilitate refugees' English language learning and transition to employment and independence in the United States. REEP was selected to field test and refine three products and components of the project, including: (1) field testing of three versions of the Basic English Skills Tests (BEST) designed to measure general language ability; (2) testing and refinement of standardized Student Performance Levels, general descriptions by level of students' language ability; and (3) elaboration and specification of a set of curriculum guidelines entitled the "CORE Curriculum Guide," to be used as a reference guide for the development of U. S. based curricula. REEP benefitted from these tasks in the improvement of its methods and in the testing and achievement of its own goals of promoting refugee self-confidence, employment, and independence in the United States. (MSE)..
-1. CAL :Topics: Testing/Assessment '...CAL uses current best practices in test development to create its assessments, which are designed to help programs evaluate students and make instructional decisions. CAL’s assessments use authentic language and communicative tasks that allow examinees to demonstrate what they know and can do in English..." Student Performance Levels (SPL's)-"descriptions of English language proficiency levels for adult non-native speakers of English, were developed so that teachers in refugee camps in southeast Asia and those in ESL programs in the U.S. could communicate effectively about adult students' Enlgish skills.
-2. BEST Plus, developed by the Center for Applied Linguistics (CAL) "... is an individually administered face-to-face adaptive oral interview designed to assess the English language proficiency of adult English language learners.."
-3. FAQ
=>Online Activation
-4. Best Literacy, from cal '...tests reading and writing skills in authentic situations specifically geared for adult English language learners in the United States. BEST Literacy can be administered individually or to groups of examinees in one hour or less..."
-5. FAQ "..Is training required to administer BEST Literacy?
The Center for Applied Linguistics does not require training to administer BEST Literacy. However state agencies, program administrators, test administrators, and test scorers should thoroughly review the BEST Literacy Test Manual - 2008 Edition in order to become familiar with proper testing and scoring procedures.
" The purpose of ACCUPLACER tests is to provide you with useful information about your academic skills in math, English, and reading. The results of the assessment, in conjunction with your academic background, goals, and interests, are used by academic advisors and counselors to determine your course selection.
You can not "pass" or "fail" the placement tests, but it is very important that you do your very best on these tests so that you will have an accurate measure of your academic skills..." COMPASS® computer-adaptive college placement test helps educators (former ACT): " * quickly evaluate incoming students' skill levels in Reading, Writing Skills, Writing Essay, Math, and ESL
* place students in appropriate courses
* connect them to the resources they need to achieve academic success
Accuplacer ESL-MSCU Association of Classroom Teacher Testers-Combined English Language Skills Assessment) "combines three formats of cloze into one: rational, fixed ratio, and multiple choice. Either of the two forms of the test will place students into seven levels of proficiency from low begining to advanced plus. Each 45 minute test has 75 items with 4 choices. Reliabilities for C.E.L.S.A. 1 and C.E.L.S.A. 2 are .93,.94 and the forms correlate .90...
Each form has one beginning, one intermediate and one advanced passage which were developed as follows: First, for each passage rational choices were selected based on years of research developing the English-second-language Placement tests which conformed to curriculum and texts used in seven levels of English-second-language proficiency. Second, after these high discriminators were chosen for blanks, the text was written around the choices in a fixed ratio format. The resulting passages were given to experts to determine if the language was natural, authentic and representative of materials used at each level. Third, students took the fixed ratio blank passages. From their errors, distractors were chosen for a multiple choice format. Fourth, other students took the multiple choice tests; after analysis of the items, the poor items were removed. Originally, six forms of E.L.S.A. were published..." Yuba Community College District
DISCUSSION BOARD
Subject: Re: Criterion referenced standardized tests? "
Joyce, to date I think two things have been driving the cut-score setting processes in programs as diverse as NAEP, the NAAL and state testing programs. The first is the belief/perception that Americans are under-skilled or not enough Americans are highly skilled and that that represents a threat to the countries competetiveness. The second is the idea that setting high standards will drive systems to change. So far, however, that doesn't seem to be happening, e.g. funding for ABE has not increased, k-12 achievement has not increased, etc. So one criticism as with the NAAL is that the bar has been set so high it makes the results appear invalid and similar critiques have been made regarding NAEP and state testing programs. The belief is strong though. A state senator once said in response to our state director of ABE (Barry Shaffer) when he had testified that raising the cut scores for graduation tests would result in more students needing ABE services: "Mr. Shaffer you obviously don't understand the point of high standards. When we have high standards, everyone will graduate."-Todd
Submit a 1 – 2 page paper in which you reflect upon your personal experience with assessing adult second language skills in light of the concepts from the class. What assessment methods and techniques do you use in your program and classroom? How can some of the concepts discussed in class and in the readings help you understand what you are already doing or what you could do in the area of adult second language assessment? Integrate the following into your reflective statement:
The assessment methods and techniques we use in our program and classroom are in various forms. For standardized testing, we use the CASAS (for ELL, ESL, ESOL students) and TABE (for GED students).
1. Why and when are your students assessed?
As I stated in the "Discussion Board" (Module 4.2), we usually assess them within the first 12 hours of their class visit...
"We use the CASAS, which we have trouble-challenges with the "pre-testing within the first 12 hours". Awhile back, when we met students for the first time, we would give them the CASAS test and they seem to be overwhelmed. Now, we decided to just wait after the first class visit or another better time (within the "first" 12 hours). That first day of class makes a huge impression on what this class will be like. Thus, this will impact their decision to return or not! Unfortunately, we sometimes never see (due to many factors-migrant work is "unpredictable") these students again and would never get the chance to give them the CASAS. At other situations, we do give them the CASAS and they would never come back (after 3-5+ more class days) and we will never have the chance to give the "post-test" to see how or if they ever improved to another level."
The purpose of these test as a "local" literacy program is for our "overseers" within the state...
"..Accountability: "Yes, the accountability or test results from NRS testing has motivated us teachers to do better for many personal reasons (e.g. keep the program going in this area for this county-wide community , job security for "paid-teachers", and want to see our local program and the region overall we represent to do well). It keeps us on our toes! .."
For our GED students, we assess them in a different way or standard...
"Our GED students are the ones that we usually focus on helping get employment and other various goals."
Overall our ELL/ELL/ESOL students are assessed differently right from the start compared to our GED students. We have them fill a "goal" sheet, which our "local program" gets credit if they do reach their goal. For example, the common goal listed for our ESOL students is "Learn English". For our GED students, it's getting their "GED Diploma" and/or "getting employment". Then throughout the course of the term or year, we write their individual goals and accomplishments each class session they participate on a separate goal sheet. These are some of the assessment tools and techniques we have been trained by the "state" for our "local" ABE program. The purpose of this is to show their "progress" and to show the individual learner/student what he/she has done so far. Therefore, to hopefully motivate them to continue to strive to reach that goal we both came to agreement in the beginning of the term/year. This informal form of assessment helps us "test-givers" or "teachers" to keep record on what we've done and what to do to help this invidual learner "reach their goal(s)". Also, to serve as data for the state in case we get "evaluated" for funding purposes. Along with this information, we have all types of other records (e.g. homework) for the individual students in a "portfolio" type file folder.
2. What assessment tools and techniques have you used? What was your purpose in using them?
(see above)
3. How do the tools and techniques you have used relate to the desired outcomes of your course or program? How would you describe the actual outcome?
The tools and techniques we have used has met both desired and undesired outcomes of our course or program. We "try" to keep improving it as we "make mistakes" and "learn". To first share some positive outcomes, we've been able to help students (GED) get their GED Certificate and employment. We've seen some progess of some of our ESOL students. For example, when we first "restarted" this program back in 2004, we had a good-size group of "migrant workers" from Mexico. One particular student name "Tulio" was one of the most ambitious, which he would come regularly. I remember taking him out to a community setting and introduced him to some people to informally practice his English. He stopped coming to our class one day and we didn't know why. Well, we found out he became manager of a particular dairy farm site! I personally feel what helped him reach this particular position was "researching" on the internet on resources related to his work. He was a dairy farmer and I found this "dairy terminology" site that translated English to Spanish. I would end up sharing this site with future "migrant workers" that worked on this particular field. He still continues to have this position to this day and he ended up inviting his family-relatives to come work in this area.
Some of the "undesired" outcomes are students that never reach their "goal" that they stated in the beginning of the year/term. We have one student name "Manuel" that kept coming regularly and stopped for about a 1/2-1 year. With our local classroom environment, we sometimes get "very busy" and forget or don't have time to "follow-up" with some of these students. Well, Manuel ended up coming back and would come regularly for awhile. However, we didn't see any "progress" based on his conversations with us. We can pin point to many factors that causes this, but we certainly feel that we could've done much better with this individual student. Some of other "undesired" outcomes is uncertain because we get so many students in the beginning, but they never finish or "come back". We will sometimes never know the outcome!
4. How would you characterize the tools and techniques you used in terms of the testing concepts presented in class?
I feel we have been using the tools and techniques already, but we just didn't know "how to use them" the right way all the time. For example, in Module 4.3 when we were assigned to read the results of the National Assessment of Adult Literacy (NAAL) 2003. I was able to related to what Dr. John Stucker stated...
"And then the other thing about these folks is, again, it's remarkable how fast these ELLs have picked up English conversation. But don't let that fool you. If you actually get a chance to assess their reading skills in English, it turns out they are extremely idiomatic when they talk to you. If one person walked in and joined us with this panel, we think gosh! That person speaks great English but when they went to do embedded literacy tests, such as the NAAL, those tests were hard for them.."
I had one student from Bulgaria that quickly learned how to conversate in English. However, when we e-mail each other (still to this day to keep in touch), I feel like I'm e-mailing a "totally different" person from who I had good English conversations with in person in the past! I just feel like I should've focussed more on the "Writting Assessment" when I was teaching him years ago.
If I took this class prior to being a "volunteer" teacher 4 years ago, I feel I would've been better equipped on teaching our ESOL students. In fact, I feel the retention rate of our students and volunteers would be better. I guess it's better late then never. Now I know from this mistake has taught me to work on all three types of assessments or four different area skills (oral-speaking and/or listening, reading, and writing) for each student instead of just spending too much time in one particular area
Overall, I feel some of the concepts (e.g. “placement,” “proficiency,” and “achievement”) discussed in class and in the readings has helped me understand what I'm already doing or what I could do in the area of adult second language assessment. This is by "application"-using what I've learned and put it into "action".
Week #5
*Note: Below are chapters I didn't have to read, which I "accidentally" misread the assignment:
Chapter 4. "The Role of Background Knowledge in the Assessment of Receptive Skills" (Bailey) Goodnewseverybody.com: Caucasian-Polish jokes
1. content schemata-"background knowledge that led you to predict"
-e.g. terrorist's mission, safari, ->fishing trip!
a. schema activator-"something that triggers the use of your existing schemata" (pp. 53)
b. advance organizer-"a shema activator; helps the listener/reader prepare to pose and test hypotheses about the natureof the incoming text" (pp. 53)
2. formal schemata-"background knowledge that led you to anticipate the structure"
-e.g. polish joke (culturally sensitive)
Chapter #5 "A Cloze-Knit Family"
pragmatic expectancy grammar-"the portion of our internalized language competence that enables us to predict likely sequences of incoming language, whether we are reading or listening
-e.g. NIGHT DROP VALLY PLASTERING syntagmatic competence-"tells what part of speech is needed; syntax-rules of word order" paradigmatic competence-"tells us the semantic features required of the missing item." discrete-point approach-"limited context" pretest
A. Cloze Tests cloze passages-texts longer than a sentence-usually at least a paragraph in length exact word message acceptable word method mutilation-"process of deleting words to create a close test" rational deletion (e.g. delete verbs) fixed ratio deletion/nth word deletion-every nth word is omitted
B. C Tests "deleting parts of words the rule of two-"individual letters of every other word hasve been replaced by blanks" GoodnewsEverybody.com Asian-Sri Lankan elephants-"Sinhalese staircase" think-aloud protocol-"...a protocol is something written down-minds of learners...
*"I believe the acceptable word scoring method is superior to the exact word procedure in terms of washback because it should promote creativity and risk-taking among the language learners" practice effect-"results on the second test may be influenced for the test takers" "counterbalancing for an ordering effect-"altering the ordering of administration" field independence-"..perceive part of a field as discrete from the surrounding field as a while, rather than embedded, or...the extent to which a person perceives analytically"
Chapter #6 "Another Dilemma: Contrasting Pairs of Concepts"
discrete-point testing-"each item on a test is intended to measure one and only one linguistic element
-e.g. a. chip b. ship c. cheap d. sheep integrative testing-one or more levels of language and or/more skills
objective scored rater judgments subjectively scored-value judgments editing task-identifying the portion of a text that contains an error statistical procedure-systematically investigate a classic problem in language assessment
prompt-a topic for an essay taks rater drift-raters scores drifted away from the scale
5.1
Chapter #4 "Oral Language Assessment" (pp. 57-81 O'Malley)
*see GoodnewsEverybody.com Class Tips for Selecting Assessment Activities
DISCUSSION BOARD
Author: F, Michelle
Subject: Job Corps Website "
Simply by googling "adult ELL assessment" I found the Job Corps website. It is well laid out, clear, and comprehensive. For me, there isn't much new information. I hope some of you might find it useful. I'll keep it in mind for all the times I'm asked to explain how to teach ELL by people new to teaching about to go teach an ELL class by themselves."
Author: W, Elizabeth
Subject: Literacy, reading and the internet "
Here is an article about literacy in general among teens. It raises the question whether the internet is helping or hurting literacy - standarized test scores are flat or declining although giving internet access to students in low-income groups did raise their scores. There is also talk about a digital reading component on standardized tests - other countries are doing this. Another question raised is do we need to teach internet reading skills? An interesting article!"
Author: S, Cecile
Subject: web assignment "Attached is my assignment on SABES, System for Adult Basic Education Support. This site contains some good information for teachers of adult ELL students.
Author: L, Danielle
Attachment: Web 6638.doc (28160 bytes)
Subject: Danielle: NCSALL website "
Hope you find this information useful!"
Author: R, Joyce
Attachment: Robles_Web_Assign.doc (33280 bytes)
Subject: Joyce_Practioner_Toolkit "
The site I describe is owned by the State of Michigan; however, the information is not state specific. If you work in a program with Spanish speakers, you might find the needs assessment samples helpful. This is an opportunity to look out how accessible a pdf can be--I was surprised!"- Joyce
Author: S, Christina 0
Attachment: Solomon_C - Web Assignment.doc (37888 bytes)
Subject: European Perspective "Hi all,
found this site from the University of Cambridge website. It was interesting to get a different point of view.
Thanks-CS
Author: F, Patricia
Attachment: ESL 6638 Web Assignment.docx (14688 bytes)
Subject: Virginia Adult Learning Resource Center- Patti "
Think outside the box with me on this one. I was amazed at how I perceived this material since gaining more knowledge of assessments in this class.
It is an ESOL Starter Kit document, but it is chock full of resources and material that could be used for authentic assessment as well as information about standardized and authentic assessments.
Danielle, check out the "Teaching ESOL Using Word Processing" and the activities within that section (p.237) for ideas with your final project.
Author: L, Anne
Attachment: ESL 6638 Web assignment.doc (27648 bytes)
Subject: web assignment "
This website deals mainly with migrant students, but the information can be applied to all ELL's.
Author: S, Caitlin
Attachment: ESL2 Web Assigment.docx (26227 bytes)
Subject: Practical Ideas on Alternative Assessment "
I chose a web article entitled "Practical Ideas on Alternative Assessment" sponsored on both the CAL and NCSALL pages. Please take a look!
Author: Meisner, Ken
Attachment: MeisnerWebAssignment.doc (53760 bytes)
Subject: Accuplacer Preparation "
I found this site by googling "Accuplacer". It was the first one on the list.
Author: R, Rallou
Attachment: Web assignment 6638.doc (26112 bytes)
Subject: Web assignment "Attached is the website I found."
Author: W, Elizabeth
Attachment: will_e_webassignment.doc (41472 bytes)
Subject: performance based assessment "
I looked at the REEP curriculum because I was interested in performance based assessment."-
liz
Other Web sites to begin with from Summer 2008 Online
Center for Advanced Research on Language Acquisition (CARLA) is one of the U.S. Department of Education's Title VI National Language Resource Centers, whose role is to improve the nation's capacity to teach and learn foreign languages effectively.
"offers the world's leading range of certificates for learners and teachers of English - taken by over 2 million people in 130 countries. They help people gain entrance to university or college, improve job prospects or measure progress in English.."
Chapter #5 "Reading Assessment" (pp. 93- O' Malley)
"The mission of I teach I learn.com is to provide opportunities for learning to students and educators from all cultural / linguistic backgrounds and economic levels."
" We have created this summary of the different types of assessment that can be used for measuring development in reading skills in the hopes that teachers will better understand how single skills can be assessed by multiple measures. This description of the various assessment techniques may also help teachers to design their own classroom assessments, and may help teachers to better understand the district or campus assessments that are already being used with their students.
Each of the elements of the framework is briefly described, and descriptions of various forms of assessments that could be used for that framework element are provided. "
*see long lists
*see Cloze Test (Ch. 5-Bailey-up above) reciprocal teaching-instructonal approach designed to increase reading comprhension by encouraging students to use reading strategies. think-alouds-"interactive and focus on active construction of meaning that emphasizes the use of prior knowledge"(pp. 120)
Reading graph·o·phone /ˈgræfəˌfoʊn/ Pronunciation Key - Show Spelled Pronunciation[graf-uh-fohn] Pronunciation Key - Show IPA Pronunciation (dictionary.reference.com)
–noun
"a phonograph for recording and reproducing sounds on wax records."
5.3
#1-"I Like.."
(Purpose: "Expressive Writing" pp. 137 O'Malley)
-New Mexico Portfolio Writing Assessment Scoring Key (pp. 11)
"1-Sentences are simple, short, and repetitous"
-Fairfax County Public Schools: ESL Student Writing Development Scale (pp 15)
"3-Writes Sentences centered around one idea"
-My Child as A Language Learner: Parent Obeservation (Grade 4-Grade 6) (pp. 16)
-Sample Criteria for Holistic Writing Task Assessment (pp 17)
"2-many elements missing"
-Sample Peer Writing Checklist (pp. 18)
"Ideas & Content-Main Idea Stands Out (developing)"
"Sentence Fluency-Sentences Logical" (achieved)
"Sentence Fluency-Sentences Vary in Length (developing)"
"Voice-Strong Sense of the Person Behind the Words (developing)
"Conventions-Grammar and Usage is Correct (developing), Punctuation is Accurate (achieved), Spelling is Correct (developing),
I would suggest to this writer using a "Holistic Scoring Rubric" (e.g. O'Malley pp. 143) that he/she should "Begin to write a paragraph by organizing ideas" Level 3); however, the intent of this writer might be a different format (e.g. poem).
#2-"Park, Cows, NY"-from Russian
(Purpose: "Expository/Informative" pp. 137 O'Malley)
-Writing Self Assessment Questionnaire
-Developmental Scale of Writing for English Language Learners St. Paul Writing Rubric (pp.
9)
-New Mexico Portfolio Writing Assessment Scoring Key (pp. 11)
"2-There is basically good sentence structure with awkward or confusing constructions"
-Developmental Scale of Writing for English Language Learners St. Paul Writing Rubric (pp.
13)
-Fairfax County Public Schools: ESL Student Writing Development Scale (pp 15)
"4-Uses some compound and complex sentences
-Sample Criteria for Holistic Writing Task Assessment (pp 17)
"2-Competent/Acceptable"
-Sample Peer Writing Checklist (pp. 18)
"Ideas & Content-Main Idea Stands Out (achieved)"
"Ideas & Content-Writing Reflects Own Thinking & Experience (developing)"
"Organization-Smooth Transitions (Developing)"
"Voice-Strong Sense of the Person Behind the Words (developing)
"Conventions-Grammar and Usage is Correct (developing), Spelling is Correct (developing)
I would too suggest to this writer using a "Holistic Scoring Rubric" (e.g. O'Malley pp. 143) that he/she should "Begin to write a paragraph by organizing ideas" Level 3). Add at least 3 sentences in each of the three topics to form paragraphs.
#3-"Giants in America?-from Laotian"
(Purpose: "Expressive/Narrative Writing" pp. 137 O'Malley)
-New Mexico Portfolio Writing Assessment Scoring Key (pp. 11)
"3-All Sentences are complete"
-Developmental Scale of Writing for English Language Learners St. Paul Writing Rubric (pp. 13)
-Fairfax County Public Schools: ESL Student Writing Development Scale (pp 15)
"5-Writes a retelling of personal experience"
-Sample Criteria for Holistic Writing Task Assessment (pp 17)
"2-topic and purpose unclear"
-Sample Peer Writing Checklist (pp. 18)
"Ideas & Content-Main Idea Stands Out (achieved)"
"Ideas & Content-Writing Reflects Own Thinking & Experience (achieved)"
"Word Choice-Words Convey the Desired Message" (achieved)
"Organization-Details Logically Placed" (achieved)
"Organization-Overall Structure of Paper Flows Well (achieved), Own Thinking & Experience (Developing)"
"Sentence Fluency-Sentences Logical" (achieved)
"Voice-Strong Sense of the Person Behind the Words (achieved)
"Conventions-Grammar and Usage is Correct (developing), Spelling is Correct (developing),
I too would suggest to this writer using a "Holistic Scoring Rubric" (e.g. O'Malley pp. 143) that he/she should improve on "Writing with grammatical/mechanical errors that seldom diminish communication" Level 4). This sample probably would be the "best" of all 3 and would serve as a "benchmark paper/anchor paper", a "classic example" (Chapter #12 "Three Approaches to Scoring Writing Samples" pp. 189 Bailey) for other students to model withing the 3 samples.
Overall, I feel the last rubric "Sample Peer Writing Checklist (pp. 18, which is "somewhat" similar to pp. 158 O'Malley) is the "best" to use for all 3 samples above. My personal reason is that the checklist format makes it easier to use for assessing these types of papers. Also, it provides the student some positive feedback (e.g. "achieved"-similar to the "analytic scoring" section in pp. 144-145 O'Malley) for the "evaluator" (teacher or peer) to use. It would be nice to put a "comment" section (like on Figure 6.3 pp. 146-147). As you can see above, there is not just only "one" particular style rubric to use. I would use a variety of scoring traits from each style rubric and mix them together for that particular assessment I would focus on. I would recommend that each writer for each sample above use a "Process Writing Checklist" (pp. 149 O'Malley) with a student-teacher conference (pp. 138 O'Malley). Later, have them use a "Writer's Checklist" (e.g. "Did you correct errors in spelling, capitalization, punctuation, and usage?" pp. 141 O'Malley).
Subject: Re: Checklist(s): Example-Sample Peer Writing... "
Sal,
Wow! I appreciated all of those rubric samples within the reading, but you out did yourself. Way to go. Great application of the material. Thanks for applying it to the assignment and sharing with us.- joyce
Author: W, Todd
Subject: Re: Checklist(s): Example-Sample Peer Writing...
"
Sal, I liked your statement: As you can see above, there is not just only "one" particular style rubric to use. I would use a variety of scoring traits from each style rubric and mix them together ......
Matching the assessment tool to the context, content, instructional objective, purpose (e.g. feedback to the learner), etc.-
Todd
Author: W, Elizabeth
Subject: Re: rubric website "
I found this website while searching for one for the web assignment. I only looked at it quickly. In order to change anything on the pre-made rbric you have to paste it into your own document. http://www.teach-nology.com/web_tools/rubrics/
5.4
#4 Methods of Common Approaches to Assessing Reading:
1. Curriculum-Based Measurement- (CBM) "..is a method teachers use to find out how students are progressing in basic academic areas such as math, reading, writing, and spelling...
For example, in reading, the child may be asked to read aloud for one minute. Each child’s scores are recorded on a graph and compared to the expected performance on the content for that year. The graph allows the teacher, and you, to see quickly how the child’s performance compares to expectations....-from studentprogress.org
2. miscue-analysis-".. originally developed by Ken Goodman for the purpose of understanding the reading process. It is a diagnostic tool that helps researchers/teachers gain insight into the reading process.
The term miscue was initiated by Ken Goodman to describe an observed response in the reading process that does not match the expected response. Goodman uses the term "miscue", rather than "error" or "mistake" to avoid value implications. He states that the departures from the text are not necessarily a negative aspect of the reading process but rather “windows on the reading process” (Goodman, 1969, p.123)."-from Wikipedia.org
*see Chapter #5 (pp. 124-125 O'Malley)
3. Multiple Choice
4. Recall
a. decoding
b. comprehension
c. both
-1.
b.
-2.
b. As O'Malley states in Chapter #5 (pp.124)-"..provides information of at least three types: (1) the reader's ability to use language and the reading process, (2) the reader's approaches to reading and reading comprhension, and (3) information for revising instructional approaches and materials. Miscue analysis involves listening to a student read aloud as well as having the student retell the story or answer comprehension questions."
-3
b. Multiple Choice is just one type of strategy that can be used to help an ELL/ESL student in their reading comprehension. O'Malley states, "...finding the main idea; identifying sources of information needed to answer a question..pp. 95" (if given the "right" multiple choice question) -can increase reading comprehension by these types of teaching comprehension strategies.
-4
c-both
Overall, an ELL/ESL student needs some "..prior knowledge and match it to information in the text", which is schema activation (O'Malley pp. 96-97). That sample under that Multiple Choice would be very difficult to "decode" if an individual ELL/ESL student has no prior knowledge of "Fussball", which can be translated to "foosball" (table with 8 rods...), "football", or even soccer (called "futbol" in many countries outside of the United States of America). In fact, I have to state "American football" instead of plain "football" to my ELL/ESL Students because they get it mixed-up with soccer. What was the right answer anyways to that Multiple Choice example-my guess was A. Jahre (what was this?, did anybody have any luck decoding this?). Lastly, that multiple choice example looked so much like "German"!
DISCUSSION BOARD:
Author: R, Joyce
Subject: Joyce_Practioner_Toolkit "
The site I describe is owned by the State of Michigan; however, the information is not state specific. If you work in a program with Spanish speakers, you might find the needs assessment samples helpful. This is an opportunity to look out how accessible a pdf can be--I was surprised! Joyce" http://www.nytimes.com/2008/07/27/books/27reading.html?_r=1&hp&oref=slogin
Literacy Debate: Online, R U Really Reading?
".. As teenagers’ scores on standardized reading tests have declined or stagnated, some argue that the hours spent prowling the Internet are the enemy of reading — diminishing literacy, wrecking attention spans and destroying a precious common culture that exists only through the reading of books.
But others say the Internet has created a new kind of reading, one that schools and society should not discount. The Web inspires a teenager like Nadia, who might otherwise spend most of her leisure time watching television, to read and write.
Author: W., Elizabeth
Subject: Re: Getting Started "
I just went and looked up "'decoding'. The definition includes letter-sound relationships. sounding out the words in the German text does help you relate the word to the English equivalent. I also got the right answer not just by comprehension because I don't know what 'gern' means even though I do know what 'spielt' means. I did use more than just guessing. I matched the words in the question to the place in the text where they occured. What is that called?"-liz
"The C.E.L.S.A. combines three formats of cloze into one: rational, fixed ratio, and multiple choice. Either of the two forms of the test will place students into seven levels of proficiency from low begining to advanced plus. Each 45 minute test has 75 items with 4 choices. Reliabilities for C.E.L.S.A. 1 and C.E.L.S.A. 2 are .93,.94 and the forms correlate .90..."
What I need to do to fill out a cloze test correctly is:
Know the terminology of the "exact word", which I would need as much information from the test giver or teaching assessing this particular test. As O'Malley states, "At the instructional level, students can proceed but only with assistance from a teacher." (pp.114). She recommends that this type of assessment be used for "intermediate-and-advanced-level readers" (pp. 114). Also, "Students need practice with the cloze in whole class and group settings before being asked to work individually", which she gives advice on how to go about this on page 118.
I think that language assessment people like having this instrument as part of the assessment tool kit because it gives the student to think more and be creative. It challenges the student to exercise their cognitive skills. In chapter #5 "Reading Assessment" by O'Malley, she states-"Teachers can use cloze procedures to determine which students need extra help or easier material..".
I personally did not "fair well" taking the test-got within the "instructional level". A couple of my answers were "close" to the "exact-word" (e.g. Q#8 start->try, Q#14 filling->filled). I would've done better if this was an "acceptable word scoring method" (pp. 71 Bailey-*wasn't required to read this chapter, but accidentally read it when I misread the assignment earlier this week), which Bailey has some interesting findings on comparing these two (exact & acceptable word method) on pp. 71.
Subject: Re: FLAFL support group needed?
"I'm sure there are others, but I thought I remembered that SABES (System for Adult Basic Education Support, which is Massechusett's name for their Adult Basic Education professional development and program support) (I had to look up the actually name :-)) has this guide http://sabes.org/resources/acron.htm.
I've tried to be complete in the course in giving the full name at least the first time I use an acronym, but your post makes me realize that I should follow that practice within each Unit.
Thank you, -
Todd
Author: L, Danielle
Subject: Re: FLAFL support group needed?
"
Since we're on this topic: I've been told by peers in K-12 ed that we are now supposed to say ESOL (Eng For Speakers of Other Lang) rather than ESL because many of these people actually speak two or more languagesalready before they learn English. I also saw the MN state colleges (forgot the acronym!) also have changed their website to read ESOL. But Hamline still uses ESL. Todd or anyone?" English as a Foreign or Second Language
From Wikipedia, the free encyclopedia
English within English-speaking countries "In the US, Canada and Australia, this use of English is called ESL (English as a second language). This term has been criticized on the grounds that many learners already speak more than one language. A counter-argument says that the word "a" in the phrase "a second language" means there is no presumption that English is the second acquired language (see also Second language). TESL is the teaching of English as a second language.
In the UK, Ireland and New Zealand, the term ESL has been replaced by ESOL (English for speakers of other languages). In these countries TESOL (teaching English to speakers of other languages) is normally used to refer to teaching English only to this group. In the UK, the term EAL (English as an additional language), rather than ESOL, is usually used when talking about primary and secondary schools.[1]
Other acronyms were created to describe the person rather than the language to be learned. The term LEP (Limited English proficiency) was created in 1975 by the Lau Remedies following a decision of the US Supreme Court. ELL (English Language Learner), used by United States governments and school systems, was created by Charlene Rivera of the Center for Equity and Excellence in Education in an effort to label learners positively, rather than ascribing a deficiency to them. LOTE (Languages other than English) is a parallel term used in Canada, Australia, and New Zealand."
DISCUSSION BOARD:
-Topic: Forum: 6.1 TESOL Standards
1. Is the list of 16 standars comprehensive, ie. did they miss anything?
I thought they did a great job on this and was well written. I've been in this field (volunteer teaching ESL-GED for 4 years this coming Fall) and I'm still learning more each day/week. I'm not sure if I missed it, but it would be nice to add "Real Life-Skills" as a way to assess our ESL/ESOL students. Throughout these 5+ weeks in this on-line course, we've discussed about meeting the "needs of these students". I don't remember the exact week, but we read about the students learning "basic life-skills" for survival. For example, being able to communicate their health problems in case of a medical crisis when visiting a hospital. Going around the community asking for directions to the post office, grocery story, library, or even the police department. Knowing what to do in an emergency (e.g. call 911). I guess the closest standard I could see this fit is with letter K.."Assessment activities document learners' progress within the ESL program toward advancement...of other educational goals." What do you all think?
2. Does the format, measures and sample evidence for each section provided enough guidance to evaluate a program?
I believe this was well-done and can't really see any improvements when it comes to the "measuring" part.
3. For those of you who are working or have worked in an adult ESL program, does your program address all of the standards and, if not what's missing?
Well, this question might be a little personal for our local program, but we might have similar "challenges" (some listed below) that we can all share to help one another:
-I would love to see an improvement in computer technology if possible in our area. This would be under 1. Program Structure and Administration G. (pp. 66). The program provides equipment for daily operations and efficient record keeping. We do have the computers (e.g. labtops) to use our Rosetta Stone and Marshall ABE software programs; however, they are not hooked up to the internet. With the high-tech media technology out there, it would be more "interactive" to use some online programs out there I've found on the internet. Thus, I would like to see "internet availability" under "_1 The program provides basic..._ computer-based management information system". Just sharing my personal preference of interest from past experiences in this program.
-Also another lack of equipment challenge is a "_tape or CD player" (pp. 67), which in the past we would like to hear some different cultural music (e.g. Barnes and Noble has a good diverse selection of cultural-ethnic music, which I bought years ago; however, misplaced them??) our students (an me also) has had an interest in sharing. I guess, I can bring my own personal "boom box", so this isn't really a big equipment issue.
-Under "J. The program maintains a learner-teacher ratio conducive to meeting learning needs and goals" (pp. 69)-our local on-going challenge is "_4 The program recruits classroom aides, paraprofessionals, or volunteers to improve the learner-teacher ratio as needed". In the beginning part of the school year, we get a lot of volunteers from the local university, but they get so busy (they attend a "highly intense academic college"-I should know because I graduated there and never had time for outside activities) throughout the year. Thus, we are left with my supervisor and I (volunteer coordinator) trying to teach GED and ESL students in the same time (once a week at the elementary school and another day of the week is just ESL students). Sorry, I already shared this during one of the discussion boards during this course-don't mean to bring this up again. However, after learning what we've learned in this 5+ week course, I plan to do some changes. I hope we can better this program to retain both our volunteers and the students we are teaching.
I'll stop here as there will be too many to share, but just to give you all some of the on-going challenges that we hope to overcome and improve our local program as we move forward in the future. Some of these standards don't apply for our local "small" program, but for the ones that oversee us.
Date: Sun Aug 03 2008 16:58
Author: R, Rallou
Subject: Re: Program Self Review "
In our program they have only three paid teachers and they rely heavily on volunteers. This is I believe because funding is limited. There is an increasing demand in ESL and I hope that students don't turn away because of the different teachers they might have during the week. I believe in consistency and I think students tend to get familiar with a teacher versus another. Such a change might result in leaving the program. The volunteers receive continuous training but I do believe that a trained ESL teacher could make a difference in the classroom. "
"By integrating assessment, instruction, and professional development, TABE Complete Language Assessment System--English™ provides teachers with accurate, reliable evaluations of adult students' English language proficiency skills...
Developed jointly by:
* American Educational Research Association (AERA)
* American Psychological Association (APA)
* National Council on Measurement in Education (NCME) "Revised significantly from the 1985 version, the 1999 Standards for Educational and Psychological Testing has more in-depth background material in each chapter, a greater number of standards, and a significantly expanded glossary and index. The new Standards reflects changes in federal law and measurement trends affecting validity; testing individuals with disabilities or different linguistic backgrounds; and new types of tests as well as new uses of existing tests. The Standards is written for the professional and for the educated layperson and addresses professional and technical issues of test development and use in education, psychology and employment. This book is a vitally important reference for professional test developers, sponsors, publishers, users, policymakers, employers, and students in education and psychology." Rights and Responsibilities of Test Takers: Guidelines and Expectations, from apa.org
Preamble "The intent of this statement is to enumerate and clarify the expectations that test takers may reasonably have about the testing process, and the expectations that those who develop, administer, and use tests may have of test takers.
...Perhaps the most fundamental right test takers have is to be able to take tests that meet high professional standards,.."
"
As a test taker, you have the right to:
4. Receive a brief oral or written explanation prior to testing about the purpose(s) for testing, the kind(s) of tests to be used, if the results will be reported to you or to others, and the planned use(s) of the results. If you have a disability, you have the right to inquire and receive information about testing accommodations. If you have difficulty in comprehending the language of the test, you have a right to know in advance of testing whether any accommodations may be available to you."
The Rights of Test Takers:
Guidelines for Testing Professionals
Test takers have the rights described below. It is the responsibility of the professionals involved in the testing process to ensure that test takers receive these rights.
4. b. "Tell test takers, prior to testing, about the planned use(s) of the test results. Upon request, the test taker should be given information about how long such test scores are typically kept on file and remain available."
National Forum on Assessment
"High quality assessment must rest on strong educational foundations. These foundations include organizing schools to meet the learning needs of all their students, understanding how students learn, establishing high standards for student learning, and providing equitable and adequate opportunity to learn."
Principle 1: The Primary Purpose of Assessment is to Improve Student Learning "...Educators assess student learning through such methods as structured and informal observations and interviews, projects and tasks, tests, performances and exhibitions, audio and videotapes, experiments, portfolios, and journals..."
Principle 2: Assessment for Other Purposes Supports Student Learning "Teachers, schools and education systems make important decisions, such as high school graduation, on the basis of information gathered over time, not a single assessment..."
Principle 3: Assessment Systems Are Fair to All Students "Assessment systems, including instruments, policies, practices and uses, are fair to all students. Assessment systems ensure that all students receive fair treatment in order not to limit students' present and future opportunities..."
Principle 4: Professional Collaboration and Development Support Assessment "...Educators determine and participate in professional development and work together to improve their craft. Their competence is strengthened by groups of teachers scoring student work at the district or state levels. Schools, districts, and states provide needed resources for professional development."
Principle 5: The Broad Community Participates in Assessment Development "Assessment systems draw on the community's knowledge and ensure support by including parents, community members, and students, together with educators and professionals with particular expertise, in the development of the system..."
Principle 6: Communication about Assessment is Regular and Clear ".... Educators and institutions communicate, in ordinary language, the purposes, methods, and results of assessment. ..."
Principle 7: Assessment Systems Are Regularly Reviewed and Improved "...Even well-designed systems must adapt to changing conditions and increased knowledge. Reviews are the basis for making decisions to alter all or part of the assessment system. Reviewers include stakeholders in the education system and independent expert analysts..."
"The Reading Skills Assessment for Preliterate Students was developed by Charles LaRue to meet a need for testing students who have very limited English literacy skills, and cannot read and write in their own language. It was created to show which literacy skills and knowledge pre-literate ESL students have. This test is not appropriate for students who have attended school and learned to read and write in any language.
However, some students have no educational background, and have never learned to write in their native language. They often have great difficulty taking written tests of any kind. This test is designed specifically for these students. It provides the instructor with a starting point for their instruction in learning to read and write English. The skills and knowledge that are needed to begin to learn to read are included in this test.
This test combines an oral interview by the instructor, along with reading and writing by the student. It allows us to test the student's knowledge of sounds and phonics, as well as other literacy skills. It reveals what skills a student does and doesn't have. The Reading Skills Test for Pre-literate Students can help determine which skills need to be taught next, and whether a pre-literate student has all of the literacy skills needed to join a mainstream ESL class.
If a student can score a total of 90 points or more on The Reading Skills Test for Preliterate Students, that is strong evidence that they have acquired some basic literacy skills and will be able to function in a regular Level 1 ESL class. The test is available in 3 versions for pre-testing and post-testing. There are instructions and scoring sheets for each version of the test. All the materials are available in pdf for download. "
"he Partnership for Reading is a collaborative effort by three federal agencies - the National Institute for Literacy (NIFL), the National Institute of Child Health and Human Development (NICHD), and the U.S. Department of Education - to bring the findings of evidence-based reading research to the educational community, families, and others with an interest in helping all people learn to read well. First established in 2000, The Partnership is now authorized by the No Child Left Behind Act of 2001 (P.L. 107-110)..." Assessment Strategies & Reading Profiles:
Research-based assessment practices for the adult education classroom
Welcome to the Interactive Website for Adult Education Practitioners
ARCS - The Adult Reading Components Study at NCSALL
"This website is based on the ARCS, a study that assessed the reading of 955 adult learners. Researchers tested participants individually on eleven skills (components) that contribute to reading ability. A list of scores for each learner became that individual's reading profile, illustrating his or her strengths and instructional needs.."
-Introduction to the Mini-Course
=>Reading Components "The ARCSRR assessed twelve reading components: phonemic awareness, word attack (phonics), word recognition, spelling, oral reading accuracy, oral reading rate, rapid automatized naming (letters), word meaning (listening), word meaning (expressive), background knowledge, short term memory for numbers, and silent reading (text) comprehension.
On this web site, we have selected five of the components to picture different patterns of reading strengths and weaknesses: word recognition, spelling, word meaning, silent reading (text) comprehension, and oral reading rate. We chose these components because they are major determinates of reading ability and users of this website will be able to find tests to assess them."
=>Print Skills (Alphabetics) "...Learning the basics of the alphabetic system--that is, understanding the predictable correspondences that exist between sounds and letters--develops alongside letter recognition. We found that among the learners in the ARCSRR 100% of those who tested below fifth grade level (21% of our sample) did not know all the consonant sounds, the basic building elements of word analysis and word recognition.
The purpose of achieving automaticity in print skills (alphabetics) is to free a reader's attention for comprehension - the real purpose of reading..."
=>Print Skills (Alphabetics)
Phonemics "A phoneme is the smallest unit of sound of an alphabetic language. For example, may has two phonemes, /m/ /ay/; sit has three phonemes, /s/ /i/ /t/, tax has four phonemes, /t/ /a/ /k/ /s/. Phonemic awareness (PA) is the ability to distinguish our speech sounds...
=>Print Skills (Alphabetics)
Word Recognition "Word recognition is the ability of a reader to recognize written words correctly and virtually effortlessly. It is sometimes referred to as isolated word recognition because it entails a reader's ability to recognize words individually--from a list, for example--without the benefit of surrounding words for contextual help..."
=>Print Skills (Alphabetics)
Word Recognition
Sight Words "..Words that should be recognized "on sight" are those that are most useful whether they are regular (those that can be sounded out) or irregular (those that do not follow word analysis rules). Sight words are memorized as whole words even if they can be sounded out. Assess a learner's knowledge of basic sight words if word recognition level is below GE 5. Assess by having the learner read a list of high frequency words from a basic word list.
You can download the Dolch Basic Words and the Fry Instant Words from this site. (Click here to go to the "Resources" page now to view these lists.) You may print and make copies of these lists...
=>Print Skills (Alphabetics)
Word Recognition
Word Analysis "Word analysis (WA) is more commonly known as "phonics" or "decoding." It is the process of using the written patterns of speech to figure out unfamiliar words. It also includes the ability to recognize and take advantage of the recurring spelling patterns of a language; this ability can then help the reader to recognize words quickly and accurately. For those readers with advanced word analysis skills, the term also refers to knowledge of the meanings and spellings of prefixes, root words, and suffixes. Word analysis instruction can be very effective in helping beginning readers learn to read with understanding.."
Affixes (prefixes and suffixes)RR "The ability to recognize and know how to use suffixes is developed throughout K-12 with its earliest introduction in first grade of inflectional suffixes that have to do with the grammar of a sentence like -s, -ed, -ing, and of derivational suffixes that change the part of speech, for example, -er meaning "one who" as in teacher, or -en meaning "made of" as in golden.
=>Print Skills (Alphabetics)
Spelling "Spelling words = Encoding = transposing speech into writing.
Reading words = Decoding = transposing writing into speech.
Good readers are able to spell at levels close to their word reading ability. This is not surprising because both abilities require the same skills: phonemic awareness, word analysis, and visual memory."
=>Print Skills (Alphabetics)
Rate & Fluenc "Oral reading can also be called "word recognition in context" and refers to the ability to read connected text aloud with accuracy, speed, and appropriate phrasing...
=>Meaning Skills "Meaning skills have to do with enlarging the listening (oral language) vocabulary, learning more about the world, understanding differences between the structure of oral and written communications, and learning how to understand text (strategies)..."
=>Meaning Skills
Word Meaning (Vocabulary) "Fundamental to the comprehension of connected text is knowing the meaning of the words used in that text. Accordingly, the more word meanings readers know, the better they will be able understand text that they can decode."
=>Meaning Skills
Background Knowledge "Background knowledge (or background information) also influences comprehension. A skilled, fluent reader who knows a lot about American history, for instance, will have an easier time understanding a passage about the Civil War than a reader who does not bring such background knowledge to the reading. We usually gain much of our background knowledge when we are children and adolescents in school..."
=>Meaning Skills
Silent Reading Comprehension
Ultimate Goal of Reading Instruction "There are other subskills of reading comprehension that require knowledge of the structure of language (compound and complex sentences, paragraphs, stories and informational text), and the ability to interact with text (metacognition) by assuming some responsibility for understanding the author's message. Many ABE learners have to learn "how" to comprehend. Teachers address these issues through instruction in reading comprehension strategies."
The Purpose of Assessment is to Drive Instruction "Adults' reading development is most often uneven. Adults have had literacy experiences and school-based reading instruction; they have coped with the skills they have; they may have gained skill in one component that has allowed them to partially compensate for and mask weaknesses in others. On the following page, Comparison of Three Reading Profiles, you will see how silent reading comprehension scores alone do not give an indication of underlying weaknesses in other components that can be expected to deter further progress." Comparison of Three Reading Profiles "As the table above shows, "Andrew," "Barbara," and "Carla" have identical Silent Reading Comprehension scores of GE 8.0, but they have different abilities on the other components. Each of these learners needs a different focus of instruction to be better readers." Browse All ARCS Comparison Profiles "Throughout this Mini-Course, we've talked a lot about reading profiles. If you would like to view the eleven ARCS Comparison profiles that we feature in the "Match a Profile" section of this site, please use the links below."
DISCUSSION BOARD:
-Other 6 Module Issues:
Author: F, Patricia
Subject: an application of what we've learned 'A front page headline and article in today's (Aug. 2) Chicago Tribune has given me an appreciation of what I have learned during this class. Six weeks ago I would have glanced at the headline ("Errors may snarl state testing") and quickly scanned the story. Yet another story about the woes of the Chicago Public School System, and there are many, but what does it have to do with me and much of the information I don't understand anyway."
Today I read the entire article and understood it!!
Here is a link to the article. http://www.chicagotribune.com/news/nationworld/chi-state-test-scores-02aug02,0,1406470.story
It isn't very long and I believe you will find many applications of what we have learned about the pitfalls of standardized testing as well as the ongoing debate as to their value in the education system.
Thank you Todd for sharing your knowledge of this subject with us. Even if I were not using it as a teacher it has enriched my life in that I can read an article like this one and have a greater understanding of the content. That is a wonderful feeling.
OK. I can understand the article but my computer skills still stink. The link doesn't work for some reason.
Google: Stephanie Banchero Chicago Tribune aug 2 2008
Hopefully that will bring the article up. I apologize for my lack of computer skills.
Subject: Free Literacy Program Courses Online
"
Hi Everyone,
I don't know that I can endorse this web site, but it is worth investigating. It offers a number of "courses" to help literacy volunteers, teachers, and administrators. My preview made me think it might help out those who are trying to maximizing the use of volunteers. Please share your opinions, if you have time. - joyce
http://literacynetwork.verizon.org/About-Thinkfinity-Literacy-Net.175.0.html
1. Have any of your beliefs regarding assessment and testing changed as a result of participation in this class?
This 6 week course has definitely influenced my beliefs regarding assessment and testing. I never really saw this to be a complicated issue before. It feels like it was a long time ago, but 4 weeks ago during Module #2-I was "skeptic" or "pessimistic", but a little "optimistic" too about testing overall. I remember the section on "Doubters and Believers" (good theme for this particular topic!) on the "Discussion Board", I was "undecided"! Now, I feel more "optimistic" on assessment and testing as I know more on what purpose it serves. Just like the saying, "the fear of the unknown"! As it’s fresh in my memory, the standards listed in "Part 4 The Program Self-Review Instrument" was very well-written and concise. However, as I stated in the "Discussion Board" (Module 6)-I feel the "head administrators" should consider a way to assess "real-life skills" to be included in their evaluation on how a ELL/ESL/ESOL student is progressing.
2. Discuss your understanding of the strengths and weaknesses of various assessment approaches and there applicability to the settings and assessment purposes with which you have experience, e.g., standardized testing, various kinds of performance assessments. With regard to the CASAS and BEST, consider these questions:
I've had experience only with CASAS, so I'm able to share only on this particular standardized testing. According to the "content standards" in CASAS...
"Since its inception, CASAS has focused on teaching and assessing basic skills in contexts that are relevant and important to adult learners. CASAS has developed and continues to refine a highly formalized hierarchy of competencies, the application of basic skills that adults need to be fully functional and productive members of society."
I've administered this test many times in the 4 years (this coming Fall) as a volunteer in the local program I'm in. I never knew that they had "360 essential life skills" that are identified by the CASAS Competencies. As I stated in the "Discussion Board" for Module 4.4...
"Wow, after 4:4 "The CASAS System" section, I have a higher respect on this test assessment. As a volunteer being trained on this, I really never "fully" understood what this assessment was all about until reading this particular section. I have a lot of "homework" I need to do now when I do give this test to our ELL/ESL students, which I hope this will improve our program-thanks!"-Sal
However, the "weaknesses" of this test is not being able to communicate with the students "orally" about what's covered in this test. They label their test "Life Skills", but it lacks the interaction-meaning, they are reading these tests, but not "acting-upon" what's given. For example, I took a sample that was around my house. In pp 10 of Form 27 (Life Skills: Reading-Beginning Literacy), question #28 states "No left turn" [with 4 different given signs, which you have to choose one]. This seems simple, but would the student remember this in a "real-life" situation. Here in Morris, we have only 1 one-way street. Unfortunately, our city doesn't have a sign exactly like the one in question #28. Instead, thy have a sign that says "No Enter". This particular question may not be applicable in this region or Minnesota; however, it maybe in another region (e.g. Twin Cities). What I'm trying to demonstrate is that some of these questions in CASAS will never be applied in their daily-life or if it does, they may not remember. Unless, the teacher (me) emphasizes this particular point or question. After just doing this particular question, I feel it's important for "us" ("test-givers", teachers, etc..) to go over these questions after we administer these ELL, ESL, ESOL students. Especially the ones they get "wrong" (e.g. date order-today 8/4/08 or August 4th of 2008; however, many other countries like Mexico has is ordered differently. Thus, I teach or "correct" many of our students that get this wrong.) After this test review, I feel CASAS (still feel "Massachussetts, Curriculum Framework for ESOL" version was more "culturally friendly") will serve its purpose in "teaching basic skills" as stated in it's "content standards". The "test creators" need the "test givers" to work-together in "teaching" the "students" these basic skills that CASAS was created for. What was listed as a "weakness" for CASAS can be "eliminated" if we "test-givers" do our part too. One of the "strengths" CASAS particularly has is it's "Quick Search Tutorial" site where users can "login" and find various materials to equip "test givers"...
""QuickSearch Online has many exciting features. A bold, colorful interface helps users to select from a wide variety of library searches. This year's edition allows "quick searches" for instructional materials by:
* Skill area: reading, math, listening, writing, speaking, critical thinking, and government and history
* A wide range of programs (including Civics, Family Literacy, Workplace)
* Specific CASAS assessment series, i.e. Life and Work, WLS, ECS
* Year of publication of instructional materials
QuickSearch Online can also search for materials that include informal classroom assessments.
The QuickSearch Online Tutorial is now available from the main menu, making it easier to learn how to use QuickSearch Online and to train new staff. Also, several reference tools appear in the Reports and Other Tools menu; for example, test competencies for every CASAS pre- and post-test. " Tests Overview"
Also, just recently taught this week (Module 6.2) is the "Guidelines for Providing Accomodations Using CASAS Assessment for Learners with Disabilities" is another good resource CASAS provides.
3. What assessment principles and concepts are reflected in this/these test(s)?
Some of the assessment principles and concepts that are reflected in this test are stated in their competencies and standards website given to us in Module 3.3:
-Multiple-choice tests "measure basic skills in
a functional context for adults and youth and are constructed from item banks of more than
5,000 test items." (pp. 3)
-"The psychometric methodology used to establish this difficulty level
comes from the Rasch model of Item Response Theory (IRT)" (pp. 3)
-The test measures all four of the basic skills: "reading, writing, listening, and speaking" (pp. 4)-along with math.
4. Why (or why not) do you feel the CASAS and/or BEST test is an effective assessment tool for adult second language learners?
I wasn't convince enought that CASAS was an effective tool until this class when we read many materials on this particular test. Just to note, I have never had any experience with the Best test in our local program. I feel I didn't have any confidence on this type of standardized testing because I wasn't informed enough on what this test was all about. As a local (in Morris, MN) "volunteer teacher" for the last 4 years (Fall of 08'), I did have regular training under the Minnesota Literacy Council at Alexandria; however, I wasn't "trained" much on this. They probably gave me the opportunities to do so when I wasn't aware of it, but I didn't take advantage of it with my other obligations (e.g. my career working with developmentally disabled adults since graduating from college back in 1999) in life. With this online course dedicated to get the ESL Teaching Licensure Certificate, my desire has opened my interest to learn more and to get equipped as a "volunteer teacher" in ESOL.
Getting back to the CASAS topic, I feel they have the resources (e.g. online) to really equip "test givers". As I recently wrote in the "Integrative Paper #1" assignment, I stated the benefits of it so far in my 4 years of experience giving this test. I feel this test can be more effective, as originally the "creators" intended it to be, if "test givers" are informed on "how to use it". For example in Paper #1, I shared a story of a student from Bulgaria that began to speak English very well through the many tools (e.g. Rosetta Stone-skills targeting reading, listening, and speaking) we use to teach ESOL to him. However, he would write to me in e-mail (still to this day), but his writing comprehension would not match his oral comprehension. If I took this online class ("Assessment") prior to being a "volunteer" teacher 4 years ago, I feel I would've been better equipped on teaching our ESOL students. Particularly knowing now and getting informed on the available CASAS resources (e.g. "Quick Search Online" that I mentioned above earlier answering question #2). In fact, I feel the retention rate of our students and volunteers would be better. I guess it's better late then never. Now I know from this mistake has taught me to work on all three types of assessments or four different area skills (oral-speaking and/or listening, reading, and writing) for each student instead of just spending too much time in one particular area.
The CASAS really did not impact "my way" of doing the local program’s curriculum or my instructional objective because I never really looked into the "purpose" and "goal" of this particular assessment. After taking this online course, it will definitely or is affecting the way I do the local program overall. We are using the Rosetta Stone, but we are implementing other variety "instructional objectives". I mentioned this change earlier in the "Discussion Board" that we started in the beginning of 2008 after re-evaluating ways to improve our "retention rate" of our ESOL students. However, after taking this online course, I'm going to encourage my supervisor to focussed other basic skills and it's target-such as, the writting assessment section. I hope to use the many sample "Writing Assessment" tools (I gave some specific ones I would like to use in the "Discussion Board" under Module 5.3) given in the Module 5.3 section.
The CASAS test assess authentic language proficiency or "direct tests" through their multiple choice questions measuring their reading skills. However, it being a paper-work test, it doesn't measure one's oral skills (this is where software programs like Rosetta Stone comes into play).
"The Comprehensive Adult Student Assessment System (CASAS) of California has developed competencies, training manuals, and assessment tools for ABE and ESL programs. The CASAS ESL Appraisal is multiple choice and includes reading and listening items. It is easy to administer because it is given to groups, but does not test oral skills (Sticht, 1990). "
6. How might the CASAS and/or BEST test(s) impact your program’s curriculum or your instructional objectives?
(stated above)
7. How will what you’ve learned change the way you have been or will use assessment in your teaching practice?
I'll focus more on the goal of using all 4 basic skills (speaking, listening, reading, and particuarly-writing). Then apply this to the ESOL student's "need assessment" (e.g. "real-life skill" settings they want to use English-home, work, school, community, etc..) base on my new strategy I thought of during this 6 week-online course, which is doing a background interview questionaaire (e.g. interests/hobbies, skills, educational background, work, etc..) with each individual student. I would like to do this with each student I meet on the "first-day". I will probably add more to their "portfolio" folder (e.g. goal sheet) to keep track of their progress during our classroom sessions.
8. What are you beliefs regarding the use of assessments for various accountability purposes?
I would say my beliefs has changed throughout this 6 week course. In the beginning, I wasn't very too confident about it as I mentioned in Module 2 after reading the given assignments.
9. What has been the most useful content on assessment that you have learned in this course?
I really can't narrow the the exact "most useful" content on assessment that I have learned in this course. Being a volunteer teacher for the last four years (Fall of 08'), I haven't really been focus with a desire on learning assessment-until now! However, being a "computer online junkie" (e.g. make personal websites as you notice in my "Participant Info-Create Homepage" we filled out during Module 1-1st Week), I feel it would be all the interactive online sources as I'll share next.
Will you apply it in your teaching? If yes, how?
I would apply this by going back on my past assignments. For example the "Web Assignement", which I did it on "Teaching Tips: Writing Assessment Tools" from emints.org (based in Missouri). As I stated in this particular assignement, I joined an online group (similar to registering with the "Quick Search Online" for CASAS) to get resources to equip myself on other assessment tools (particularly the basic "writing" skill) to use on my ESOL students. Overall, I will be regularly looking back at the notes I took on a special online site I developed throughout this six week course at:
Liberal Arts: English-Assessment
http://liberalarts.goodnewseverybody.com/english.assessment.html
Project: Design of an Assessment Instrument – Due August 16th
Assessment Context
• What is the purpose(s) of the assessment?
The purpose of this assessment is to see how students are improving in the "four-different area skills" (Listening, Speaking, Reading, & Writing). Thus, to help them feel more confident in applying them in their "real-life" settings (e.g. workplace, home, community, etc..). For this particular exercise, we'll focus on one setting, which will be visiting a hospital/clinc. The student will be assessed on how he/she does in the "pre-admission" (registration form) process, for reasons stated as such:
"The more you can read and fill out ahead of time, the easier the admitting process will be. It will also give you more time to digest the information and to ask questions and get answers. The hospital also has copies of rules, regulations, and statements of patient rights for you to read. Many hospitals now make these forms and information available through their Web sites. Contact the hospital admitting or administrator's office to find out the easiest way for you to get this material in advance."-CalHospitalCompare.org
As a member in committees (e.g. COPC: Community Dialogue) serving our small-town community, I hear many "cultural or language barrier" issues from various services (e.g. hospital, police, social services, etc..). I'm hoping an assessment like this will build confidence of our ESOL students to fill out all types of registration forms (e.g. Money Gram to send money, job applications, Citizenship forms, etc..).
We hope this would be given as both a summative and fromative placement test to inform both student and teacher.
*taken from Module 3.6 Student-Pair Assignement
• What kind of assessment/test is it to be? proficiency, diagnostic, placement, curriculum based, standards or competency based
The type of assessment this will be is language proficiency. To see how all language "basic" skills (Listening, Speaking, Reading, & Writing) are performed.
• What abilities/skills/instructional objectives are to be assessed?
Listening & Speaking
I'll read the evaluation/assessment of their conversation with the person behind the hospital desk during the "admission" or "registration" process. I'm hoping the hospital will fill it out and give me any feedback.
Speaking
I'll watch and evaluate their conversation with the person behind the desk" during the "admission" or "registration" process.
Reading
How well they do in filling out the "Pre Admission" Questionaaire (below)?
Writing
How well this do in filling out the "Pre Admission" Questionaaire (below)?
• What kinds of decisions will be made or actions taken as a result of the test?
To see what changes can be made to improve this type of assessment in the future. See if we can develop some form that ESL students can bring in their next "hopital" or "doctor" visit to better communicate their reason of visit.
• What constraints or obstacles might be experienced due to time, stakeholders, location, student population, etc.?
Some of the constraints or obstacles that might be experienced are will vary according to the factor(s):
-time: The time period of visit by the ESOL student will vary depending on the medical reason for visiting the clinc or hopital-if they will even have to visit overall.
-location:
-student population:
-english level: Each of our ESOL students have various educational backgrounds (e.g. high school graduate, college graduate, etc...), so not all students will be able to perform each task.
-confidentiality: each student's comfort "level" will vary during each step.
Assessment Instrument
• Content: Describe the construct/content that you are assessing and any component parts if applicable. For example, aural proficiency, grammar, completing a job application, etc.
I'll be evaluating how the student's do in filling out a "Pre-Admission" Questionnaire that they would usually fill out when they visit a clinic-hospital.
• Operations: A list of the tasks that the student will have to carry out, e.g., scanning and locating information
In order to carry out this exercise, the student would need the following information:
-Address, phone #, etc..
*some of my students don't know this by "heart"
-health insurance information
-medical history
Below is a list of all the tasks that the student will have to carry out:
1. Know how to fill out a "Pre-Admission" Evaluation... Pre-Admission Steps, from calhospitalcompare.org "What Information Does the Hospital Need from You? By providing the following information to the hospital before you are admitted, your days before and immediately after entering the hospital will be less hectic, the admitting process will be smoother, and your stay will be safer. The information to provide to the hospital admission office includes:
* Your personal contact information;
* Contact information for the person who will act as your spokesperson;
* Names and contact information of your doctors;
* Details of your health insurance;
* List of your medications, including over-the-counter and vitamins;
* Facts about your health, such as allergies and prior or existing conditions;
* Special needs you may have, such as diet or language assistance; and
* Copies of your Advance Directive and any other health care-related documents.
*see Stevens County Medical Center-Services
• Student profile: A description of your student population.
-Students Profile:
Many of our students (Mexico, Brazil, and Eastern Europen bloc nations) in the Morris Literacy Project are "migrant famers" that work in the surrounding area farms doing various work (e.g. dairy, hog, and typical field work). All of our students have been adults (late teens to the elderly); however, we currently have a student is only a teen. They like to come in groups (Spanish, Portuguese, Russian, etc..) , so the language barrier hasn't been a challenge. However, we they don't come in groups, they somehow work cohesively together.
• Types of items: A description with examples of the kinds of tasks students will have to complete (multiple choice, mock job interview)
• Format and Timing: When and how will you administer the test.
Depending on the steps each student takes. The first step should be easily assessed in the classroom. However, the actual visit to the hospital or clinc will vary from student to student-depending on "if" they need to visit the hopistal or clinic for any reason.
• Criteria levels of performance: This refers to the scoring approach, e.g. a rubric.
LEVEL 1
Level 1-Comprehension
Cannot understand even simple conversation
Level 1-Fluency
Speech is so halting and fragmancy into make conversation virtually impossible
Level 1-Vocabulary
Vocabulary limitations are so extreme as to make conversation virtually impossible
Level 1-Pronunciation
Pronunciation problems so severe as to make speech virtually impossible
Level 1-Grammar
Errors is grammar and word order so severe as to make speech virtually intelligible
LEVEL 2
Level 2-Comprehension
Has great difficulty following what is said. Can comprehend only "social conversations" spoken slowly and with fragment repitition.
Level 2-Fluency
Usually... ,.. forced into silence by language limitations.
Level 2-Vocabulary
Misuse of words and very limited vocabulary...comprhension quite difficult
Level 2-Pronunciation
Very hard to understand because of pronunciation problems. Must frequently repeat in order to be understood.
Level 2-Grammar
Grammar and word order errors make comprehension difficult. Most often rephrase or restrict what is said to basic patterns.
LEVEL 3
Level 3-Comprehension
Comprehends most of what is said at slower than normal speed with repetition
Level 3-Fluency
Speech is everyday communication and classroom discussion is frequnetly disrupted by the student's search for the correct manner of expression.
Level 3-Vocabulary
Frequently uses the wrong words conversation somewhat limited because of inadequate vocabulary
Level 3-Pronunciation
Pronunciation problems associate concentration on the part of the listener and occasionally lead to misunderstanding.
Level 3-Grammar
States frequent errors of grammar and word order which occasionally obscure meaning.
LEVEL 4
Level 4-Comprehension
Understands nearly everything in normal speed although occasional repetition may be necessary
Level 4-Fluency
Speech is everyday communication and classroom discussion with occasional lapses while the student searches for the correct manner of expression.
Level 4-Vocabulary
Occasionally uses appropriate terms or ..phrase ideas because of inadequate vocabulary
Level 4-Pronunciation
Always intelligble though one is conscious of a definite accent and occasional inappropriate unforseen patterns.
Level 4-Grammar
Occasionally makes grammatical or word order errors which did not obscure meaning.
LEVEL 5
Level 5-Comprehension
Understands everyday conversation and normal classroom discussion without difficulty
Level 5-Fluency
Speech is everyday conversation and in classroom discussion afluent and effortless, appearing that of a native speaker.
Level 5-Vocabulary
Use of vocabulary and idioms approximatics that of a native speaker
Level 5-Pronunciation
Pronunciation and approximate a native speaker's
Level 5-Grammar
Grammatical usage and word order approximate a naitve speaker's
Continuum of Speaking Behaviors
BEGINNING
Beginning-Orally
Shares some of own experiences in conversation
Beginning-Vocabulary
Uses a basic, core vocabulary that includes many non-specific words
Beginning-Asking Questions
Obtaining information by questioning in a struggle, still exploring structure of questions.
Beginning-Answering Questions
Learning to use question words and structuring-some answers seem irrelevant.
DEVELOPING
Developing-Orally
Frequently shares own experiences
Developing-Vocabulary
Vocabulary is becoming more specific; beginning to make new words his/her own by generalizing to new situations
Developing-Asking Questions
Asks wh- questions; is developing precision in technique
Developing-Answering Questions
Anwer wh-questions
INDEPENDENT
Independent-Orally
Orally shares information and own experiences
Independent-Vocabulary
Uses a variety of words across many settings and contexts. Hears words during conversation and readly make them his/her own.
Independent-Asking Questions
Obtains necessary information by asking questions during dialogue
Independent-Answering Questions
Responds to variety of questions of increasing complexity
• Discussion of test quality (reliability, validity, practicality, washback)
The format of this type of assessment will vary according to the different factors. One factor would be the degree of the level of our ESOL learner based on CASAS results and "language proficiecy" overall. Another factor will be the type of "registration form" the ESOL student will be using according to his/her "real-life" skill circumstances or current situation (e.g. hospital, police, social services, etc..). For example, I had a student (from Honduras) years ago that asked me to help fill-out a "phone-cable-internet" application to get this particular service in his apartment. I had to "physically" go with him to the service provider's office to interpret his desire of need. I feel this wouldn't be a "reliable" assessment, but a "valid" ("a test is appropriate and meaningful" )one. In terms of "practicality", I feel this assessment will be going through some changes as I find out on how to "rearrange" the format of all three types of assessments. Also, I'll probably get some feedback from individual community members (e.g. local hospital, professors or instructors at the local university, ESOL students, etc...) through community gatherings or individual meetings.
I feel the "washback" on this assessment would bring key particular "real-life" topics to teach our ESOL students as it comes up week by week. I hope to take some type of statistic analysis or record on what are the "most frequent" topics to teach on. Lastly and most importantly, our ESOL students will "get the most of this" as they know what "we" will teach them. Thus, they will know that we are paying attention to their "need analysis", which is the main reason why "we" are there for them!
"According to the 1990 United States Census, there are 14 million persons living in the United States who do not speak English well,2 and when these persons seek treatment from American physicians, they must interact with a health care system designed to serve English speakers. Anecdotal reports suggest that many physicians believe that non-English-speaking (NES) patients are more challenging to care for and require more time during clinic visits than English-speaking patients. If these perceptions are correct, physicians who were allotted the same amount of time to see NES and English-speaking patients might get less done during appointments with NES patients, and NES patients would be at risk of receiving lower-quality care. Health care organizations that serve large numbers of NES patients would be obligated to change scheduling, productivity, and staffing policies. However, despite the potential size and significance of this issue, we have found no published studies that objectively measure whether NES patients take more time during clinic visits."
" A key to effective health promotion and service provision for linguistically diverse communities is implementing linguistically competent policies, structures and practices. Implementing language access is, however, complex and requires knowledge, skills and resources. This feature of the National Center for Cultural Competence website is designed to provide answers to frequently asked questions about how to implement language access.".. Developing Quality Spanish Language Materials
"
This test was developed by a team of native speaker ESL instructors in order to help both students and teachers determine a learner’s current English language skill levels in the areas of listening, grammar, and reading. Each skill area has been divided into three sections reflecting three levels: beginner, intermediate, and advanced. When the test has been completed, test takers will receive an e-mail message with the results."
"Here is a comprehensive language test to test your current level of English. It tests your listening, grammar and reading skills and then gives you a rating between beginner and advanced. The test will take a few seconds to load, so please be patient. The whole test shouldn't take more than ten minutes to complete. If you have more time you might also like to try this placement test..."
".. evaluates advanced-level English language competence of adult nonnative speakers of English such as
*
Students applying to United States, Canadian, British, and other educational institutions where the language of instruction is English.
*
Professionals who need English for work or training purposes.
*
Anyone interested in obtaining a general assessment of their English language proficiency for educational or employment opportunities.
Tools
ESL ReadingSmart Demo
"Visit http://www.eslreadingsmart.com for a free trial!
ESLreadingsmart.com is a Web-based reading and curriculum program for ESL students and teachers. It has been statistically proven to raise students' reading and language scores.
Here is a summary of the features of this effective and innovative program:
• Individualized learning path based on students' placement test results
• Achievement reading test after completion of instructional requirements
• Reading program from beginner to advanced ESL/ELD students, grades 4-12
• Newcomers' program for students who do not speak English
• Assessment system for teachers to track class and student progress
• Reporting system for students to track their own progress"
"Brief presentation of an in-world assessment tool we are currently using on the new Second Life English SIM.
Welcome to the revolution...
Do you speak SLEnglish?" English Village in Second Life, from youtube.com "Here is a video I put together for an interview I am having this Friday by a Korean television station. Let me know what you think!
I am really grateful to have had Hathead Rickenbacker to work together with - we had a lot of fun filming the video!"
"Reading and writing are very important when taking online courses.
Please try the English Self-Assessment Test to find out if your English is good enough to take a Canadian university course online..."
Thank you for visiting GoodnewsEverybody! Please feel free to e-mail me (Sal) at info@goodnewseverybody.com on any comments, suggestions (e.g. any new websites),complaints, or anytype of feedback to improve this website.