Summer 2008
ESL 6638 Assessment of the Adult ESL Student Online
(2 sem. cr.) Dates: June 28-August 9
"Learn the basic principles of testing and evaluation, and how assessment is related to curriculum. Learn how to conduct valid formal and informal assessment of ESL learners. Develop authentic assessment tools for academic and workplace settings. Discuss entrance and exit criteria for ESL services and how to assess student progress. Explore the politics of testing and assessment. Online course and login information at www.hamline.edu/gseonline. Optional orientation session at Hamline on June 28, 10:00 a.m.-12:00p.m. Drew Science 305. Target Audience: ESL teachers of adult learners.
Required Books:
The Essentials of English: A Writer's Handbook (with APA Style) (Spiral-bound)
by Ann Hogue (Author), from amazon.com List Price: $41.33
Understanding and Using English Grammar (Paperback)
by Betty Schrampfer Azar (Author) "Following are some dialogues between Speaker A and Speaker B..." (more)
, from amazon.com List Price: $48.67
Terms
Pyschometrics-"...field of study concerned with the theory and technique of educational and psychological measurement, which includes the measurement of knowledge, abilities, attitudes, and personality traits. The field is primarily concerned with the study of measurement instruments such as questionnaires and tests. It involves two major research tasks, namely: (i) the construction of instruments and procedures for measurement; and (ii) the development and refinement of theoretical approaches to measurement..."
eugenics-"...is a social philosophy which advocates the improvement of human hereditary traits through various forms of intervention.[2] Throughout history, eugenics has been regarded by its various advocates as a social responsibility, an altruistic stance of a society, meant to create healthier and more intelligent people, to save resources, and lessen human suffering."
Craniometry-"..is the technique of measuring the bones of the skull. It is distinct from phrenology, the study of personality and character, and physiognomy, the study of facial features. However, these fields have all claimed the ability to predict traits or intelligence."
*see Rubrics
"scoring scale, is typically created which contains the essential criteria for the task and appropriate levels of performance for each criterion...
holistic rubric-"assigns a level of performance by assessing performance across multiple criteria as a whole"
analytic rubric articulates levels of performance for each criterion so the teacher can assess student performance on each criterion.
valid-"degree to which a certain inference from a test is appropriate and meaningful" (AERA, APA, & NCME, 1985).."
reliable-consistent
washback-effect a test has on teaching and learning
What Is "Washback" and Why Do Some Language Researchers Avoid the Term? , from jalt.org
modality-channel (aural/oral) of language use
Related Sites:
indirect vs. direct tests
*see A Comparison of Direct & Indirect Assessments of Writing Skill, by Hunter M. Breland
discrete-point vs. integrative testing
*see Methods of Assessment, btinternet.com
"Discrete point tests: cannot be pragmatic. There is no ordinary discourse situation and no normal language use context where a learner might be asked to listen and distinguish between "ship" & "sheep" or perform active to passive transformations."
What is a discrete point approach to testing?
"Electronic quiz tools usually involve a discrete point approach to testing as opposed to an integrated or authentic approach, such as papers and projects. Discrete point tests are made up of test questions each of which is meant to measure one content point. Discrete point testing is associated with multiple choice and true/false formats, which have been criticized for testing only recognition knowledge and facilitating guessing and cheating. However, if they are used for an appropriate PURPOSE and if the test questions are well constructed, discrete point tests can be used for effective teaching and learning..."
objective-vs. subjective scoring
*see
Objective and subjective evaluation
"Objective tests are often constructed with selected-response item formats, such as multiple-choice, matching, and true-false. An advantage to including selected-response items in objectively scored tests is that the range of possible answers is limited to the options provided by the test writer—the test taker cannot supply alternative, acceptable responses.
Related Sites:
*concurrent validity (wikipedia)-"is demonstrated where a test correlates well with a measure that has previously been validated. The two measures may be for the same construct, or for different, but presumably related, constructs.
For example, if a test measuring job satisfaction gives similar results to those gathered using a job satisfaction test which has been validated in past investigations the new measurement has concurrent validity. Alternately, a measure of job satisfaction might be correlated with work performance. Note that with concurrent validity, the two measures are taken at the same time. This is in contrast to predictive validity, where one measure occurs earlier and is meant to predict some later measure."
*inter-rater reliability (wikipedia)-"iss the degree of agreement among raters. It gives a score of how much homogeneity, or consensus, there is in the ratings given by judges. It is useful in refining the tools given to human judges, for example by determining if a particular scale is appropriate for measuring a particular variable. If various raters do not agree, either the scale is defective or the raters need to be re-trained."
CURRICULUM-BASED TESTS-"..measurement that uses "direct observation and recording of a student's performance in the local curriculum as a basis for gathering information to make instructional decisions" (Deno, 1987, p. 41)...provides a structured way to see how well a child performs on the materials the teachers is assigning the class.(from umn.edu/~devenz)
".. refer to instruments that assess one element/component of a skill at a time" (Hamline Univ)
COMPETENCY-BASED TESTS-""
What is CBT & Characteristics of CBT
"..A skill performed to a specific standard under specific conditions...Assessment of competency takes the participant’s knowledge and attitudes into account but requires actual performance of the competency as the primary source of evidence. ..
DIAGNOSTIC TESTS -""
Test your English language ability
Online diagnostic tests
*internal consistency(wikipedia)-"is a measure based on the correlations between different items on the same test (or the same subscale on a larger test). It measures whether several items that propose to measure the same general construct produce similar scores. For example, if a respondent expressed agreement with the statements "I like to ride bicycles" and "I've enjoyed riding bicycles in the past", and disagreement with the statement "I hate bicycles", this would be indicative of good internal consistency of the test."
Tests
People
-4 key figures
Theories
Current Issues
History Lesson
DISCUSSION BOARD:
-Topic: Doubters and Believers
Date: Tue Jul 15 2008 23:27
Author: M, Sal
Subject: certificate to show! Re: #2 oppressive and humiliating
"yes, students would like to see scores or some way to know they are doing well. I had one student (from Bulgaria) that asked me and my "supervisor" if we could give him a certificate. He wanted to go back to his home country to show his people (future place of employment, family, etc..) that he took an ESL course while working (was a intern as a vet in a local nearby pig farm) in America! It was the first time my "supervisor" and I ever been asked this, which we will try to do more often now!"
Date: Wed Jul 16 2008 15:10
Author: W, Todd
Subject: Re: certificate to show! Re: #2 oppressive and humiliating
"
Several programs around the country, including Minneapolis, issue certificates to students based on reaching certain score/proficiency levels on the CASAS tests. My understanding is that this process has been well-received in Minneapolis by both students and teachers. Implementing this process also promoted a great deal of consideration of the CASAS compentencies and whether they constitute a reasonable/quality/useful/valuable content domain of instruction for an ESL program. My understanding is that that process was also well received and that many teachers who were skeptical of relying heavily on the CASAS competencies have found the compentencies, on closer examination, to be reasonable/useful.-Todd
Date: Fri Jul 11 2008 21:06
Author: C, Matthew
Subject: Re: #2 oppressive and humiliating
"
When I worked at a intensive English language school, I often had students who craved more drill than we offered. Some would purchase Raymond Murphy's Grammar in Use texts on their own. The underground movement got so popular that we built an elective around the text. At the time, my Academic Director argued that at least that way we could insure that they had some communication based activities along with the grammar.
-Topic: History of Educational Testing
Forum: History of Educational Testing
Date: Sun Jul 06 2008 13:57
Author: H, Gregory
Subject: History of Testing
"Hello Classmates,
I just read the Power Point about the History of testing and it scares me. It seems that our whole approach to achievement testing in the U.S.A. has imitated experiments and practices of people who we deemed were "credible experts". What´s up with the craniometry test to determine intelligence? It seems a little troubling to me that the "IQ" test came from a method Alfred Binet developed to find out the causes why French school children where not meeting curriculum standards. Apparently no one knows how that test can be equated to a true representation of intelligence.
After reading over the 8th grade proficiency exam from a Kansas school, I wonder if this should be more like aCommunityy College proficiency exam nowdays. The questions are almost all based on theory or rote knowledge.
This article gets me a bit riled up, which I believe is the point.
Are achievement tests some means of control? Are standardized tests some means of social domination?
I think we should try to find out a way to test achievement based upon real life situations and performance of tasks. It seems like almost all tests are based on theory or are out of context for the people taking the tests.
Date: Thu Jul 10 2008 14:36
Author: R, Joyce
Subject: Re: Zappardino and the History of Educational Testing
"
I'd like to be candid and say that I never thought studying the history of assessment would be so intersting. (Sorry, Todd) The good news is that this is intriguing.
Historically, I was unaware of the newnesss of the study of the mind psychology" and so aligining Binet's efforts and the entree into scientic methods of measuring elements of the mind helped me to understand how the scientific and lay communities could be awed and motivated to accept answers about the unknown elements of the mind. Aren't we all tempted to take alittle test to find out more about ourselves?
The idea that testing is a socially imbedded activity is important for us to remember as instructors who interface cross-culturally. The cloak of science that covered those early researchers political agendas may well seem obvious to some cultures represented by our students i.e. they may discern that we have motives beyond their benefit in our obsession with formal testing.
Zappardino seems to refer to the "faith" we have/ had in the numbers yielded from testing. I couldn't help but think how often we say "numbers don't lie". We sort of exalt numbers/statistics and the interpretations. Maybe I'm carrying this too far, but there is a scripture that says, "God is not a man that he should lie." So for me, this "numbers don't lie" thing puts numeric results on a pretty high level.
I'm in the dark about how one changes the culture of standardized testing, but I don't want to be a gatekeeper that holds folks at bay from educational or other opportunity, helping them "settle" and come to terms with a lesser view of themselves and their potential. I want to remember the idea of testing being culturally bound, even my own created tests.
Culturally speaking, I have one small anecdote: When we moved to MN from CA our youngest daughter was given a kindergarten readiness test. She was asked to name five jor six things you can do with water. She replied, "Drink it, swim in it, wash dishes." The tester smiled condescendingly, "How about skate on it? When it snows, make snowmen?" later she told me that I might need to read more to my child to help her think in broader contexts. WHAT? Hey, we lived on the beach! We drank it, swam in it and washed up! - Joyce
-Topic: Other Module 2 Issues
Forum: Other Module 2 Issues
Date: Thu Jul 10 2008 00:11
Author: M, Sal
Subject: Good tip for assessment from O'Malley
"
I'm still finishing the reading for Module 2, but I wanted to comment on one particular advice O'Malley gave. It was on "Designing Authentic Assessments" (Ch. 2 pp. 17-18). I live in a small college town (5,000 pop) here in Morris, MN and thought the tip on "conducting professional development..." would be very useful here. I plan to contact some professors at the local campus and ask for their advice on planning authentic assessment. Very informative so far in this reading assignment!
Date: Thu Jul 10 2008 22:40
Author: M, Sal
Subject: The importance of meeting the students' needs
'
Yes, reading Chapter 1 "Assessment as Information Gathering" by Kathleen M. Bailey was very helpful in the part of meeting the students' needs (pp.5-6). We are sometimes given guidelines on what to teach. However, we also need to consider the students' needs, which we'll know once we know the information about them as quoted by Diane's (teacher) dialogue with Kathi (author giving advice). I see this as very true when I teach ESL to local migrant "dairy farmers" compared to other students with a different profession/job. What I've learned from teaching ESL as a volunteer in the last 4 years, the students seem more interested on learning terminology relating to their workplace. They "want" to learn English to improve their work situation. As supposed to a learner that does not work in the same occupation/field or even as a homemaker. "
Date: Fri Jul 11 2008 10:15
Author: M, Sal
Subject: "Massive Systemic ignorance and blind reliance on standardized est scores..."...
"
Wow, the quote above I partly took from Bailey's response on Ellie's (teacher w/three titles in a California school) response to the "school system" she was part of just summarizes one of many stories out there! Reading this (Chapter #3-Conflicting Purposes of Assessment) really got my attention and brought some personal experiences (as a student in grade school and teacher in ESL as an adult). In "grade school", I was close to being sent back because of my "learning disabilities". My parents immigrated here from the Philippines and never taught me their native language (Tagalog). They were afraid of me and my sibling being "left out" in our English-speaking dominant "new home country" (USA). However, my parents would speak Tagalog to each other in the home and their "English" (spoken differently in the Philippies, like Britain somewhat) was a little different. Thus, going to school, I would have trouble-another factor was their "passive" cultural life-style. English grammar was never every my strong quality (math was) growing-up! Anyways, going back to being a "current ESL teacher" (volunteer), I can relate to some of my students (both adult and non-adult). When I tell people in the small rural-college town community (Morris, Minnesota) that some of my students have "college degrees", they get suprised! One of my students (grown-up adult) was a "migrant worker" from Mexico, who got a college degree as a veterinarian . He came to our class because he just wanted to learn English! Just reading the story of Ellie makes me wonder how many school systems has "mis judged" or labeled students with a "learning disability" due to their language being "Non-English"! Wow, great story!
Forum: Other Module 2 Issues
Date: Fri Jul 11 2008 17:43
Author: M, Sal
Subject: Contrasting Pairs of Concepts-examples?
"
Hi all,
I really haven't had that much experience with creating my own testing and scoring of ELL or ESL Students. I was wondering if anybody has and willing to share their experiences relating to the "three contrasting pairs of concepts" Bailey shares in Chapter 6?
Forum: Other Module 2 Issues
Date: Fri Jul 11 2008 18:48
Author: M, Jeannine
Subject: Re: Contrasting Pairs of Concepts-examples?
" One of our English department faculty, Susan Bosher, has done a lot of work with multiple choice testing with ELL health careers students. She uses language simplification techniques. I have one of her journal articles on my other computer. I'll find it and send it as an attachment. In the past I've taken my exams to Susan and had her review the questions. It is extremely helpful. I've also found that the modifications I make based on her suggestions make the test much more understandable for all students- not just English learners. Writing exams, as you probably know, is really hard. JMH "
Author: F, Patricia
Subject: Re: Statement #4 How well do I know my students?
"I also agree with the statement that we can't be everywhere at all times. Even a poorly designed test can give us some information. Say we hand the student a test during registration on the first day that asks them to write their grocery list from last week rather than their name, address, etc. That is neither reliable nor valid. If the student is unable to write one item down on the paper, however, we can take and educated guess that they may not be able to read the instructions or may not be able to write the answers.
Greg, here is a prime example of your statement on being mislead by a student's proficiency in one area. I had a 50 something man from Mexico that came to class one evening. He talked with all the instructors, trading stories and information for more than 15 minutes at a level I would have said was close to a native speaker of English. When we handed him the registration forms he informed us he could not read or write in English or Spanish. That was during my first semester of teaching and it was a lesson I will never forget.
Author: L, Anne
Subject: 1-4 thoughts
"
...2- I tend to go back and forth on this one. If the assessments are relevant and address the needs of the students, then I think assessments are not oppressive and humiliating. However, “throwing” someone into an assessment who has never tested before is oppressive and humiliating.
I. The Four Skills (Listening, Speaking, Reading, & Writing) and Hierarchical Components of Language
A. Syntax and Lexicon
1. syntax (wikipedia)-"study of the principles and rules for constructing sentences in natural languages"
2. Lexicon (wikipedia)-"its vocabulary, including its words and expressions. More formally, it is a language's inventory of lexemes."
B. Morphology (wikipedia)-"field of linguistics that studies the internal structure of words. (Words as units in the lexicon are the subject matter of lexicology.) While words are generally accepted as being (with clitics) the smallest units of syntax, it is clear that in most (if not all) languages, words can be related to other words by rules. For example, English speakers recognize that the words dog, dogs, and dog-catcher are closely related. English speakers recognize these relations from their tacit knowledge of the rules of word-formation in English. They intuit that dog is to dogs as cat is to cats; similarly, dog is to dog-catcher as dish is to dishwasher. The rules understood by the speaker reflect specific patterns (or regularities) in the way words are formed from smaller units and how those smaller units interact in speech. In this way, morphology is the branch of linguistics that studies patterns of word-formation within and across languages, and attempts to formulate rules that model the knowledge of the speakers of those languages..."
C. Phonetics and Phonology
1. Phonetics (wikipedia)-"the study of the physical sounds of human speech. It is concerned with the physical properties of speech sounds (phones), and the processes of their physiological production, auditory reception, and neurophysiological perception."
2. Phonology- (wikipedia)-"is a subfield of linguistics which studies the sound system of a specific language or set of languages. Whereas phonetics is about the physical production and perception of the sounds of speech, phonology describes the way sounds function within a given language or across languages."
Related Sites:
Other Terminology
3.1
"..Other research studies found that teachers’ perceptions
of content standards and the professional development
provided for them play a strong role in the successful
implementation of standards. Specific professional development
about standards-based curriculum and assessment
contributed to better student math achievement in
California, according to a self-report of 1,000 K-12 teachers
(Cohen & Hill, 2000). Case studies and descriptions of professional
development to support standards-based education
in K-12 document a wide range of activities used by
school districts to train teachers, including workshops and
conferences, institutes, observations of expert teachers,
mentoring, study groups, grade-level meetings with other
teachers, site-based inquiry seminars, training and support
to use new curricula and standards, and certification training
(Bye, 2004; Dutro, Fisk,..-pp 5 of 6
Other Related Sites:
3.4
Three Challenges in English Language Assessment, html format & pdf format by Carol A. Chapelle & Joan Jamieson
"..Terms such as “placement,”
“proficiency,” and
“achievement” might be used
in response to a question about
what a test is supposed to be
used for. These terms may work
for the public, but teachers
need a more systematic,
thorough, and accurate way of
considering the purpose of an
English Language assessment
because the purpose of the
assessment is critical for
choosing or developing a good
test. We think of test purpose
as consisting of three
interrelated concepts:
• One way to look at test
purpose is to consider the
inference that test users
want to make on the basis
of test scores. For example,
we designed Longman
English Assessment to allow
test users to make an
inference about examinees’
English Language reading
comprehension, listening
comprehension, and
knowledge of written
structures. Making an
inference about these three
aspects of English ability
was one way of looking at
the test purpose.
• A second way is to
consider the use of the test
results. Typically, a test is
given because a decision
has to be made about
examinees, for example,
their readiness to study at a
university, their eligibility
for a job, or the grade that
hey should receive in a
course. Longman English
Assessment is not intended
for any of these relatively
high-stakes test uses;
instead, it is intended to
give learners an idea of
their level of English
ability and to offer level-
appropriate advice for
improving their English.
• The third aspect of test
purpose is any other effect
that the test is intended to
have, such as serving as a
motivating, interesting, and
informative experience for
learners."
3.5
Final Project
Design of an Assessment Instrument: "Real-Life Skills"
July 17th of 2008
CONTEXT
This assessment is designed for classroom settings to help student learn "daily-living skills (e.g. intro conversations, grocery shopping, and other day-to-day social activities in the community). The "Real-Life English workbooks" (see Steck-Vaughn Company or amazon.com) has different levels, which uses various community "real-life" settings. Students want to be better "active-participants" in their workplace, home, and community while they are residents of this small rural college community (10,000+ population in the Stevens County area we serve). Thus, teacher too want to be "better" in working with each individual students to achieve their goals to succeed-not only in America, but also when they go back to their "home country". This assessment will encourage all three stakeholders (students, teachers, and program) through these results in various ways of performance.
SETTING
-Students Profile:
Many of our students (Mexico, Brazil, and Eastern Europen bloc nations) in the Morris Literacy Project are "migrant famers" that work in the surrounding area farms doing various work (e.g. dairy, hog, and typical field work). All of our students have been adults (late teens to the elderly); however, we currently have a student is only a teen. They like to come in groups (Spanish, Portuguese, Russian, etc..) , so the language barrier hasn't been a challenge. However, we they don't come in groups, they somehow work cohesively together.
-Audience:
The results would be very helpful for teachers (current and future volunteers) on how to improve our approach in teaching this particular topic ("real-life" skills). This may also help our continued new volunteers (mostly new in-coming college students) more interested in our on-going program (Morris Literacy Project). Knowing "what works" and "what doesn't work" (our approach in teaching) may help retain future volunteer teachers and more importantly-students that come to check out how we do class.
PURPOSE:
The purpose of this assessment is to see how students are improving in the "four-different
area skills" (Listening, Speaking, Reading, & Writing). Thus, to help them feel more confident in applying them in their "real-life" settings (e.g. workplace, home, community, etc..). We hope this would be given as both a summative and fromative placement test to inform both student and teacher.
QUESTIONS:
Several questions will come up from this assessment-1: Are the student's skills helping them in their various workplace, home, and/or commmunity settings? Are we consistently retaining our number of volunteer teachers and students in a certain period of time? Do they demonstrate a degree of progress to move to the next level of books of "Real-Life"?
4.1
Terms
Itemized Response Theory:
Wikipedia
Chapter #1-Item Calibration & Ability Assessment, from creative-wisdom (multi-media)
-Guttman Scale, Wikipedia
"..if they can be ranked in some order so that, for a rational respondent, the response pattern can be captured by a single index on that ordered scale. In other words, on a Guttman scale, items are arranged in an order so that an individual who agrees with a particular item also agrees with items of lower rank-order. For example, a series of items could be (1) "I am willing to be near ice cream"; (2) "I am willing to smell ice cream"; (3) "I am willing to eat ice cream"; and (4) "I love to eat ice cream". Agreement with any one item implies agreement with the lower-order items..."
=>Multiple Choice Tests: Test scoring and analysis
Christina Ballantyne (tlc.murdoch.edu.au)
tentative student proficiency-portion of correct answers for each student
tentative item difficulty-the pass rate for each item
convergeance-"there will be some differences int he model.."
Chapter #2-Item Characteristic Curve-"the relationships between the probabilities and the skill levels"
Related Sites:
Assessment Issues:
"..The ICC is a plot of Probability that the Item will be answered correctly against Ability. The shape of the ICC reflects the influence of the three factors:
* Increasing the difficulty of an item causes the curve to shift right - as candidates need to be more able to have the same chance of passing.
* Increasing the discrimination of an item causes the gradient of the curve to increase. Candidates below a given ability are less likely to answer correctly, whilst candidates above a given ability are more likely to answer correctly.
* Increasing the chance raises the baseline of the curve.
This simple simulation allows the user to investigate the factors governing the shape of the Item Characteristic Curve. All three well known IRT models are represented (referred to as IRT1, IRT2 and IRT3) and Item Characteristic Curves can be super-imposed on one another to see how they relate.''
IRT Model (ony 1 parameter=item selection: A&B-2 Parameter; A, B, & G-3 Parameter=diagnosis parameter=2/3 parameters)
"B parameter"/threshold parameter-item difficulty
"A parameter"-item discrimination
"G parameter"-item guessing
Lord's Paradox-a phenomenon of the probability of giving to the correct answer to the blue is .55 and red item is more difficult
intercept
Chapter #3-Item Information Function
item information function-"mathematical way to know how much info ICC provides, the more precision you have in estimating a parameter->the more you know of the value of that parameter
Item information function of the 2PL model, from metheval.uni-jena.de
Chapter #4-Test Information Function
test information function-identifying information for entire exam
alternate forms-having different items with different levels
form balancing-swapping items (e.g. easy, difficult, and hard) between the item pool and the forms is needed
Chapter #5-Item Person Map
odd ratio-ratio of non-desired events (Q)/divided by desired events (P) or Q/P
logit-log (odd ratio)
easy if below zero (e.g. -.7)
item person map-item (test question itself-easy, average, difficult) and person (item proficiency value) can be viewed simultaneously
Q: odd of 1/6 passing?
A: .20
Chapter #6-Misfit
misfit-about the response pattern (e.g. European history question accidentally added in a "American History Exam"); when there is a huge discrepancy (model and data don't fit with each other)
residuals-discrepancies
Chi-square fit-a statistical approach for the most common test of goodnesss (correct & incorrect; categorized by the skill level
O-E 2 (squared)/ E [O-observed, E-expected]
#2 Approaches to detect misfit:
1-graphical method-model and data are superimposed
2-Chi-square test of goodness of fit-chi square divided by the degrees of freedom
Q: In regression we obtain a systematic pattern of residuals
A: F
Residual Plots and Regression Assumptions, from stat.tamu.edu
Degree of Freedom
""Degree of freedom" (df) is an "intimate stranger" to statistics students. Every quantitative-based research paper requires reporting of degrees of freedom associated with the test results such as "F(df1, df2)," yet very few people understand why it is essential to do so. Although the concept "degree of freedom" is taught in introductory statistics classes, many students learn the literal definition of this term rather than its deeper meaning. Failure to understand "degrees of freedom" has a side effect: Students and inexperienced researchers mis-interpret a "perfect-fitted" model or an "over-fitted" model as a good model. To rectify this situation, two approaches of illustrating df in terms of sample size and dimensionality are recommended. To enhance the pedagogy of this concept, a multimedia program written in Macromedia Flash (Macromedia, 2001) was developed. In the program, different aspects of the above concepts are visualized and interactive features (questions and answers) are added to encourage the learners to think about the deeper meanings of df."
df=n (# of obeservations)-r (# of necessary relationships)
Computer Adaptive Testing:
Related Sites:
What is Item Response Theory, IRT? A Tentative Taxonomy, from rasch.org
4.2
4.3
#3 major features:
1. language-bilingual?
2. supplemental assessment
3.oral reading assessment
3rd Reason:
67% of jobs created over the next decade required college degree
#4 literacy levels
"..Next, although 13% of American adults spoke only Spanish or another language before starting school, these individuals accounted for 37% of the below basic population. .."
-Dr. John Stucker
".. I would be working with the other teachers and asking questions about how is our outreach? Are we really reaching these folks in our community? How is our teaching approaching going? How we are doing with the young people who are sometimes getting pushed out of high school?..
as Sheida pointed out, you see the highest percentage of people who report learning disabilities as well as other disabilities. And most learning disabilities involve reading even if they aren't labeled as reading disabilities. So these are people who have identified reading problems. Other studies we have done show that there is a higher percentage of their native US born people who have been in special education, have gotten extra help, whose teachers realize they needed extra help, and whose parents realized they needed it.."
And then the other thing about these folks is, again, it's remarkable how fast these ELLs have picked up English conversation. But don't let that fool you. If you actually get a chance to assess their reading skills in English, it turns out they are extremely idiomatic when they talk to you. If one person walked in and joined us with this panel, we think gosh! That person speaks great English but when they went to do embedded literacy tests, such as the NAAL, those tests were hard for them. Let's talk a little bit about the basic people. I think this is an extremely important population we don't want to neglect, because they are close to intermediate. And we know from other studies that intermediate is the point where you see big changes to the positive in income, civic participation, personal reading habits, all the things that embed literacy in our life and make it so meaningful and important...
...If we can identify their specific needs, their needs for fluency and that sort of literate, academic vocabulary, it may be that we can get them into the intermediate level faster and that opens up a lot of opportunities for them in terms of involvement in post-secondary and all the other things that we talked about..
-Brian Bosworth, Future-Works
".. Fourth point I want to make, most employers say they are as concerned or even more concerned about other workplace basic skills as they are about traditional measures of literacy and what do they mean by those other workplace skills? Technology skills or computer literacy is increasingly important to all employers. They identify career management and lifelong learning skills, and by this I mean the navigational skills associated with career management and lifelong learning. The employers identify, thinking critically, acting logically and solving problems as a critical basic skill. They identify finding and using information, knowing when to ask a question, knowing when you don't have the right information to answer that question and knowing where to go to get that information. They identify teamwork as a basic skill. The ability to work in groups with others cooperatively. They identify basic employability skills as well, sometimes called social skills, attendance, timeliness, work ethic, showing up sometimes is the most critical basic skill. Employers do identify math, reading, writing as important deficits in their workplace. But we find, through our survey work that employers are more likely to suggest some of these other basic workplace skills are as important or more important to them than the conventional measures of literacy.."
They would be contextualized in the language and the situations of the workplace. They would use work as a way to develop the workplace basic skills. They would be focused on a credential, a credential, which satisfies entrants to the next level of education and training....
Q&A:
-Brian
'..NCSALL colleague, Steve Reeder out at Portland state working with other researchers developed a way that you can sort of do a synthetic NAALs on your county or community, based on local demographic data. And some of the other characteristics that occur both on the NAAL assessment, and the background questionnaire and that you can also gather about your local population. .."
-Sheila
"...Sheida White: I think vocabulary is an area where ESL adults have particular difficulty in...because unlike native speakers of English who have the listening vocabulary, and so their problem is essentially decoding the word and then they can map their overall vocabulary to the text that is before them. Oftentimes, ESL adults do not have the overall vocabulary, so they have to learn that in addition to learning the...how to read the words and so it's a little bit more difficult for them to do that. So I would say vocabulary is a particular area where ESL adults can benefit from. .."
National Assessment of Adult Literacy
"By comparing results from 1992 and 2003, NAAL provides the first indicator in a decade of the nation's progress in adult literacy. NAAL also provides information on adults' literacy performance and related background characteristics to researchers, practitioners, policymakers, and the general public."
-Brian
Q: "We marketed these classes through various venues, but attendance has been too low to continue the classes. Any suggestions on what they can do?
A: Brian Bosworth: "Well, I think my quick suggestion and I hate to sound like a broken record is that offering the instruction in the workplace in partnership with the employer where most of these folks are for 40 hours a week, isn't a bad strategy in terms of securing participation and securing attendance."
National Institute of Literacy, discussion list
-Literacy in War and Peace
Tom Sticht's Work in Adult Literacy Education
By Thomas Sticht (from nald.ca)
"...In the winter of 1967 I got a call from a friend working in Monterey, California at a field unit of the Human Resources Research Office (HumRRO) of the George Washington University. He told me the military was joining President Johnson's War on Poverty by inducting 100, 000 new recruits a year who were low in reading ability. HumRRO thought it might be necessary to do more teaching by "show and tell" so that the New Standards Personnel (NSP, as they were called) could learn by listening rather than by reading. With my background working with blind students, HumRRO thought I would be a good person to do research on listening and reading abilities of the NSP..."
..In 1992, I was invited to London by what is now the Basic Skills Agency (BSA), to speak at its national conference. Princess Anne, the Patron of the BSA attended and sat through my presentation which emphasized the importance of getting "double duty dollars" through the intergenerational transfer of literacy from parents to their children. Newspapers reported that the Princess Royal found these ideas quite sensible. I worked with the BSA for several years on their national family literacy program.
In 1994 I was honored by being elected to the Reading Hall of Fame and in 1997 Timothy Shanahan and Susan Neuman, writing in the Reading Research Quarterly , selected the work colleagues and I did in the late 1960s and early 1970s at HumRRO as one of the thirteen most influential lines of literacy research since 1961.
Reach Higher, America: Overcoming Crisis in the U.S. Workforce...
Related Sites:
pdf format
Foreward
"..Lee Kwan Yew, the first Prime Minister of Singapore and architect of its economic miracle over the past half century,
once reasoned that a healthy economy depended on four fundamental requirements: dependable electrical power,
clean potable water, world-class transportation, and an educated and trained workforce. In our current global
economy, we can no longer take these precepts for granted."
....., from national commission adult literacy
".. calls for a dramatically revamped service system with the
capacity to effectively serve 20 million adults annually by the year 2020..
Section A. FACING THE PRESENT
2. Our Treacherous Path:
...Americans should have been stunned when the National Assessment of Adult Literacy (NAAL),
released in 2005, revealed that a staggering 30 million American adults scored at “below
basic”—meaning they could perform no more than the most rudimentary literacy tasks.
Another 63 million adults could perform only simple, basic everyday literacy activities.
The NAAL findings are ominous because most good jobs require at least some education beyond
high school. The NAAL found that of the approximately 222 million adults aged 16 or older living
in households or prisons in the United States, some 93 million lack literacy at a level needed to
enroll in the postsecondary education or job training that current and future jobs require...
[International Scale of Top H.S. Graduates-US #11; Korea #1]
3. The New Landscape
[National Demographic of States with College Diplomas-Texas #1]
This is a tall order, but it is what the U.S. job market demands. The Bureau of Labor Statistics
forecasts that between 2004 and 2014, 24 of the 30 fastest growing occupations will require
workers with postsecondary education or training...
Lack of Teacher Certification and Benefits. (pp 13)
High-quality instruction is essential to
foster student retention and produce successful outcomes. But salaries for adult educators
are low, and many positions lack benefits. This limits the pool of potential instructors and leads
to high turnover.
While many adult education instructors are credentialed for K–12 teaching, few states require
that adult education teachers show mastery of the specialized knowledge and skills needed to
teach adults. Moreover, states have not established certification systems to ensure that all
instructors have this ability.Worse yet, the majority of adult educators nationwide are working
part-time, often as a second job. This is another clear sign that adult education is not considered
a serious professional enterprise.
-David J. Rosen
"World Education, Boston, MA
International consultant in non-formal education to the government of the Philippines. Sponsored by the Asian Development Bank and the Royal Norwegian Government. Helped to develop national adult education system including elementary and secondary equivalency certificate alternatives. 2/95-3/95; 11/96-12/96; 6/97-7/97; 1/98-2/98; 7/98; and 2/2000.Project awarded UNESCO Literacy Project of the Year recognition in 2000. Quality Assurance Consultant on performance-based assessment, curriculum development, and teacher training, for school-based and out of school youth Integrated Production and Pest Management farmer field school project. July, 2004 - April 2005.
Consultant to the World Education Literacy Division on developing an intermediary role in Boston-based workforce development, 2004; conduct of scan of national adult literacy organizations for the National Institute for Literacy, April, 2005 - present; development of state Professional Development Distance Learning plan for the System for Adult Basic Education Support (SABES), November, 2005 - April, 2006.
ONLINE DISCUSSION:
Sal
"After watching (webcast form) and then reading the transcript (document form) of this report, I'm remembered as a child of easily understanding picture stories (e.g. quantitative literacy-71 million/32 % performed) instead of just words (e.g. prose-60 million) & (e.g. documents-48 million). Despite their data results concluding the adults did more poorly in the quantitative (using numbers) section, I feel math or working with numbers is complicated for certain groups.
The high percentage of only one language (Spanish) population that did poorly wasn't a surprise from my experience working with many migrant workers from Mexico. Also, the same goes with the population that didn't finish high school because of my experience with GED students in our ABE program.
The "Adults Just Don't Know How Stupid They Are" discussion was somewhat interesting, which I thought giving an international scope brought some enlightenment. The data on the importance of "apprenticeships" shows the importance of early programs (e.g. "OJT"-On the Job Training; Internships, etc..) in high school age level at least to college age level. I feel there is that different level of "pride" on every individual that we already know everything that we need to know on various subjects or skills (e.g. driving).
I feel it's the responsibility of "all of us" (researchers and adults themselves) on determining who is right about whose literacy levels pose a "problem". Instead of "criticizing" each other, we need to "work together" as a global society in literacy. Ever heard of the quote, "What we got here is failure to communicate" (used in a rock song from one of my favorite 80's band-Guns N' Roses)? I feel as ESL teachers or potential ESL teachers, we have a BIG task-role in teaching literacy to the "world". Thus, bridging different nations of different cultures of many ethnic backgrounds. Literacy is one way to communicate to each other, which can solve many of our social problems (e.g. wars that are somewhat caused by "misunderstanding" of one another)
An example of "working together" that's in the process in the area I'm teaching ESL as a volunteer is with our local dairy farm. They employ over 250+ "Spanish-speaking" migrant workers (many from Mexico) and they took the initiative (after hearing from the community) to start a "Spanish language" program for their "English-speaking" American workers. The guy that teaches "Basic Spanish" to them just moved up here from the Twin Cities the beginning of this year. Ironically, his wife was an ELL/ESL student in our program. That is how we found about this! They plan to hire a teacher to teach "English" to their "Spanish-speaking" migrant workers down the road.
Lastly, I would like to comment on the "Ohio Portfolio System", which I feel is a very good idea. I've learned somewhat about doing portfolios for our ELL/ESL students in the past. However, it is sometimes hard to do one when we are challenged with "retention" rates of our students. This is another "on-going" subject to discuss about later!
""arbitrary" literacy skills"-"hand on learning experience ( Newgate school here in Minneapolis)-Todd W.
' Algebra skills may not be part of my everyday life but I think the ability to 'think' mathmatically enhances brain function and math is a language..."-Jeannie M.
"...http://wdr.doleta.gov/SCANS/.... I believe SCANS was mentioned in one of our readings. At the workforce center, I had to make sure my students were achieving these skills. I developed a curriculum I called SCANS QUEST to help the supervisors and our student workers engage in tasks that supported SCANS.-Joyce
"In 1990, the Secretary of Labor appointed a commission to determine the skills our young people need to succeed in the world of work. The commission's fundamental purpose was to encourage a high-performance economy characterized by high-skill, high-wage employment. Although the commission completed its work in 1992, its findings and recommendations continue to be a valuable source of information for individuals and organizations involved in education and workforce development."
4.4 CASAS
4.5 Bests Tests
4.6 Accuplacer
Submit a 1 – 2 page paper in which you reflect upon your personal experience with assessing adult second language skills in light of the concepts from the class. What assessment methods and techniques do you use in your program and classroom? How can some of the concepts discussed in class and in the readings help you understand what you are already doing or what you could do in the area of adult second language assessment? Integrate the following into your reflective statement:
The assessment methods and techniques we use in our program and classroom are in various forms. For standardized testing, we use the CASAS (for ELL, ESL, ESOL students) and TABE (for GED students).
1. Why and when are your students assessed?
As I stated in the "Discussion Board" (Module 4.2), we usually assess them within the first 12 hours of their class visit...
"We use the CASAS, which we have trouble-challenges with the "pre-testing within the first 12 hours". Awhile back, when we met students for the first time, we would give them the CASAS test and they seem to be overwhelmed. Now, we decided to just wait after the first class visit or another better time (within the "first" 12 hours). That first day of class makes a huge impression on what this class will be like. Thus, this will impact their decision to return or not! Unfortunately, we sometimes never see (due to many factors-migrant work is "unpredictable") these students again and would never get the chance to give them the CASAS. At other situations, we do give them the CASAS and they would never come back (after 3-5+ more class days) and we will never have the chance to give the "post-test" to see how or if they ever improved to another level."
The purpose of these test as a "local" literacy program is for our "overseers" within the state...
"..Accountability: "Yes, the accountability or test results from NRS testing has motivated us teachers to do better for many personal reasons (e.g. keep the program going in this area for this county-wide community , job security for "paid-teachers", and want to see our local program and the region overall we represent to do well). It keeps us on our toes! .."
For our GED students, we assess them in a different way or standard...
"Our GED students are the ones that we usually focus on helping get employment and other various goals."
Overall our ELL/ELL/ESOL students are assessed differently right from the start compared to our GED students. We have them fill a "goal" sheet, which our "local program" gets credit if they do reach their goal. For example, the common goal listed for our ESOL students is "Learn English". For our GED students, it's getting their "GED Diploma" and/or "getting employment". Then throughout the course of the term or year, we write their individual goals and accomplishments each class session they participate on a separate goal sheet. These are some of the assessment tools and techniques we have been trained by the "state" for our "local" ABE program. The purpose of this is to show their "progress" and to show the individual learner/student what he/she has done so far. Therefore, to hopefully motivate them to continue to strive to reach that goal we both came to agreement in the beginning of the term/year. This informal form of assessment helps us "test-givers" or "teachers" to keep record on what we've done and what to do to help this invidual learner "reach their goal(s)". Also, to serve as data for the state in case we get "evaluated" for funding purposes. Along with this information, we have all types of other records (e.g. homework) for the individual students in a "portfolio" type file folder.
2. What assessment tools and techniques have you used? What was your purpose in using them?
(see above)
3. How do the tools and techniques you have used relate to the desired outcomes of your course or program? How would you describe the actual outcome?
The tools and techniques we have used has met both desired and undesired outcomes of our course or program. We "try" to keep improving it as we "make mistakes" and "learn". To first share some positive outcomes, we've been able to help students (GED) get their GED Certificate and employment. We've seen some progess of some of our ESOL students. For example, when we first "restarted" this program back in 2004, we had a good-size group of "migrant workers" from Mexico. One particular student name "Tulio" was one of the most ambitious, which he would come regularly. I remember taking him out to a community setting and introduced him to some people to informally practice his English. He stopped coming to our class one day and we didn't know why. Well, we found out he became manager of a particular dairy farm site! I personally feel what helped him reach this particular position was "researching" on the internet on resources related to his work. He was a dairy farmer and I found this "dairy terminology" site that translated English to Spanish. I would end up sharing this site with future "migrant workers" that worked on this particular field. He still continues to have this position to this day and he ended up inviting his family-relatives to come work in this area.
Some of the "undesired" outcomes are students that never reach their "goal" that they stated in the beginning of the year/term. We have one student name "Manuel" that kept coming regularly and stopped for about a 1/2-1 year. With our local classroom environment, we sometimes get "very busy" and forget or don't have time to "follow-up" with some of these students. Well, Manuel ended up coming back and would come regularly for awhile. However, we didn't see any "progress" based on his conversations with us. We can pin point to many factors that causes this, but we certainly feel that we could've done much better with this individual student. Some of other "undesired" outcomes is uncertain because we get so many students in the beginning, but they never finish or "come back". We will sometimes never know the outcome!
4. How would you characterize the tools and techniques you used in terms of the testing concepts presented in class?
I feel we have been using the tools and techniques already, but we just didn't know "how to use them" the right way all the time. For example, in Module 4.3 when we were assigned to read the results of the National Assessment of Adult Literacy (NAAL) 2003. I was able to related to what Dr. John Stucker stated...
"And then the other thing about these folks is, again, it's remarkable how fast these ELLs have picked up English conversation. But don't let that fool you. If you actually get a chance to assess their reading skills in English, it turns out they are extremely idiomatic when they talk to you. If one person walked in and joined us with this panel, we think gosh! That person speaks great English but when they went to do embedded literacy tests, such as the NAAL, those tests were hard for them.."
I had one student from Bulgaria that quickly learned how to conversate in English. However, when we e-mail each other (still to this day to keep in touch), I feel like I'm e-mailing a "totally different" person from who I had good English conversations with in person in the past! I just feel like I should've focussed more on the "Writting Assessment" when I was teaching him years ago.
If I took this class prior to being a "volunteer" teacher 4 years ago, I feel I would've been better equipped on teaching our ESOL students. In fact, I feel the retention rate of our students and volunteers would be better. I guess it's better late then never. Now I know from this mistake has taught me to work on all three types of assessments or four different area skills (oral-speaking and/or listening, reading, and writing) for each student instead of just spending too much time in one particular area
Overall, I feel some of the concepts (e.g. “placement,” “proficiency,” and “achievement”) discussed in class and in the readings has helped me understand what I'm already doing or what I could do in the area of adult second language assessment. This is by "application"-using what I've learned and put it into "action".
*Note: Below are chapters I didn't have to read, which I "accidentally" misread the assignment:
Chapter 4. "The Role of Background Knowledge in the Assessment of Receptive Skills" (Bailey)
Goodnewseverybody.com: Caucasian-Polish jokes
1. content schemata-"background knowledge that led you to predict"
-e.g. terrorist's mission, safari, ->fishing trip!
a. schema activator-"something that triggers the use of your existing schemata" (pp. 53)
b. advance organizer-"a shema activator; helps the listener/reader prepare to pose and test hypotheses about the natureof the incoming text" (pp. 53)
2. formal schemata-"background knowledge that led you to anticipate the structure"
-e.g. polish joke (culturally sensitive)
Chapter #5 "A Cloze-Knit Family"
pragmatic expectancy grammar-"the portion of our internalized language competence that enables us to predict likely sequences of incoming language, whether we are reading or listening
-e.g. NIGHT DROP VALLY PLASTERING
syntagmatic competence-"tells what part of speech is needed; syntax-rules of word order"
paradigmatic competence-"tells us the semantic features required of the missing item."
discrete-point approach-"limited context"
pretest
A. Cloze Tests
cloze passages-texts longer than a sentence-usually at least a paragraph in length
exact word message
acceptable word method
mutilation-"process of deleting words to create a close test"
rational deletion (e.g. delete verbs)
fixed ratio deletion/nth word deletion-every nth word is omitted
B. C Tests
"deleting parts of words
the rule of two-"individual letters of every other word hasve been replaced by blanks"
GoodnewsEverybody.com Asian-Sri Lankan elephants-"Sinhalese staircase"
think-aloud protocol-"...a protocol is something written down-minds of learners...
*"I believe the acceptable word scoring method is superior to the exact word procedure in terms of washback because it should promote creativity and risk-taking among the language learners"
practice effect-"results on the second test may be influenced for the test takers"
"counterbalancing for an ordering effect-"altering the ordering of administration"
field independence-"..perceive part of a field as discrete from the surrounding field as a while, rather than embedded, or...the extent to which a person perceives analytically"
Chapter #6 "Another Dilemma: Contrasting Pairs of Concepts"
discrete-point testing-"each item on a test is intended to measure one and only one linguistic element
-e.g. a. chip b. ship c. cheap d. sheep
integrative testing-one or more levels of language and or/more skills
objective scored
rater judgments
subjectively scored-value judgments
editing task-identifying the portion of a text that contains an error
statistical procedure-systematically investigate a classic problem in language assessment
prompt-a topic for an essay taks
rater drift-raters scores drifted away from the scale
5.1
Chapter #4 "Oral Language Assessment" (pp. 57-81 O'Malley)
*see GoodnewsEverybody.com Class Tips for Selecting Assessment Activities
DISCUSSION BOARD
Author: F, Michelle
Other Web sites to begin with from Summer 2008 Online
Subject: Job Corps Website
"
Simply by googling "adult ELL assessment" I found the Job Corps website. It is well laid out, clear, and comprehensive. For me, there isn't much new information. I hope some of you might find it useful. I'll keep it in mind for all the times I'm asked to explain how to teach ELL by people new to teaching about to go teach an ELL class by themselves."
Author: W, Elizabeth
Subject: Literacy, reading and the internet
"
Here is an article about literacy in general among teens. It raises the question whether the internet is helping or hurting literacy - standarized test scores are flat or declining although giving internet access to students in low-income groups did raise their scores. There is also talk about a digital reading component on standardized tests - other countries are doing this. Another question raised is do we need to teach internet reading skills? An interesting article!"
Author: S, Cecile
Subject: web assignment
"Attached is my assignment on SABES, System for Adult Basic Education Support. This site contains some good information for teachers of adult ELL students.
Author: L, Danielle
Attachment: Web 6638.doc (28160 bytes)
Subject: Danielle: NCSALL website
"
Hope you find this information useful!"
Author: R, Joyce
Attachment: Robles_Web_Assign.doc (33280 bytes)
Subject: Joyce_Practioner_Toolkit
"
The site I describe is owned by the State of Michigan; however, the information is not state specific. If you work in a program with Spanish speakers, you might find the needs assessment samples helpful. This is an opportunity to look out how accessible a pdf can be--I was surprised!"- Joyce
Author: S, Christina
0
Attachment: Solomon_C - Web Assignment.doc (37888 bytes)
Subject: European Perspective
"Hi all,
found this site from the University of Cambridge website. It was interesting to get a different point of view.
Thanks-CS
Author: F, Patricia
Attachment: ESL 6638 Web Assignment.docx (14688 bytes)
Subject: Virginia Adult Learning Resource Center- Patti
"
Think outside the box with me on this one. I was amazed at how I perceived this material since gaining more knowledge of assessments in this class.
It is an ESOL Starter Kit document, but it is chock full of resources and material that could be used for authentic assessment as well as information about standardized and authentic assessments.
Danielle, check out the "Teaching ESOL Using Word Processing" and the activities within that section (p.237) for ideas with your final project.
Author: L, Anne
Attachment: ESL 6638 Web assignment.doc (27648 bytes)
Subject: web assignment
"
This website deals mainly with migrant students, but the information can be applied to all ELL's.
Author: S, Caitlin
Attachment: ESL2 Web Assigment.docx (26227 bytes)
Subject: Practical Ideas on Alternative Assessment
"
I chose a web article entitled "Practical Ideas on Alternative Assessment" sponsored on both the CAL and NCSALL pages. Please take a look!
Author: Meisner, Ken
Attachment: MeisnerWebAssignment.doc (53760 bytes)
Subject: Accuplacer Preparation
"
I found this site by googling "Accuplacer". It was the first one on the list.
Author: R, Rallou
Attachment: Web assignment 6638.doc (26112 bytes)
Subject: Web assignment
"Attached is the website I found."
Author: W, Elizabeth
Attachment: will_e_webassignment.doc (41472 bytes)
Subject: performance based assessment
"
I looked at the REEP curriculum because I was interested in performance based assessment."-
liz
Second Language Assessments
"offers the world's leading range of certificates for learners and teachers of English - taken by over 2 million people in 130 countries. They help people gain entrance to university or college, improve job prospects or measure progress in English.."
Chapter #5 "Reading Assessment" (pp. 93- O' Malley)
Chapter # 6 "Writing Assessment" (pp. 135-)
Miscellaneous
5.2
Reading
graph·o·phone /ˈgræfəˌfoʊn/ Pronunciation Key - Show Spelled Pronunciation[graf-uh-fohn] Pronunciation Key - Show IPA Pronunciation (dictionary.reference.com)
–noun
"a phonograph for recording and reproducing sounds on wax records."
5.3
#1-"I Like.."
(Purpose: "Expressive Writing" pp. 137 O'Malley)
-New Mexico Portfolio Writing Assessment Scoring Key (pp. 11)
"1-Sentences are simple, short, and repetitous"
-Fairfax County Public Schools: ESL Student Writing Development Scale (pp 15)
"3-Writes Sentences centered around one idea"
-My Child as A Language Learner: Parent Obeservation (Grade 4-Grade 6) (pp. 16)
-Sample Criteria for Holistic Writing Task Assessment (pp 17)
"2-many elements missing"
-Sample Peer Writing Checklist (pp. 18)
"Ideas & Content-Main Idea Stands Out (developing)"
"Sentence Fluency-Sentences Logical" (achieved)
"Sentence Fluency-Sentences Vary in Length (developing)"
"Voice-Strong Sense of the Person Behind the Words (developing)
"Conventions-Grammar and Usage is Correct (developing), Punctuation is Accurate (achieved), Spelling is Correct (developing),
I would suggest to this writer using a "Holistic Scoring Rubric" (e.g. O'Malley pp. 143) that he/she should "Begin to write a paragraph by organizing ideas" Level 3); however, the intent of this writer might be a different format (e.g. poem).
#2-"Park, Cows, NY"-from Russian
(Purpose: "Expository/Informative" pp. 137 O'Malley)
-Writing Self Assessment Questionnaire
-Developmental Scale of Writing for English Language Learners St. Paul Writing Rubric (pp.
9)
-New Mexico Portfolio Writing Assessment Scoring Key (pp. 11)
"2-There is basically good sentence structure with awkward or confusing constructions"
-Developmental Scale of Writing for English Language Learners St. Paul Writing Rubric (pp.
13)
-Fairfax County Public Schools: ESL Student Writing Development Scale (pp 15)
"4-Uses some compound and complex sentences
-Sample Criteria for Holistic Writing Task Assessment (pp 17)
"2-Competent/Acceptable"
-Sample Peer Writing Checklist (pp. 18)
"Ideas & Content-Main Idea Stands Out (achieved)"
"Ideas & Content-Writing Reflects Own Thinking & Experience (developing)"
"Organization-Smooth Transitions (Developing)"
"Voice-Strong Sense of the Person Behind the Words (developing)
"Conventions-Grammar and Usage is Correct (developing), Spelling is Correct (developing)
I would too suggest to this writer using a "Holistic Scoring Rubric" (e.g. O'Malley pp. 143) that he/she should "Begin to write a paragraph by organizing ideas" Level 3). Add at least 3 sentences in each of the three topics to form paragraphs.
#3-"Giants in America?-from Laotian"
(Purpose: "Expressive/Narrative Writing" pp. 137 O'Malley)
-New Mexico Portfolio Writing Assessment Scoring Key (pp. 11)
"3-All Sentences are complete"
-Developmental Scale of Writing for English Language Learners St. Paul Writing Rubric (pp. 13)
-Fairfax County Public Schools: ESL Student Writing Development Scale (pp 15)
"5-Writes a retelling of personal experience"
-Sample Criteria for Holistic Writing Task Assessment (pp 17)
"2-topic and purpose unclear"
-Sample Peer Writing Checklist (pp. 18)
"Ideas & Content-Main Idea Stands Out (achieved)"
"Ideas & Content-Writing Reflects Own Thinking & Experience (achieved)"
"Word Choice-Words Convey the Desired Message" (achieved)
"Organization-Details Logically Placed" (achieved)
"Organization-Overall Structure of Paper Flows Well (achieved), Own Thinking & Experience (Developing)"
"Sentence Fluency-Sentences Logical" (achieved)
"Voice-Strong Sense of the Person Behind the Words (achieved)
"Conventions-Grammar and Usage is Correct (developing), Spelling is Correct (developing),
I too would suggest to this writer using a "Holistic Scoring Rubric" (e.g. O'Malley pp. 143) that he/she should improve on "Writing with grammatical/mechanical errors that seldom diminish communication" Level 4). This sample probably would be the "best" of all 3 and would serve as a "benchmark paper/anchor paper", a "classic example" (Chapter #12 "Three Approaches to Scoring Writing Samples" pp. 189 Bailey) for other students to model withing the 3 samples.
Overall, I feel the last rubric "Sample Peer Writing Checklist (pp. 18, which is "somewhat" similar to pp. 158 O'Malley) is the "best" to use for all 3 samples above. My personal reason is that the checklist format makes it easier to use for assessing these types of papers. Also, it provides the student some positive feedback (e.g. "achieved"-similar to the "analytic scoring" section in pp. 144-145 O'Malley) for the "evaluator" (teacher or peer) to use. It would be nice to put a "comment" section (like on Figure 6.3 pp. 146-147). As you can see above, there is not just only "one" particular style rubric to use. I would use a variety of scoring traits from each style rubric and mix them together for that particular assessment I would focus on. I would recommend that each writer for each sample above use a "Process Writing Checklist" (pp. 149 O'Malley) with a student-teacher conference (pp. 138 O'Malley). Later, have them use a "Writer's Checklist" (e.g. "Did you correct errors in spelling, capitalization, punctuation, and usage?" pp. 141 O'Malley).
Subject: Re: Checklist(s): Example-Sample Peer Writing...
"
Sal,
Wow! I appreciated all of those rubric samples within the reading, but you out did yourself. Way to go. Great application of the material. Thanks for applying it to the assignment and sharing with us.- joyce
Author: W, Todd
Subject: Re: Checklist(s): Example-Sample Peer Writing...
"
Sal, I liked your statement: As you can see above, there is not just only "one" particular style rubric to use. I would use a variety of scoring traits from each style rubric and mix them together ......
Matching the assessment tool to the context, content, instructional objective, purpose (e.g. feedback to the learner), etc.-
Todd
Author: W, Elizabeth
Subject: Re: rubric website
"
I found this website while searching for one for the web assignment. I only looked at it quickly. In order to change anything on the pre-made rbric you have to paste it into your own document.
http://www.teach-nology.com/web_tools/rubrics/
5.4
#4 Methods of Common Approaches to Assessing Reading:
1. Curriculum-Based Measurement- (CBM) "..is a method teachers use to find out how students are progressing in basic academic areas such as math, reading, writing, and spelling...
For example, in reading, the child may be asked to read aloud for one minute. Each child’s scores are recorded on a graph and compared to the expected performance on the content for that year. The graph allows the teacher, and you, to see quickly how the child’s performance compares to expectations....-from studentprogress.org
2. miscue-analysis-".. originally developed by Ken Goodman for the purpose of understanding the reading process. It is a diagnostic tool that helps researchers/teachers gain insight into the reading process.
The term miscue was initiated by Ken Goodman to describe an observed response in the reading process that does not match the expected response. Goodman uses the term "miscue", rather than "error" or "mistake" to avoid value implications. He states that the departures from the text are not necessarily a negative aspect of the reading process but rather “windows on the reading process” (Goodman, 1969, p.123)."-from Wikipedia.org
*see Chapter #5 (pp. 124-125 O'Malley)
3. Multiple Choice
4. Recall
a. decoding
b. comprehension
c. both
-1.
b.
-2.
b. As O'Malley states in Chapter #5 (pp.124)-"..provides information of at least three types: (1) the reader's ability to use language and the reading process, (2) the reader's approaches to reading and reading comprhension, and (3) information for revising instructional approaches and materials. Miscue analysis involves listening to a student read aloud as well as having the student retell the story or answer comprehension questions."
-3
b. Multiple Choice is just one type of strategy that can be used to help an ELL/ESL student in their reading comprehension. O'Malley states, "...finding the main idea; identifying sources of information needed to answer a question..pp. 95" (if given the "right" multiple choice question) -can increase reading comprehension by these types of teaching comprehension strategies.
-4
c-both
Overall, an ELL/ESL student needs some "..prior knowledge and match it to information in the text", which is schema activation (O'Malley pp. 96-97). That sample under that Multiple Choice would be very difficult to "decode" if an individual ELL/ESL student has no prior knowledge of "Fussball", which can be translated to "foosball" (table with 8 rods...), "football", or even soccer (called "futbol" in many countries outside of the United States of America). In fact, I have to state "American football" instead of plain "football" to my ELL/ESL Students because they get it mixed-up with soccer. What was the right answer anyways to that Multiple Choice example-my guess was A. Jahre (what was this?, did anybody have any luck decoding this?). Lastly, that multiple choice example looked so much like "German"!
DISCUSSION BOARD:
Author: R, Joyce
Subject: Joyce_Practioner_Toolkit
"
The site I describe is owned by the State of Michigan; however, the information is not state specific. If you work in a program with Spanish speakers, you might find the needs assessment samples helpful. This is an opportunity to look out how accessible a pdf can be--I was surprised! Joyce"
http://www.nytimes.com/2008/07/27/books/27reading.html?_r=1&hp&oref=slogin
Literacy Debate: Online, R U Really Reading?
".. As teenagers’ scores on standardized reading tests have declined or stagnated, some argue that the hours spent prowling the Internet are the enemy of reading — diminishing literacy, wrecking attention spans and destroying a precious common culture that exists only through the reading of books.
But others say the Internet has created a new kind of reading, one that schools and society should not discount. The Web inspires a teenager like Nadia, who might otherwise spend most of her leisure time watching television, to read and write.
Author: W., Elizabeth
Subject: Re: Getting Started
"
I just went and looked up "'decoding'. The definition includes letter-sound relationships. sounding out the words in the German text does help you relate the word to the English equivalent. I also got the right answer not just by comprehension because I don't know what 'gern' means even though I do know what 'spielt' means. I did use more than just guessing. I matched the words in the question to the place in the text where they occured. What is that called?"-liz
5.5 Cloze Test
*see Ch.5 pp. 114-115 O'Malley
What I need to do to fill out a cloze test correctly is:
Know the terminology of the "exact word", which I would need as much information from the test giver or teaching assessing this particular test. As O'Malley states, "At the instructional level, students can proceed but only with assistance from a teacher." (pp.114). She recommends that this type of assessment be used for "intermediate-and-advanced-level readers" (pp. 114). Also, "Students need practice with the cloze in whole class and group settings before being asked to work individually", which she gives advice on how to go about this on page 118.
I think that language assessment people like having this instrument as part of the assessment tool kit because it gives the student to think more and be creative. It challenges the student to exercise their cognitive skills. In chapter #5 "Reading Assessment" by O'Malley, she states-"Teachers can use cloze procedures to determine which students need extra help or easier material..".
I personally did not "fair well" taking the test-got within the "instructional level". A couple of my answers were "close" to the "exact-word" (e.g. Q#8 start->try, Q#14 filling->filled). I would've done better if this was an "acceptable word scoring method" (pp. 71 Bailey-*wasn't required to read this chapter, but accidentally read it when I misread the assignment earlier this week), which Bailey has some interesting findings on comparing these two (exact & acceptable word method) on pp. 71.
Samples:
5.6
DISCUSSION BOARD 5 Module
6.1
DISCUSSION BOARD:
-Topic: Forum: 6.1 TESOL Standards
1. Is the list of 16 standars comprehensive, ie. did they miss anything?
I thought they did a great job on this and was well written. I've been in this field (volunteer teaching ESL-GED for 4 years this coming Fall) and I'm still learning more each day/week. I'm not sure if I missed it, but it would be nice to add "Real Life-Skills" as a way to assess our ESL/ESOL students. Throughout these 5+ weeks in this on-line course, we've discussed about meeting the "needs of these students". I don't remember the exact week, but we read about the students learning "basic life-skills" for survival. For example, being able to communicate their health problems in case of a medical crisis when visiting a hospital. Going around the community asking for directions to the post office, grocery story, library, or even the police department. Knowing what to do in an emergency (e.g. call 911). I guess the closest standard I could see this fit is with letter K.."Assessment activities document learners' progress within the ESL program toward advancement...of other educational goals." What do you all think?
2. Does the format, measures and sample evidence for each section provided enough guidance to evaluate a program?
I believe this was well-done and can't really see any improvements when it comes to the "measuring" part.
3. For those of you who are working or have worked in an adult ESL program, does your program address all of the standards and, if not what's missing?
Well, this question might be a little personal for our local program, but we might have similar "challenges" (some listed below) that we can all share to help one another:
-I would love to see an improvement in computer technology if possible in our area. This would be under 1. Program Structure and Administration G. (pp. 66). The program provides equipment for daily operations and efficient record keeping. We do have the computers (e.g. labtops) to use our Rosetta Stone and Marshall ABE software programs; however, they are not hooked up to the internet. With the high-tech media technology out there, it would be more "interactive" to use some online programs out there I've found on the internet. Thus, I would like to see "internet availability" under "_1 The program provides basic..._ computer-based management information system". Just sharing my personal preference of interest from past experiences in this program.
-Also another lack of equipment challenge is a "_tape or CD player" (pp. 67), which in the past we would like to hear some different cultural music (e.g. Barnes and Noble has a good diverse selection of cultural-ethnic music, which I bought years ago; however, misplaced them??) our students (an me also) has had an interest in sharing. I guess, I can bring my own personal "boom box", so this isn't really a big equipment issue.
-Under "J. The program maintains a learner-teacher ratio conducive to meeting learning needs and goals" (pp. 69)-our local on-going challenge is "_4 The program recruits classroom aides, paraprofessionals, or volunteers to improve the learner-teacher ratio as needed". In the beginning part of the school year, we get a lot of volunteers from the local university, but they get so busy (they attend a "highly intense academic college"-I should know because I graduated there and never had time for outside activities) throughout the year. Thus, we are left with my supervisor and I (volunteer coordinator) trying to teach GED and ESL students in the same time (once a week at the elementary school and another day of the week is just ESL students). Sorry, I already shared this during one of the discussion boards during this course-don't mean to bring this up again. However, after learning what we've learned in this 5+ week course, I plan to do some changes. I hope we can better this program to retain both our volunteers and the students we are teaching.
I'll stop here as there will be too many to share, but just to give you all some of the on-going challenges that we hope to overcome and improve our local program as we move forward in the future. Some of these standards don't apply for our local "small" program, but for the ones that oversee us.
Date: Sun Aug 03 2008 16:58
Author: R, Rallou
Subject: Re: Program Self Review
"
In our program they have only three paid teachers and they rely heavily on volunteers. This is I believe because funding is limited. There is an increasing demand in ESL and I hope that students don't turn away because of the different teachers they might have during the week. I believe in consistency and I think students tend to get familiar with a teacher versus another. Such a change might result in leaving the program. The volunteers receive continuous training but I do believe that a trained ESL teacher could make a difference in the classroom. "
6.2
6.3
As a test taker, you have the right to:
4. Receive a brief oral or written explanation prior to testing about the purpose(s) for testing, the kind(s) of tests to be used, if the results will be reported to you or to others, and the planned use(s) of the results. If you have a disability, you have the right to inquire and receive information about testing accommodations. If you have difficulty in comprehending the language of the test, you have a right to know in advance of testing whether any accommodations may be available to you."
The Rights of Test Takers:
Guidelines for Testing Professionals
Test takers have the rights described below. It is the responsibility of the professionals involved in the testing process to ensure that test takers receive these rights.
4. b. "Tell test takers, prior to testing, about the planned use(s) of the test results. Upon request, the test taker should be given information about how long such test scores are typically kept on file and remain available."
6.4
DISCUSSION BOARD:
-Other 6 Module Issues:
Author: F, Patricia
Subject: an application of what we've learned
'A front page headline and article in today's (Aug. 2) Chicago Tribune has given me an appreciation of what I have learned during this class. Six weeks ago I would have glanced at the headline ("Errors may snarl state testing") and quickly scanned the story. Yet another story about the woes of the Chicago Public School System, and there are many, but what does it have to do with me and much of the information I don't understand anyway."
Today I read the entire article and understood it!!
Here is a link to the article.
http://www.chicagotribune.com/news/nationworld/chi-state-test-scores-02aug02,0,1406470.story
It isn't very long and I believe you will find many applications of what we have learned about the pitfalls of standardized testing as well as the ongoing debate as to their value in the education system.
Thank you Todd for sharing your knowledge of this subject with us. Even if I were not using it as a teacher it has enriched my life in that I can read an article like this one and have a greater understanding of the content. That is a wonderful feeling.
OK. I can understand the article but my computer skills still stink. The link doesn't work for some reason.
Google: Stephanie Banchero Chicago Tribune aug 2 2008
Hopefully that will bring the article up. I apologize for my lack of computer skills.
Subject: Free Literacy Program Courses Online
"
Hi Everyone,
I don't know that I can endorse this web site, but it is worth investigating. It offers a number of "courses" to help literacy volunteers, teachers, and administrators. My preview made me think it might help out those who are trying to maximizing the use of volunteers. Please share your opinions, if you have time. - joyce
http://literacynetwork.verizon.org/About-Thinkfinity-Literacy-Net.175.0.html
1. Have any of your beliefs regarding assessment and testing changed as a result of participation in this class?
This 6 week course has definitely influenced my beliefs regarding assessment and testing. I never really saw this to be a complicated issue before. It feels like it was a long time ago, but 4 weeks ago during Module #2-I was "skeptic" or "pessimistic", but a little "optimistic" too about testing overall. I remember the section on "Doubters and Believers" (good theme for this particular topic!) on the "Discussion Board", I was "undecided"! Now, I feel more "optimistic" on assessment and testing as I know more on what purpose it serves. Just like the saying, "the fear of the unknown"! As it’s fresh in my memory, the standards listed in "Part 4 The Program Self-Review Instrument" was very well-written and concise. However, as I stated in the "Discussion Board" (Module 6)-I feel the "head administrators" should consider a way to assess "real-life skills" to be included in their evaluation on how a ELL/ESL/ESOL student is progressing.
2. Discuss your understanding of the strengths and weaknesses of various assessment approaches and there applicability to the settings and assessment purposes with which you have experience, e.g., standardized testing, various kinds of performance assessments. With regard to the CASAS and BEST, consider these questions:
I've had experience only with CASAS, so I'm able to share only on this particular standardized testing. According to the "content standards" in CASAS...
"Since its inception, CASAS has focused on teaching and assessing basic skills in contexts that are relevant and important to adult learners. CASAS has developed and continues to refine a highly formalized hierarchy of competencies, the application of basic skills that adults need to be fully functional and productive members of society."
I've administered this test many times in the 4 years (this coming Fall) as a volunteer in the local program I'm in. I never knew that they had "360 essential life skills" that are identified by the CASAS Competencies. As I stated in the "Discussion Board" for Module 4.4...
"Wow, after 4:4 "The CASAS System" section, I have a higher respect on this test assessment. As a volunteer being trained on this, I really never "fully" understood what this assessment was all about until reading this particular section. I have a lot of "homework" I need to do now when I do give this test to our ELL/ESL students, which I hope this will improve our program-thanks!"-Sal
However, the "weaknesses" of this test is not being able to communicate with the students "orally" about what's covered in this test. They label their test "Life Skills", but it lacks the interaction-meaning, they are reading these tests, but not "acting-upon" what's given. For example, I took a sample that was around my house. In pp 10 of Form 27 (Life Skills: Reading-Beginning Literacy), question #28 states "No left turn" [with 4 different given signs, which you have to choose one]. This seems simple, but would the student remember this in a "real-life" situation. Here in Morris, we have only 1 one-way street. Unfortunately, our city doesn't have a sign exactly like the one in question #28. Instead, thy have a sign that says "No Enter". This particular question may not be applicable in this region or Minnesota; however, it maybe in another region (e.g. Twin Cities). What I'm trying to demonstrate is that some of these questions in CASAS will never be applied in their daily-life or if it does, they may not remember. Unless, the teacher (me) emphasizes this particular point or question. After just doing this particular question, I feel it's important for "us" ("test-givers", teachers, etc..) to go over these questions after we administer these ELL, ESL, ESOL students. Especially the ones they get "wrong" (e.g. date order-today 8/4/08 or August 4th of 2008; however, many other countries like Mexico has is ordered differently. Thus, I teach or "correct" many of our students that get this wrong.) After this test review, I feel CASAS (still feel "Massachussetts, Curriculum Framework for ESOL" version was more "culturally friendly") will serve its purpose in "teaching basic skills" as stated in it's "content standards". The "test creators" need the "test givers" to work-together in "teaching" the "students" these basic skills that CASAS was created for. What was listed as a "weakness" for CASAS can be "eliminated" if we "test-givers" do our part too. One of the "strengths" CASAS particularly has is it's "Quick Search Tutorial" site where users can "login" and find various materials to equip "test givers"...
""QuickSearch Online has many exciting features. A bold, colorful interface helps users to select from a wide variety of library searches. This year's edition allows "quick searches" for instructional materials by:
* Skill area: reading, math, listening, writing, speaking, critical thinking, and government and history
* A wide range of programs (including Civics, Family Literacy, Workplace)
* Specific CASAS assessment series, i.e. Life and Work, WLS, ECS
* Year of publication of instructional materials
QuickSearch Online can also search for materials that include informal classroom assessments.
The QuickSearch Online Tutorial is now available from the main menu, making it easier to learn how to use QuickSearch Online and to train new staff. Also, several reference tools appear in the Reports and Other Tools menu; for example, test competencies for every CASAS pre- and post-test. " Tests Overview"
Also, just recently taught this week (Module 6.2) is the "Guidelines for Providing Accomodations Using CASAS Assessment for Learners with Disabilities" is another good resource CASAS provides.
Related Sites:
3. What assessment principles and concepts are reflected in this/these test(s)?
Some of the assessment principles and concepts that are reflected in this test are stated in their competencies and standards website given to us in Module 3.3:
-Multiple-choice tests "measure basic skills in
a functional context for adults and youth and are constructed from item banks of more than
5,000 test items." (pp. 3)
-"The psychometric methodology used to establish this difficulty level
comes from the Rasch model of Item Response Theory (IRT)" (pp. 3)
-The test measures all four of the basic skills: "reading, writing, listening, and speaking" (pp. 4)-along with math.
4. Why (or why not) do you feel the CASAS and/or BEST test is an effective assessment tool for adult second language learners?
I wasn't convince enought that CASAS was an effective tool until this class when we read many materials on this particular test. Just to note, I have never had any experience with the Best test in our local program. I feel I didn't have any confidence on this type of standardized testing because I wasn't informed enough on what this test was all about. As a local (in Morris, MN) "volunteer teacher" for the last 4 years (Fall of 08'), I did have regular training under the Minnesota Literacy Council at Alexandria; however, I wasn't "trained" much on this. They probably gave me the opportunities to do so when I wasn't aware of it, but I didn't take advantage of it with my other obligations (e.g. my career working with developmentally disabled adults since graduating from college back in 1999) in life. With this online course dedicated to get the ESL Teaching Licensure Certificate, my desire has opened my interest to learn more and to get equipped as a "volunteer teacher" in ESOL.
Getting back to the CASAS topic, I feel they have the resources (e.g. online) to really equip "test givers". As I recently wrote in the "Integrative Paper #1" assignment, I stated the benefits of it so far in my 4 years of experience giving this test. I feel this test can be more effective, as originally the "creators" intended it to be, if "test givers" are informed on "how to use it". For example in Paper #1, I shared a story of a student from Bulgaria that began to speak English very well through the many tools (e.g. Rosetta Stone-skills targeting reading, listening, and speaking) we use to teach ESOL to him. However, he would write to me in e-mail (still to this day), but his writing comprehension would not match his oral comprehension. If I took this online class ("Assessment") prior to being a "volunteer" teacher 4 years ago, I feel I would've been better equipped on teaching our ESOL students. Particularly knowing now and getting informed on the available CASAS resources (e.g. "Quick Search Online" that I mentioned above earlier answering question #2). In fact, I feel the retention rate of our students and volunteers would be better. I guess it's better late then never. Now I know from this mistake has taught me to work on all three types of assessments or four different area skills (oral-speaking and/or listening, reading, and writing) for each student instead of just spending too much time in one particular area.
The CASAS really did not impact "my way" of doing the local program’s curriculum or my instructional objective because I never really looked into the "purpose" and "goal" of this particular assessment. After taking this online course, it will definitely or is affecting the way I do the local program overall. We are using the Rosetta Stone, but we are implementing other variety "instructional objectives". I mentioned this change earlier in the "Discussion Board" that we started in the beginning of 2008 after re-evaluating ways to improve our "retention rate" of our ESOL students. However, after taking this online course, I'm going to encourage my supervisor to focussed other basic skills and it's target-such as, the writting assessment section. I hope to use the many sample "Writing Assessment" tools (I gave some specific ones I would like to use in the "Discussion Board" under Module 5.3) given in the Module 5.3 section.
5. To what extent do the CASAS and/or BEST test assess authentic language proficiency?
The CASAS test assess authentic language proficiency or "direct tests" through their multiple choice questions measuring their reading skills. However, it being a paper-work test, it doesn't measure one's oral skills (this is where software programs like Rosetta Stone comes into play).
Related Sites:
6. How might the CASAS and/or BEST test(s) impact your program’s curriculum or your instructional objectives?
(stated above)
7. How will what you’ve learned change the way you have been or will use assessment in your teaching practice?
I'll focus more on the goal of using all 4 basic skills (speaking, listening, reading, and particuarly-writing). Then apply this to the ESOL student's "need assessment" (e.g. "real-life skill" settings they want to use English-home, work, school, community, etc..) base on my new strategy I thought of during this 6 week-online course, which is doing a background interview questionaaire (e.g. interests/hobbies, skills, educational background, work, etc..) with each individual student. I would like to do this with each student I meet on the "first-day". I will probably add more to their "portfolio" folder (e.g. goal sheet) to keep track of their progress during our classroom sessions.
8. What are you beliefs regarding the use of assessments for various accountability purposes?
I would say my beliefs has changed throughout this 6 week course. In the beginning, I wasn't very too confident about it as I mentioned in Module 2 after reading the given assignments.
9. What has been the most useful content on assessment that you have learned in this course?
I really can't narrow the the exact "most useful" content on assessment that I have learned in this course. Being a volunteer teacher for the last four years (Fall of 08'), I haven't really been focus with a desire on learning assessment-until now! However, being a "computer online junkie" (e.g. make personal websites as you notice in my "Participant Info-Create Homepage" we filled out during Module 1-1st Week), I feel it would be all the interactive online sources as I'll share next.
Will you apply it in your teaching? If yes, how?
I would apply this by going back on my past assignments. For example the "Web Assignement", which I did it on "Teaching Tips: Writing Assessment Tools" from emints.org (based in Missouri). As I stated in this particular assignement, I joined an online group (similar to registering with the "Quick Search Online" for CASAS) to get resources to equip myself on other assessment tools (particularly the basic "writing" skill) to use on my ESOL students. Overall, I will be regularly looking back at the notes I took on a special online site I developed throughout this six week course at:
Liberal Arts: English-Assessment http://liberalarts.goodnewseverybody.com/english.assessment.html
Assessment Context • What is the purpose(s) of the assessment?
The purpose of this assessment is to see how students are improving in the "four-different area skills" (Listening, Speaking, Reading, & Writing). Thus, to help them feel more confident in applying them in their "real-life" settings (e.g. workplace, home, community, etc..). For this particular exercise, we'll focus on one setting, which will be visiting a hospital/clinc. The student will be assessed on how he/she does in the "pre-admission" (registration form) process, for reasons stated as such:
"The more you can read and fill out ahead of time, the easier the admitting process will be. It will also give you more time to digest the information and to ask questions and get answers. The hospital also has copies of rules, regulations, and statements of patient rights for you to read. Many hospitals now make these forms and information available through their Web sites. Contact the hospital admitting or administrator's office to find out the easiest way for you to get this material in advance."-CalHospitalCompare.org
As a member in committees (e.g. COPC: Community Dialogue) serving our small-town community, I hear many "cultural or language barrier" issues from various services (e.g. hospital, police, social services, etc..). I'm hoping an assessment like this will build confidence of our ESOL students to fill out all types of registration forms (e.g. Money Gram to send money, job applications, Citizenship forms, etc..).
We hope this would be given as both a summative and fromative placement test to inform both student and teacher.
*taken from Module 3.6 Student-Pair Assignement
• What kind of assessment/test is it to be? proficiency, diagnostic, placement, curriculum based, standards or competency based
The type of assessment this will be is language proficiency. To see how all language "basic" skills (Listening, Speaking, Reading, & Writing) are performed.
• What abilities/skills/instructional objectives are to be assessed?
Listening & Speaking
I'll read the evaluation/assessment of their conversation with the person behind the hospital desk during the "admission" or "registration" process. I'm hoping the hospital will fill it out and give me any feedback.
Speaking
I'll watch and evaluate their conversation with the person behind the desk" during the "admission" or "registration" process.
Reading
How well they do in filling out the "Pre Admission" Questionaaire (below)?
Writing
How well this do in filling out the "Pre Admission" Questionaaire (below)?
• What kinds of decisions will be made or actions taken as a result of the test?
To see what changes can be made to improve this type of assessment in the future. See if we can develop some form that ESL students can bring in their next "hopital" or "doctor" visit to better communicate their reason of visit.
• What constraints or obstacles might be experienced due to time, stakeholders, location, student population, etc.?
Some of the constraints or obstacles that might be experienced are will vary according to the factor(s):
-time: The time period of visit by the ESOL student will vary depending on the medical reason for visiting the clinc or hopital-if they will even have to visit overall.
-location:
-student population:
-english level: Each of our ESOL students have various educational backgrounds (e.g. high school graduate, college graduate, etc...), so not all students will be able to perform each task.
-confidentiality: each student's comfort "level" will vary during each step.
Assessment Instrument • Content: Describe the construct/content that you are assessing and any component parts if applicable. For example, aural proficiency, grammar, completing a job application, etc.
I'll be evaluating how the student's do in filling out a "Pre-Admission" Questionnaire that they would usually fill out when they visit a clinic-hospital.
• Operations: A list of the tasks that the student will have to carry out, e.g., scanning and locating information
In order to carry out this exercise, the student would need the following information:
-Address, phone #, etc..
*some of my students don't know this by "heart"
-health insurance information
-medical history
Below is a list of all the tasks that the student will have to carry out:
1. Know how to fill out a "Pre-Admission" Evaluation...
Pre-Admission Steps, from calhospitalcompare.org
"What Information Does the Hospital Need from You? By providing the following information to the hospital before you are admitted, your days before and immediately after entering the hospital will be less hectic, the admitting process will be smoother, and your stay will be safer. The information to provide to the hospital admission office includes:
* Your personal contact information;
* Contact information for the person who will act as your spokesperson;
* Names and contact information of your doctors;
* Details of your health insurance;
* List of your medications, including over-the-counter and vitamins;
* Facts about your health, such as allergies and prior or existing conditions;
* Special needs you may have, such as diet or language assistance; and
* Copies of your Advance Directive and any other health care-related documents.
*see Stevens County Medical Center-Services
• Student profile: A description of your student population.
-Students Profile: Many of our students (Mexico, Brazil, and Eastern Europen bloc nations) in the Morris Literacy Project are "migrant famers" that work in the surrounding area farms doing various work (e.g. dairy, hog, and typical field work). All of our students have been adults (late teens to the elderly); however, we currently have a student is only a teen. They like to come in groups (Spanish, Portuguese, Russian, etc..) , so the language barrier hasn't been a challenge. However, we they don't come in groups, they somehow work cohesively together.
• Types of items: A description with examples of the kinds of tasks students will have to complete (multiple choice, mock job interview)
• Format and Timing: When and how will you administer the test.
Depending on the steps each student takes. The first step should be easily assessed in the classroom. However, the actual visit to the hospital or clinc will vary from student to student-depending on "if" they need to visit the hopistal or clinic for any reason.
• Criteria levels of performance: This refers to the scoring approach, e.g. a rubric.
Registration "Writing" Self-Assessment Questionnaire
*see salsRegistrationWritingAssessmentQuestionnaire.doc
Registration "Reading" Rubric-Assessment Questionnaire
*see salsRegistrationReadingRubricAssessmentQuestionnaire.doc
Registration "Oral" Observation-Assessment (pp 7 of 12 in Oral Rubric)
*see salsRegistrationOralAssessmentQuestionnaire.doc
LEVEL 1
Level 1-Comprehension
Cannot understand even simple conversation
Level 1-Fluency
Speech is so halting and fragmancy into make conversation virtually impossible
Level 1-Vocabulary
Vocabulary limitations are so extreme as to make conversation virtually impossible
Level 1-Pronunciation
Pronunciation problems so severe as to make speech virtually impossible
Level 1-Grammar
Errors is grammar and word order so severe as to make speech virtually intelligible
LEVEL 2
Level 2-Comprehension
Has great difficulty following what is said. Can comprehend only "social conversations" spoken slowly and with fragment repitition.
Level 2-Fluency
Usually... ,.. forced into silence by language limitations.
Level 2-Vocabulary
Misuse of words and very limited vocabulary...comprhension quite difficult
Level 2-Pronunciation
Very hard to understand because of pronunciation problems. Must frequently repeat in order to be understood.
Level 2-Grammar
Grammar and word order errors make comprehension difficult. Most often rephrase or restrict what is said to basic patterns.
LEVEL 3
Level 3-Comprehension
Comprehends most of what is said at slower than normal speed with repetition
Level 3-Fluency
Speech is everyday communication and classroom discussion is frequnetly disrupted by the student's search for the correct manner of expression.
Level 3-Vocabulary
Frequently uses the wrong words conversation somewhat limited because of inadequate vocabulary
Level 3-Pronunciation
Pronunciation problems associate concentration on the part of the listener and occasionally lead to misunderstanding.
Level 3-Grammar
States frequent errors of grammar and word order which occasionally obscure meaning.
LEVEL 4
Level 4-Comprehension
Understands nearly everything in normal speed although occasional repetition may be necessary
Level 4-Fluency
Speech is everyday communication and classroom discussion with occasional lapses while the student searches for the correct manner of expression.
Level 4-Vocabulary
Occasionally uses appropriate terms or ..phrase ideas because of inadequate vocabulary
Level 4-Pronunciation
Always intelligble though one is conscious of a definite accent and occasional inappropriate unforseen patterns.
Level 4-Grammar
Occasionally makes grammatical or word order errors which did not obscure meaning.
LEVEL 5
Level 5-Comprehension
Understands everyday conversation and normal classroom discussion without difficulty
Level 5-Fluency
Speech is everyday conversation and in classroom discussion afluent and effortless, appearing that of a native speaker.
Level 5-Vocabulary
Use of vocabulary and idioms approximatics that of a native speaker
Level 5-Pronunciation
Pronunciation and approximate a native speaker's
Level 5-Grammar
Grammatical usage and word order approximate a naitve speaker's
Continuum of Speaking Behaviors
BEGINNING
Beginning-Orally
Shares some of own experiences in conversation
Beginning-Vocabulary
Uses a basic, core vocabulary that includes many non-specific words
Beginning-Asking Questions
Obtaining information by questioning in a struggle, still exploring structure of questions.
Beginning-Answering Questions
Learning to use question words and structuring-some answers seem irrelevant.
DEVELOPING
Developing-Orally
Frequently shares own experiences
Developing-Vocabulary
Vocabulary is becoming more specific; beginning to make new words his/her own by generalizing to new situations
Developing-Asking Questions
Asks wh- questions; is developing precision in technique
Developing-Answering Questions
Anwer wh-questions
INDEPENDENT
Independent-Orally
Orally shares information and own experiences
Independent-Vocabulary
Uses a variety of words across many settings and contexts. Hears words during conversation and readly make them his/her own.
Independent-Asking Questions
Obtains necessary information by asking questions during dialogue
Independent-Answering Questions
Responds to variety of questions of increasing complexity
• Discussion of test quality (reliability, validity, practicality, washback)
The format of this type of assessment will vary according to the different factors. One factor would be the degree of the level of our ESOL learner based on CASAS results and "language proficiecy" overall. Another factor will be the type of "registration form" the ESOL student will be using according to his/her "real-life" skill circumstances or current situation (e.g. hospital, police, social services, etc..). For example, I had a student (from Honduras) years ago that asked me to help fill-out a "phone-cable-internet" application to get this particular service in his apartment. I had to "physically" go with him to the service provider's office to interpret his desire of need. I feel this wouldn't be a "reliable" assessment, but a "valid" ("a test is appropriate and meaningful" )one. In terms of "practicality", I feel this assessment will be going through some changes as I find out on how to "rearrange" the format of all three types of assessments. Also, I'll probably get some feedback from individual community members (e.g. local hospital, professors or instructors at the local university, ESOL students, etc...) through community gatherings or individual meetings.
I feel the "washback" on this assessment would bring key particular "real-life" topics to teach our ESOL students as it comes up week by week. I hope to take some type of statistic analysis or record on what are the "most frequent" topics to teach on. Lastly and most importantly, our ESOL students will "get the most of this" as they know what "we" will teach them. Thus, they will know that we are paying attention to their "need analysis", which is the main reason why "we" are there for them!
Related Sites:
Studies
Assessment Case Stud of an ELL/ESL/EFL Student
"Class Presentation"
Tests Online
Tools
ESL ReadingSmart Demo
"Visit http://www.eslreadingsmart.com for a free trial!
ESLreadingsmart.com is a Web-based reading and curriculum program for ESL students and teachers. It has been statistically proven to raise students' reading and language scores.
Here is a summary of the features of this effective and innovative program:
• Individualized learning path based on students' placement test results
• Achievement reading test after completion of instructional requirements
• Reading program from beginner to advanced ESL/ELD students, grades 4-12
• Newcomers' program for students who do not speak English
• Assessment system for teachers to track class and student progress
• Reporting system for students to track their own progress"
Date
Learn Hindi - Vocabulary: Numbers
" Learn numbers in Hindi. To learn a new language from other native speakers, visit Rockle Lingomigo at: http://www.rockle.com/language/amigo"
Test