Liberal Arts: English-Assessment

Summer 2008

ESL 6638 Assessment of the Adult ESL Student Online
(2 sem. cr.) Dates: June 28-August 9
"Learn the basic principles of testing and evaluation, and how assessment is related to curriculum. Learn how to conduct valid formal and informal assessment of ESL learners. Develop authentic assessment tools for academic and workplace settings. Discuss entrance and exit criteria for ESL services and how to assess student progress. Explore the politics of testing and assessment. Online course and login information at www.hamline.edu/gseonline. Optional orientation session at Hamline on June 28, 10:00 a.m.-12:00p.m. Drew Science 305. Target Audience: ESL teachers of adult learners.

"2 semester credits. Enrollment limit: 29. Online dates: June 28-August 9. Cost $460. Required text and course packet. See Policies & Procedures.
Instructor: Todd Wagner is the assessment and evaluation specialist for Adult Basic Education at the Minnesota Department of Education

Required Books:
The Essentials of English: A Writer's Handbook (with APA Style) (Spiral-bound) by Ann Hogue (Author), from amazon.com List Price: $41.33
Understanding and Using English Grammar (Paperback) by Betty Schrampfer Azar (Author) "Following are some dialogues between Speaker A and Speaker B..." (more) , from amazon.com List Price: $48.67

Terms

Pyschometrics-"...field of study concerned with the theory and technique of educational and psychological measurement, which includes the measurement of knowledge, abilities, attitudes, and personality traits. The field is primarily concerned with the study of measurement instruments such as questionnaires and tests. It involves two major research tasks, namely: (i) the construction of instruments and procedures for measurement; and (ii) the development and refinement of theoretical approaches to measurement..."
eugenics-"...is a social philosophy which advocates the improvement of human hereditary traits through various forms of intervention.[2] Throughout history, eugenics has been regarded by its various advocates as a social responsibility, an altruistic stance of a society, meant to create healthier and more intelligent people, to save resources, and lessen human suffering."
Craniometry-"..is the technique of measuring the bones of the skull. It is distinct from phrenology, the study of personality and character, and physiognomy, the study of facial features. However, these fields have all claimed the ability to predict traits or intelligence."

*see Rubrics
"scoring scale, is typically created which contains the essential criteria for the task and appropriate levels of performance for each criterion...
holistic rubric-"assigns a level of performance by assessing performance across multiple criteria as a whole"
analytic rubric articulates levels of performance for each criterion so the teacher can assess student performance on each criterion.

valid-"degree to which a certain inference from a test is appropriate and meaningful" (AERA, APA, & NCME, 1985).."
reliable-consistent
washback-effect a test has on teaching and learning
What Is "Washback" and Why Do Some Language Researchers Avoid the Term? , from jalt.org
modality-channel (aural/oral) of language use

Related Sites:

indirect vs. direct tests
*see A Comparison of Direct & Indirect Assessments of Writing Skill, by Hunter M. Breland
discrete-point vs. integrative testing
*see Methods of Assessment, btinternet.com
"Discrete point tests: cannot be pragmatic. There is no ordinary discourse situation and no normal language use context where a learner might be asked to listen and distinguish between "ship" & "sheep" or perform active to passive transformations."
What is a discrete point approach to testing?
"Electronic quiz tools usually involve a discrete point approach to testing as opposed to an integrated or authentic approach, such as papers and projects. Discrete point tests are made up of test questions each of which is meant to measure one content point. Discrete point testing is associated with multiple choice and true/false formats, which have been criticized for testing only recognition knowledge and facilitating guessing and cheating. However, if they are used for an appropriate PURPOSE and if the test questions are well constructed, discrete point tests can be used for effective teaching and learning..."
objective-vs. subjective scoring
*see Objective and subjective evaluation
"Objective tests are often constructed with selected-response item formats, such as multiple-choice, matching, and true-false. An advantage to including selected-response items in objectively scored tests is that the range of possible answers is limited to the options provided by the test writer—the test taker cannot supply alternative, acceptable responses.

Related Sites:

*concurrent validity (wikipedia)-"is demonstrated where a test correlates well with a measure that has previously been validated. The two measures may be for the same construct, or for different, but presumably related, constructs.
For example, if a test measuring job satisfaction gives similar results to those gathered using a job satisfaction test which has been validated in past investigations the new measurement has concurrent validity. Alternately, a measure of job satisfaction might be correlated with work performance. Note that with concurrent validity, the two measures are taken at the same time. This is in contrast to predictive validity, where one measure occurs earlier and is meant to predict some later measure."
*inter-rater reliability (wikipedia)-"iss the degree of agreement among raters. It gives a score of how much homogeneity, or consensus, there is in the ratings given by judges. It is useful in refining the tools given to human judges, for example by determining if a particular scale is appropriate for measuring a particular variable. If various raters do not agree, either the scale is defective or the raters need to be re-trained."

CURRICULUM-BASED TESTS-"..measurement that uses "direct observation and recording of a student's performance in the local curriculum as a basis for gathering information to make instructional decisions" (Deno, 1987, p. 41)...provides a structured way to see how well a child performs on the materials the teachers is assigning the class.(from umn.edu/~devenz)
".. refer to instruments that assess one element/component of a skill at a time" (Hamline Univ)
COMPETENCY-BASED TESTS-""
What is CBT & Characteristics of CBT
"..A skill performed to a specific standard under specific conditions...Assessment of competency takes the participant’s knowledge and attitudes into account but requires actual performance of the competency as the primary source of evidence. ..
DIAGNOSTIC TESTS -""
Test your English language ability Online diagnostic tests

*internal consistency(wikipedia)-"is a measure based on the correlations between different items on the same test (or the same subscale on a larger test). It measures whether several items that propose to measure the same general construct produce similar scores. For example, if a respondent expressed agreement with the statements "I like to ride bicycles" and "I've enjoyed riding bicycles in the past", and disagreement with the statement "I hate bicycles", this would be indicative of good internal consistency of the test."

Tests

People

-4 key figures

Theories

Current Issues

History Lesson

DISCUSSION BOARD:
-Topic: Doubters and Believers
Date: Tue Jul 15 2008 23:27
Author: M, Sal
Subject: certificate to show! Re: #2 oppressive and humiliating
"yes, students would like to see scores or some way to know they are doing well. I had one student (from Bulgaria) that asked me and my "supervisor" if we could give him a certificate. He wanted to go back to his home country to show his people (future place of employment, family, etc..) that he took an ESL course while working (was a intern as a vet in a local nearby pig farm) in America! It was the first time my "supervisor" and I ever been asked this, which we will try to do more often now!"
Date: Wed Jul 16 2008 15:10
Author: W, Todd
Subject: Re: certificate to show! Re: #2 oppressive and humiliating
" Several programs around the country, including Minneapolis, issue certificates to students based on reaching certain score/proficiency levels on the CASAS tests. My understanding is that this process has been well-received in Minneapolis by both students and teachers. Implementing this process also promoted a great deal of consideration of the CASAS compentencies and whether they constitute a reasonable/quality/useful/valuable content domain of instruction for an ESL program. My understanding is that that process was also well received and that many teachers who were skeptical of relying heavily on the CASAS competencies have found the compentencies, on closer examination, to be reasonable/useful.-Todd
Date: Fri Jul 11 2008 21:06
Author: C, Matthew
Subject: Re: #2 oppressive and humiliating
" When I worked at a intensive English language school, I often had students who craved more drill than we offered. Some would purchase Raymond Murphy's Grammar in Use texts on their own. The underground movement got so popular that we built an elective around the text. At the time, my Academic Director argued that at least that way we could insure that they had some communication based activities along with the grammar.
-Topic: History of Educational Testing
Forum: History of Educational Testing
Date: Sun Jul 06 2008 13:57
Author: H, Gregory
Subject: History of Testing
"Hello Classmates,
I just read the Power Point about the History of testing and it scares me. It seems that our whole approach to achievement testing in the U.S.A. has imitated experiments and practices of people who we deemed were "credible experts". What´s up with the craniometry test to determine intelligence? It seems a little troubling to me that the "IQ" test came from a method Alfred Binet developed to find out the causes why French school children where not meeting curriculum standards. Apparently no one knows how that test can be equated to a true representation of intelligence.
After reading over the 8th grade proficiency exam from a Kansas school, I wonder if this should be more like aCommunityy College proficiency exam nowdays. The questions are almost all based on theory or rote knowledge.
This article gets me a bit riled up, which I believe is the point.
Are achievement tests some means of control? Are standardized tests some means of social domination?
I think we should try to find out a way to test achievement based upon real life situations and performance of tasks. It seems like almost all tests are based on theory or are out of context for the people taking the tests.
Date: Thu Jul 10 2008 14:36
Author: R, Joyce
Subject: Re: Zappardino and the History of Educational Testing
" I'd like to be candid and say that I never thought studying the history of assessment would be so intersting. (Sorry, Todd) The good news is that this is intriguing.
Historically, I was unaware of the newnesss of the study of the mind psychology" and so aligining Binet's efforts and the entree into scientic methods of measuring elements of the mind helped me to understand how the scientific and lay communities could be awed and motivated to accept answers about the unknown elements of the mind. Aren't we all tempted to take alittle test to find out more about ourselves?
The idea that testing is a socially imbedded activity is important for us to remember as instructors who interface cross-culturally. The cloak of science that covered those early researchers political agendas may well seem obvious to some cultures represented by our students i.e. they may discern that we have motives beyond their benefit in our obsession with formal testing.
Zappardino seems to refer to the "faith" we have/ had in the numbers yielded from testing. I couldn't help but think how often we say "numbers don't lie". We sort of exalt numbers/statistics and the interpretations. Maybe I'm carrying this too far, but there is a scripture that says, "God is not a man that he should lie." So for me, this "numbers don't lie" thing puts numeric results on a pretty high level.
I'm in the dark about how one changes the culture of standardized testing, but I don't want to be a gatekeeper that holds folks at bay from educational or other opportunity, helping them "settle" and come to terms with a lesser view of themselves and their potential. I want to remember the idea of testing being culturally bound, even my own created tests.
Culturally speaking, I have one small anecdote: When we moved to MN from CA our youngest daughter was given a kindergarten readiness test. She was asked to name five jor six things you can do with water. She replied, "Drink it, swim in it, wash dishes." The tester smiled condescendingly, "How about skate on it? When it snows, make snowmen?" later she told me that I might need to read more to my child to help her think in broader contexts. WHAT? Hey, we lived on the beach! We drank it, swam in it and washed up! - Joyce -Topic: Other Module 2 Issues
Forum: Other Module 2 Issues
Date: Thu Jul 10 2008 00:11
Author: M, Sal
Subject: Good tip for assessment from O'Malley
" I'm still finishing the reading for Module 2, but I wanted to comment on one particular advice O'Malley gave. It was on "Designing Authentic Assessments" (Ch. 2 pp. 17-18). I live in a small college town (5,000 pop) here in Morris, MN and thought the tip on "conducting professional development..." would be very useful here. I plan to contact some professors at the local campus and ask for their advice on planning authentic assessment. Very informative so far in this reading assignment!
Date: Thu Jul 10 2008 22:40
Author: M, Sal
Subject: The importance of meeting the students' needs
' Yes, reading Chapter 1 "Assessment as Information Gathering" by Kathleen M. Bailey was very helpful in the part of meeting the students' needs (pp.5-6). We are sometimes given guidelines on what to teach. However, we also need to consider the students' needs, which we'll know once we know the information about them as quoted by Diane's (teacher) dialogue with Kathi (author giving advice). I see this as very true when I teach ESL to local migrant "dairy farmers" compared to other students with a different profession/job. What I've learned from teaching ESL as a volunteer in the last 4 years, the students seem more interested on learning terminology relating to their workplace. They "want" to learn English to improve their work situation. As supposed to a learner that does not work in the same occupation/field or even as a homemaker. "
Date: Fri Jul 11 2008 10:15
Author: M, Sal
Subject: "Massive Systemic ignorance and blind reliance on standardized est scores..."...
" Wow, the quote above I partly took from Bailey's response on Ellie's (teacher w/three titles in a California school) response to the "school system" she was part of just summarizes one of many stories out there! Reading this (Chapter #3-Conflicting Purposes of Assessment) really got my attention and brought some personal experiences (as a student in grade school and teacher in ESL as an adult). In "grade school", I was close to being sent back because of my "learning disabilities". My parents immigrated here from the Philippines and never taught me their native language (Tagalog). They were afraid of me and my sibling being "left out" in our English-speaking dominant "new home country" (USA). However, my parents would speak Tagalog to each other in the home and their "English" (spoken differently in the Philippies, like Britain somewhat) was a little different. Thus, going to school, I would have trouble-another factor was their "passive" cultural life-style. English grammar was never every my strong quality (math was) growing-up! Anyways, going back to being a "current ESL teacher" (volunteer), I can relate to some of my students (both adult and non-adult). When I tell people in the small rural-college town community (Morris, Minnesota) that some of my students have "college degrees", they get suprised! One of my students (grown-up adult) was a "migrant worker" from Mexico, who got a college degree as a veterinarian . He came to our class because he just wanted to learn English! Just reading the story of Ellie makes me wonder how many school systems has "mis judged" or labeled students with a "learning disability" due to their language being "Non-English"! Wow, great story!
Forum: Other Module 2 Issues
Date: Fri Jul 11 2008 17:43
Author: M, Sal
Subject: Contrasting Pairs of Concepts-examples?
" Hi all,
I really haven't had that much experience with creating my own testing and scoring of ELL or ESL Students. I was wondering if anybody has and willing to share their experiences relating to the "three contrasting pairs of concepts" Bailey shares in Chapter 6?
Forum: Other Module 2 Issues
Date: Fri Jul 11 2008 18:48
Author: M, Jeannine
Subject: Re: Contrasting Pairs of Concepts-examples?
" One of our English department faculty, Susan Bosher, has done a lot of work with multiple choice testing with ELL health careers students. She uses language simplification techniques. I have one of her journal articles on my other computer. I'll find it and send it as an attachment. In the past I've taken my exams to Susan and had her review the questions. It is extremely helpful. I've also found that the modifications I make based on her suggestions make the test much more understandable for all students- not just English learners. Writing exams, as you probably know, is really hard. JMH "
Author: F, Patricia
Subject: Re: Statement #4 How well do I know my students?
"I also agree with the statement that we can't be everywhere at all times. Even a poorly designed test can give us some information. Say we hand the student a test during registration on the first day that asks them to write their grocery list from last week rather than their name, address, etc. That is neither reliable nor valid. If the student is unable to write one item down on the paper, however, we can take and educated guess that they may not be able to read the instructions or may not be able to write the answers.
Greg, here is a prime example of your statement on being mislead by a student's proficiency in one area. I had a 50 something man from Mexico that came to class one evening. He talked with all the instructors, trading stories and information for more than 15 minutes at a level I would have said was close to a native speaker of English. When we handed him the registration forms he informed us he could not read or write in English or Spanish. That was during my first semester of teaching and it was a lesson I will never forget.
Author: L, Anne
Subject: 1-4 thoughts

" ...2- I tend to go back and forth on this one. If the assessments are relevant and address the needs of the students, then I think assessments are not oppressive and humiliating. However, “throwing” someone into an assessment who has never tested before is oppressive and humiliating.

I. The Four Skills (Listening, Speaking, Reading, & Writing) and Hierarchical Components of Language
A. Syntax and Lexicon
1. syntax (wikipedia)-"study of the principles and rules for constructing sentences in natural languages"
2. Lexicon (wikipedia)-"its vocabulary, including its words and expressions. More formally, it is a language's inventory of lexemes."
B. Morphology (wikipedia)-"field of linguistics that studies the internal structure of words. (Words as units in the lexicon are the subject matter of lexicology.) While words are generally accepted as being (with clitics) the smallest units of syntax, it is clear that in most (if not all) languages, words can be related to other words by rules. For example, English speakers recognize that the words dog, dogs, and dog-catcher are closely related. English speakers recognize these relations from their tacit knowledge of the rules of word-formation in English. They intuit that dog is to dogs as cat is to cats; similarly, dog is to dog-catcher as dish is to dishwasher. The rules understood by the speaker reflect specific patterns (or regularities) in the way words are formed from smaller units and how those smaller units interact in speech. In this way, morphology is the branch of linguistics that studies patterns of word-formation within and across languages, and attempts to formulate rules that model the knowledge of the speakers of those languages..."
C. Phonetics and Phonology
1. Phonetics (wikipedia)-"the study of the physical sounds of human speech. It is concerned with the physical properties of speech sounds (phones), and the processes of their physiological production, auditory reception, and neurophysiological perception."
2. Phonology- (wikipedia)-"is a subfield of linguistics which studies the sound system of a specific language or set of languages. Whereas phonetics is about the physical production and perception of the sounds of speech, phonology describes the way sounds function within a given language or across languages."

Related Sites:

Other Terminology

3.1

"..Other research studies found that teachers’ perceptions of content standards and the professional development provided for them play a strong role in the successful implementation of standards. Specific professional development about standards-based curriculum and assessment contributed to better student math achievement in California, according to a self-report of 1,000 K-12 teachers (Cohen & Hill, 2000). Case studies and descriptions of professional development to support standards-based education in K-12 document a wide range of activities used by school districts to train teachers, including workshops and conferences, institutes, observations of expert teachers, mentoring, study groups, grade-level meetings with other teachers, site-based inquiry seminars, training and support to use new curricula and standards, and certification training (Bye, 2004; Dutro, Fisk,..-pp 5 of 6

Other Related Sites:

3.4

Three Challenges in English Language Assessment, html format & pdf format by Carol A. Chapelle & Joan Jamieson
"..Terms such as “placement,” “proficiency,” and “achievement” might be used in response to a question about what a test is supposed to be used for. These terms may work for the public, but teachers need a more systematic, thorough, and accurate way of considering the purpose of an English Language assessment because the purpose of the assessment is critical for choosing or developing a good test. We think of test purpose as consisting of three interrelated concepts: • One way to look at test purpose is to consider the inference that test users want to make on the basis of test scores. For example, we designed Longman English Assessment to allow test users to make an inference about examinees’ English Language reading comprehension, listening comprehension, and knowledge of written structures. Making an inference about these three aspects of English ability was one way of looking at the test purpose. • A second way is to consider the use of the test results. Typically, a test is given because a decision has to be made about examinees, for example, their readiness to study at a university, their eligibility for a job, or the grade that hey should receive in a course. Longman English Assessment is not intended for any of these relatively high-stakes test uses; instead, it is intended to give learners an idea of their level of English ability and to offer level- appropriate advice for improving their English. • The third aspect of test purpose is any other effect that the test is intended to have, such as serving as a motivating, interesting, and informative experience for learners."

3.5

Final Project
Design of an Assessment Instrument: "Real-Life Skills"
July 17th of 2008

CONTEXT
This assessment is designed for classroom settings to help student learn "daily-living skills (e.g. intro conversations, grocery shopping, and other day-to-day social activities in the community). The "Real-Life English workbooks" (see Steck-Vaughn Company or amazon.com) has different levels, which uses various community "real-life" settings. Students want to be better "active-participants" in their workplace, home, and community while they are residents of this small rural college community (10,000+ population in the Stevens County area we serve). Thus, teacher too want to be "better" in working with each individual students to achieve their goals to succeed-not only in America, but also when they go back to their "home country". This assessment will encourage all three stakeholders (students, teachers, and program) through these results in various ways of performance.

SETTING
-Students Profile:
Many of our students (Mexico, Brazil, and Eastern Europen bloc nations) in the Morris Literacy Project are "migrant famers" that work in the surrounding area farms doing various work (e.g. dairy, hog, and typical field work). All of our students have been adults (late teens to the elderly); however, we currently have a student is only a teen. They like to come in groups (Spanish, Portuguese, Russian, etc..) , so the language barrier hasn't been a challenge. However, we they don't come in groups, they somehow work cohesively together.
-Audience:
The results would be very helpful for teachers (current and future volunteers) on how to improve our approach in teaching this particular topic ("real-life" skills). This may also help our continued new volunteers (mostly new in-coming college students) more interested in our on-going program (Morris Literacy Project). Knowing "what works" and "what doesn't work" (our approach in teaching) may help retain future volunteer teachers and more importantly-students that come to check out how we do class.

PURPOSE:
The purpose of this assessment is to see how students are improving in the "four-different area skills" (Listening, Speaking, Reading, & Writing). Thus, to help them feel more confident in applying them in their "real-life" settings (e.g. workplace, home, community, etc..). We hope this would be given as both a summative and fromative placement test to inform both student and teacher.
QUESTIONS:
Several questions will come up from this assessment-1: Are the student's skills helping them in their various workplace, home, and/or commmunity settings? Are we consistently retaining our number of volunteer teachers and students in a certain period of time? Do they demonstrate a degree of progress to move to the next level of books of "Real-Life"?

4.1

Terms

Itemized Response Theory:
Wikipedia
Chapter #1-Item Calibration & Ability Assessment, from creative-wisdom (multi-media)
-Guttman Scale, Wikipedia
"..if they can be ranked in some order so that, for a rational respondent, the response pattern can be captured by a single index on that ordered scale. In other words, on a Guttman scale, items are arranged in an order so that an individual who agrees with a particular item also agrees with items of lower rank-order. For example, a series of items could be (1) "I am willing to be near ice cream"; (2) "I am willing to smell ice cream"; (3) "I am willing to eat ice cream"; and (4) "I love to eat ice cream". Agreement with any one item implies agreement with the lower-order items..."
=>Multiple Choice Tests: Test scoring and analysis Christina Ballantyne (tlc.murdoch.edu.au)
tentative student proficiency-portion of correct answers for each student
tentative item difficulty-the pass rate for each item
convergeance-"there will be some differences int he model.."
Chapter #2-Item Characteristic Curve-"the relationships between the probabilities and the skill levels"
Related Sites:
Assessment Issues:
"..The ICC is a plot of Probability that the Item will be answered correctly against Ability. The shape of the ICC reflects the influence of the three factors:
* Increasing the difficulty of an item causes the curve to shift right - as candidates need to be more able to have the same chance of passing.
* Increasing the discrimination of an item causes the gradient of the curve to increase. Candidates below a given ability are less likely to answer correctly, whilst candidates above a given ability are more likely to answer correctly.
* Increasing the chance raises the baseline of the curve.
This simple simulation allows the user to investigate the factors governing the shape of the Item Characteristic Curve. All three well known IRT models are represented (referred to as IRT1, IRT2 and IRT3) and Item Characteristic Curves can be super-imposed on one another to see how they relate.''
IRT Model (ony 1 parameter=item selection: A&B-2 Parameter; A, B, & G-3 Parameter=diagnosis parameter=2/3 parameters)
"B parameter"/threshold parameter-item difficulty
"A parameter"-item discrimination
"G parameter"-item guessing
Lord's Paradox-a phenomenon of the probability of giving to the correct answer to the blue is .55 and red item is more difficult
intercept
Chapter #3-Item Information Function
item information function-"mathematical way to know how much info ICC provides, the more precision you have in estimating a parameter->the more you know of the value of that parameter
Item information function of the 2PL model, from metheval.uni-jena.de
Chapter #4-Test Information Function
test information function-identifying information for entire exam
alternate forms-having different items with different levels
form balancing-swapping items (e.g. easy, difficult, and hard) between the item pool and the forms is needed
Chapter #5-Item Person Map
odd ratio-ratio of non-desired events (Q)/divided by desired events (P) or Q/P
logit-log (odd ratio)
easy if below zero (e.g. -.7)
item person map-item (test question itself-easy, average, difficult) and person (item proficiency value) can be viewed simultaneously
Q: odd of 1/6 passing?
A: .20
Chapter #6-Misfit
misfit-about the response pattern (e.g. European history question accidentally added in a "American History Exam"); when there is a huge discrepancy (model and data don't fit with each other)
residuals-discrepancies
Chi-square fit-a statistical approach for the most common test of goodnesss (correct & incorrect; categorized by the skill level
O-E 2 (squared)/ E [O-observed, E-expected]
#2 Approaches to detect misfit:
1-graphical method-model and data are superimposed
2-Chi-square test of goodness of fit-chi square divided by the degrees of freedom
Q: In regression we obtain a systematic pattern of residuals
A: F
Residual Plots and Regression Assumptions, from stat.tamu.edu
Degree of Freedom
""Degree of freedom" (df) is an "intimate stranger" to statistics students. Every quantitative-based research paper requires reporting of degrees of freedom associated with the test results such as "F(df1, df2)," yet very few people understand why it is essential to do so. Although the concept "degree of freedom" is taught in introductory statistics classes, many students learn the literal definition of this term rather than its deeper meaning. Failure to understand "degrees of freedom" has a side effect: Students and inexperienced researchers mis-interpret a "perfect-fitted" model or an "over-fitted" model as a good model. To rectify this situation, two approaches of illustrating df in terms of sample size and dimensionality are recommended. To enhance the pedagogy of this concept, a multimedia program written in Macromedia Flash (Macromedia, 2001) was developed. In the program, different aspects of the above concepts are visualized and interactive features (questions and answers) are added to encourage the learners to think about the deeper meanings of df."
df=n (# of obeservations)-r (# of necessary relationships)

Computer Adaptive Testing:
Related Sites:
What is Item Response Theory, IRT? A Tentative Taxonomy, from rasch.org

4.2

4.3

#3 major features:
1. language-bilingual?
2. supplemental assessment
3.oral reading assessment
3rd Reason:
67% of jobs created over the next decade required college degree
#4 literacy levels
"..Next, although 13% of American adults spoke only Spanish or another language before starting school, these individuals accounted for 37% of the below basic population. .."
-Dr. John Stucker
".. I would be working with the other teachers and asking questions about how is our outreach? Are we really reaching these folks in our community? How is our teaching approaching going? How we are doing with the young people who are sometimes getting pushed out of high school?..
as Sheida pointed out, you see the highest percentage of people who report learning disabilities as well as other disabilities. And most learning disabilities involve reading even if they aren't labeled as reading disabilities. So these are people who have identified reading problems. Other studies we have done show that there is a higher percentage of their native US born people who have been in special education, have gotten extra help, whose teachers realize they needed extra help, and whose parents realized they needed it.."
And then the other thing about these folks is, again, it's remarkable how fast these ELLs have picked up English conversation. But don't let that fool you. If you actually get a chance to assess their reading skills in English, it turns out they are extremely idiomatic when they talk to you. If one person walked in and joined us with this panel, we think gosh! That person speaks great English but when they went to do embedded literacy tests, such as the NAAL, those tests were hard for them. Let's talk a little bit about the basic people. I think this is an extremely important population we don't want to neglect, because they are close to intermediate. And we know from other studies that intermediate is the point where you see big changes to the positive in income, civic participation, personal reading habits, all the things that embed literacy in our life and make it so meaningful and important...
...If we can identify their specific needs, their needs for fluency and that sort of literate, academic vocabulary, it may be that we can get them into the intermediate level faster and that opens up a lot of opportunities for them in terms of involvement in post-secondary and all the other things that we talked about..
-Brian Bosworth, Future-Works
".. Fourth point I want to make, most employers say they are as concerned or even more concerned about other workplace basic skills as they are about traditional measures of literacy and what do they mean by those other workplace skills? Technology skills or computer literacy is increasingly important to all employers. They identify career management and lifelong learning skills, and by this I mean the navigational skills associated with career management and lifelong learning. The employers identify, thinking critically, acting logically and solving problems as a critical basic skill. They identify finding and using information, knowing when to ask a question, knowing when you don't have the right information to answer that question and knowing where to go to get that information. They identify teamwork as a basic skill. The ability to work in groups with others cooperatively. They identify basic employability skills as well, sometimes called social skills, attendance, timeliness, work ethic, showing up sometimes is the most critical basic skill. Employers do identify math, reading, writing as important deficits in their workplace. But we find, through our survey work that employers are more likely to suggest some of these other basic workplace skills are as important or more important to them than the conventional measures of literacy.."
They would be contextualized in the language and the situations of the workplace. They would use work as a way to develop the workplace basic skills. They would be focused on a credential, a credential, which satisfies entrants to the next level of education and training....
Q&A:
-Brian
'..NCSALL colleague, Steve Reeder out at Portland state working with other researchers developed a way that you can sort of do a synthetic NAALs on your county or community, based on local demographic data. And some of the other characteristics that occur both on the NAAL assessment, and the background questionnaire and that you can also gather about your local population. .."
-Sheila
"...Sheida White: I think vocabulary is an area where ESL adults have particular difficulty in...because unlike native speakers of English who have the listening vocabulary, and so their problem is essentially decoding the word and then they can map their overall vocabulary to the text that is before them. Oftentimes, ESL adults do not have the overall vocabulary, so they have to learn that in addition to learning the...how to read the words and so it's a little bit more difficult for them to do that. So I would say vocabulary is a particular area where ESL adults can benefit from. .."
National Assessment of Adult Literacy
"By comparing results from 1992 and 2003, NAAL provides the first indicator in a decade of the nation's progress in adult literacy. NAAL also provides information on adults' literacy performance and related background characteristics to researchers, practitioners, policymakers, and the general public."
-Brian
Q: "We marketed these classes through various venues, but attendance has been too low to continue the classes. Any suggestions on what they can do?
A: Brian Bosworth: "Well, I think my quick suggestion and I hate to sound like a broken record is that offering the instruction in the workplace in partnership with the employer where most of these folks are for 40 hours a week, isn't a bad strategy in terms of securing participation and securing attendance."
National Institute of Literacy, discussion list
-Literacy in War and Peace Tom Sticht's Work in Adult Literacy Education By Thomas Sticht (from nald.ca)
"...In the winter of 1967 I got a call from a friend working in Monterey, California at a field unit of the Human Resources Research Office (HumRRO) of the George Washington University. He told me the military was joining President Johnson's War on Poverty by inducting 100, 000 new recruits a year who were low in reading ability. HumRRO thought it might be necessary to do more teaching by "show and tell" so that the New Standards Personnel (NSP, as they were called) could learn by listening rather than by reading. With my background working with blind students, HumRRO thought I would be a good person to do research on listening and reading abilities of the NSP..."
..In 1992, I was invited to London by what is now the Basic Skills Agency (BSA), to speak at its national conference. Princess Anne, the Patron of the BSA attended and sat through my presentation which emphasized the importance of getting "double duty dollars" through the intergenerational transfer of literacy from parents to their children. Newspapers reported that the Princess Royal found these ideas quite sensible. I worked with the BSA for several years on their national family literacy program.
In 1994 I was honored by being elected to the Reading Hall of Fame and in 1997 Timothy Shanahan and Susan Neuman, writing in the Reading Research Quarterly , selected the work colleagues and I did in the late 1960s and early 1970s at HumRRO as one of the thirteen most influential lines of literacy research since 1961. Reach Higher, America: Overcoming Crisis in the U.S. Workforce...
Related Sites:
pdf format
Foreward
"..Lee Kwan Yew, the first Prime Minister of Singapore and architect of its economic miracle over the past half century, once reasoned that a healthy economy depended on four fundamental requirements: dependable electrical power, clean potable water, world-class transportation, and an educated and trained workforce. In our current global economy, we can no longer take these precepts for granted."
....., from national commission adult literacy
".. calls for a dramatically revamped service system with the capacity to effectively serve 20 million adults annually by the year 2020..
Section A. FACING THE PRESENT
2. Our Treacherous Path:
...Americans should have been stunned when the National Assessment of Adult Literacy (NAAL), released in 2005, revealed that a staggering 30 million American adults scored at “below basic”—meaning they could perform no more than the most rudimentary literacy tasks. Another 63 million adults could perform only simple, basic everyday literacy activities.
The NAAL findings are ominous because most good jobs require at least some education beyond high school. The NAAL found that of the approximately 222 million adults aged 16 or older living in households or prisons in the United States, some 93 million lack literacy at a level needed to enroll in the postsecondary education or job training that current and future jobs require...
[International Scale of Top H.S. Graduates-US #11; Korea #1]
3. The New Landscape
[National Demographic of States with College Diplomas-Texas #1]
This is a tall order, but it is what the U.S. job market demands. The Bureau of Labor Statistics forecasts that between 2004 and 2014, 24 of the 30 fastest growing occupations will require workers with postsecondary education or training...
Lack of Teacher Certification and Benefits. (pp 13)
High-quality instruction is essential to foster student retention and produce successful outcomes. But salaries for adult educators are low, and many positions lack benefits. This limits the pool of potential instructors and leads to high turnover. While many adult education instructors are credentialed for K–12 teaching, few states require that adult education teachers show mastery of the specialized knowledge and skills needed to teach adults. Moreover, states have not established certification systems to ensure that all instructors have this ability.Worse yet, the majority of adult educators nationwide are working part-time, often as a second job. This is another clear sign that adult education is not considered a serious professional enterprise.
-David J. Rosen
"World Education, Boston, MA
International consultant in non-formal education to the government of the Philippines. Sponsored by the Asian Development Bank and the Royal Norwegian Government. Helped to develop national adult education system including elementary and secondary equivalency certificate alternatives. 2/95-3/95; 11/96-12/96; 6/97-7/97; 1/98-2/98; 7/98; and 2/2000.Project awarded UNESCO Literacy Project of the Year recognition in 2000. Quality Assurance Consultant on performance-based assessment, curriculum development, and teacher training, for school-based and out of school youth Integrated Production and Pest Management farmer field school project. July, 2004 - April 2005.
Consultant to the World Education Literacy Division on developing an intermediary role in Boston-based workforce development, 2004; conduct of scan of national adult literacy organizations for the National Institute for Literacy, April, 2005 - present; development of state Professional Development Distance Learning plan for the System for Adult Basic Education Support (SABES), November, 2005 - April, 2006.
ONLINE DISCUSSION:
Sal
"After watching (webcast form) and then reading the transcript (document form) of this report, I'm remembered as a child of easily understanding picture stories (e.g. quantitative literacy-71 million/32 % performed) instead of just words (e.g. prose-60 million) & (e.g. documents-48 million). Despite their data results concluding the adults did more poorly in the quantitative (using numbers) section, I feel math or working with numbers is complicated for certain groups.
The high percentage of only one language (Spanish) population that did poorly wasn't a surprise from my experience working with many migrant workers from Mexico. Also, the same goes with the population that didn't finish high school because of my experience with GED students in our ABE program.
The "Adults Just Don't Know How Stupid They Are" discussion was somewhat interesting, which I thought giving an international scope brought some enlightenment. The data on the importance of "apprenticeships" shows the importance of early programs (e.g. "OJT"-On the Job Training; Internships, etc..) in high school age level at least to college age level. I feel there is that different level of "pride" on every individual that we already know everything that we need to know on various subjects or skills (e.g. driving).
I feel it's the responsibility of "all of us" (researchers and adults themselves) on determining who is right about whose literacy levels pose a "problem". Instead of "criticizing" each other, we need to "work together" as a global society in literacy. Ever heard of the quote, "What we got here is failure to communicate" (used in a rock song from one of my favorite 80's band-Guns N' Roses)? I feel as ESL teachers or potential ESL teachers, we have a BIG task-role in teaching literacy to the "world". Thus, bridging different nations of different cultures of many ethnic backgrounds. Literacy is one way to communicate to each other, which can solve many of our social problems (e.g. wars that are somewhat caused by "misunderstanding" of one another)
An example of "working together" that's in the process in the area I'm teaching ESL as a volunteer is with our local dairy farm. They employ over 250+ "Spanish-speaking" migrant workers (many from Mexico) and they took the initiative (after hearing from the community) to start a "Spanish language" program for their "English-speaking" American workers. The guy that teaches "Basic Spanish" to them just moved up here from the Twin Cities the beginning of this year. Ironically, his wife was an ELL/ESL student in our program. That is how we found about this! They plan to hire a teacher to teach "English" to their "Spanish-speaking" migrant workers down the road.
Lastly, I would like to comment on the "Ohio Portfolio System", which I feel is a very good idea. I've learned somewhat about doing portfolios for our ELL/ESL students in the past. However, it is sometimes hard to do one when we are challenged with "retention" rates of our students. This is another "on-going" subject to discuss about later!
""arbitrary" literacy skills"-"hand on learning experience ( Newgate school here in Minneapolis)-Todd W.
' Algebra skills may not be part of my everyday life but I think the ability to 'think' mathmatically enhances brain function and math is a language..."-Jeannie M.
"...http://wdr.doleta.gov/SCANS/.... I believe SCANS was mentioned in one of our readings. At the workforce center, I had to make sure my students were achieving these skills. I developed a curriculum I called SCANS QUEST to help the supervisors and our student workers engage in tasks that supported SCANS.-Joyce
"In 1990, the Secretary of Labor appointed a commission to determine the skills our young people need to succeed in the world of work. The commission's fundamental purpose was to encourage a high-performance economy characterized by high-skill, high-wage employment. Although the commission completed its work in 1992, its findings and recommendations continue to be a valuable source of information for individuals and organizations involved in education and workforce development."
4.4 CASAS

4.5 Bests Tests

4.6 Accuplacer

Integrative Paper #1 – Due July 25th

Submit a 1 – 2 page paper in which you reflect upon your personal experience with assessing adult second language skills in light of the concepts from the class. What assessment methods and techniques do you use in your program and classroom? How can some of the concepts discussed in class and in the readings help you understand what you are already doing or what you could do in the area of adult second language assessment? Integrate the following into your reflective statement:

The assessment methods and techniques we use in our program and classroom are in various forms. For standardized testing, we use the CASAS (for ELL, ESL, ESOL students) and TABE (for GED students).

1. Why and when are your students assessed?

As I stated in the "Discussion Board" (Module 4.2), we usually assess them within the first 12 hours of their class visit...

"We use the CASAS, which we have trouble-challenges with the "pre-testing within the first 12 hours". Awhile back, when we met students for the first time, we would give them the CASAS test and they seem to be overwhelmed. Now, we decided to just wait after the first class visit or another better time (within the "first" 12 hours). That first day of class makes a huge impression on what this class will be like. Thus, this will impact their decision to return or not! Unfortunately, we sometimes never see (due to many factors-migrant work is "unpredictable") these students again and would never get the chance to give them the CASAS. At other situations, we do give them the CASAS and they would never come back (after 3-5+ more class days) and we will never have the chance to give the "post-test" to see how or if they ever improved to another level."

The purpose of these test as a "local" literacy program is for our "overseers" within the state...

"..Accountability: "Yes, the accountability or test results from NRS testing has motivated us teachers to do better for many personal reasons (e.g. keep the program going in this area for this county-wide community , job security for "paid-teachers", and want to see our local program and the region overall we represent to do well). It keeps us on our toes! .."

For our GED students, we assess them in a different way or standard...

"Our GED students are the ones that we usually focus on helping get employment and other various goals."

Overall our ELL/ELL/ESOL students are assessed differently right from the start compared to our GED students. We have them fill a "goal" sheet, which our "local program" gets credit if they do reach their goal. For example, the common goal listed for our ESOL students is "Learn English". For our GED students, it's getting their "GED Diploma" and/or "getting employment". Then throughout the course of the term or year, we write their individual goals and accomplishments each class session they participate on a separate goal sheet. These are some of the assessment tools and techniques we have been trained by the "state" for our "local" ABE program. The purpose of this is to show their "progress" and to show the individual learner/student what he/she has done so far. Therefore, to hopefully motivate them to continue to strive to reach that goal we both came to agreement in the beginning of the term/year. This informal form of assessment helps us "test-givers" or "teachers" to keep record on what we've done and what to do to help this invidual learner "reach their goal(s)". Also, to serve as data for the state in case we get "evaluated" for funding purposes. Along with this information, we have all types of other records (e.g. homework) for the individual students in a "portfolio" type file folder.

2. What assessment tools and techniques have you used? What was your purpose in using them?

(see above)

3. How do the tools and techniques you have used relate to the desired outcomes of your course or program? How would you describe the actual outcome?

The tools and techniques we have used has met both desired and undesired outcomes of our course or program. We "try" to keep improving it as we "make mistakes" and "learn". To first share some positive outcomes, we've been able to help students (GED) get their GED Certificate and employment. We've seen some progess of some of our ESOL students. For example, when we first "restarted" this program back in 2004, we had a good-size group of "migrant workers" from Mexico. One particular student name "Tulio" was one of the most ambitious, which he would come regularly. I remember taking him out to a community setting and introduced him to some people to informally practice his English. He stopped coming to our class one day and we didn't know why. Well, we found out he became manager of a particular dairy farm site! I personally feel what helped him reach this particular position was "researching" on the internet on resources related to his work. He was a dairy farmer and I found this "dairy terminology" site that translated English to Spanish. I would end up sharing this site with future "migrant workers" that worked on this particular field. He still continues to have this position to this day and he ended up inviting his family-relatives to come work in this area.

Some of the "undesired" outcomes are students that never reach their "goal" that they stated in the beginning of the year/term. We have one student name "Manuel" that kept coming regularly and stopped for about a 1/2-1 year. With our local classroom environment, we sometimes get "very busy" and forget or don't have time to "follow-up" with some of these students. Well, Manuel ended up coming back and would come regularly for awhile. However, we didn't see any "progress" based on his conversations with us. We can pin point to many factors that causes this, but we certainly feel that we could've done much better with this individual student. Some of other "undesired" outcomes is uncertain because we get so many students in the beginning, but they never finish or "come back". We will sometimes never know the outcome!

4. How would you characterize the tools and techniques you used in terms of the testing concepts presented in class?

I feel we have been using the tools and techniques already, but we just didn't know "how to use them" the right way all the time. For example, in Module 4.3 when we were assigned to read the results of the National Assessment of Adult Literacy (NAAL) 2003. I was able to related to what Dr. John Stucker stated...

"And then the other thing about these folks is, again, it's remarkable how fast these ELLs have picked up English conversation. But don't let that fool you. If you actually get a chance to assess their reading skills in English, it turns out they are extremely idiomatic when they talk to you. If one person walked in and joined us with this panel, we think gosh! That person speaks great English but when they went to do embedded literacy tests, such as the NAAL, those tests were hard for them.."

I had one student from Bulgaria that quickly lear