DPS Memo 03-017.doc

FLORIDA DEPARTMENT OF EDUCATION

�

CHARLIE CRIST

Commissioner

M E M O R A N D U M

€

TO:

District School Superintendents

€

FROM:

Betty Coxe

DATE:

August 9, 2002

BETTY COXE

€

DEPUTY COMMISSIONER

€

EDUCATIONAL PROGRAMS

€

CONTACT PERSON

NAME:

Dr. Thomas Fisher

€

PHONE:

(850) 488-8198

€

SUNCOM:

278-8198

€

EMAIL:

thomas.fisher@fldoe.org

€

DPS:

03-017

SUBJECT

Return of 2002 FCAT Writing Responses on Compact Disk

The purpose of this memorandum and the attachment is to provide information about

the return of student responses from the FCAT Writing assessment in 2002.

Due to

recent technological advances in the high-speed scanning of student answer

documents, it is now possible to create an electronic copy of the FCAT Writing

responses.

Imaging is a process similar to desktop scanning; however, it is much

faster. The scanned images are stored as electronic files and saved on compact disks

(CDs) for distribution.

Guidelines for use of the FCAT Writing CDs are addressed

herein and in the question-and-answer document attached.

Two sets of CDs containing student responses from the February 2002 FCAT Writing

assessment have been distributed to school districts: one set is organized by school,

and one set is organized by district.

The CDs were shipped to district coordinators

along with the shipment of FCAT Writing demographic reports.

Each district will

determine local distribution procedures for the CDs.

August 9, 2002

€

Memorandum – FCAT Writing CDs

€

Page 2

€

The FCAT Writing CDs include a copy of both pages on which a student might have

written his or her response. If the student wrote on only one page, the second page will

be blank. The student’s name, student identification number, and score are included on

both of these pages.

The school-level CDs are organized alphabetically by the

student’s last name for the school where the test was administered. The district-level

CDs are organized alphabetically for the entire district. Information on the CD is stored

in PDF format and can be accessed using Adobe Acrobat Reader

�

. The CD also

contains this program.

Individual responses can be easily located by searching for

either the student’s name or identification number.

The file contains the name and

identification number used on the student’s answer document for the 2002 FCAT

Writing administration.

Information found on the CDs is confidential and should be handled with the same

security precautions used to protect all student work, especially test scores. Because

the FCAT Writing CDs contain each student’s actual response for the 2002 test, as well

as the score he or she earned, schools and districts must ensure that individuals who

are given access to this information have a legitimate educational interest in the child or

children whose responses are retrieved.

Districts and schools should already have

guidelines in place to address the storage and retrieval of test information.

The attached question-and-answer document addresses, in more detail, the use of the

student responses on the CDs. If you have questions, please call Dr. Thomas Fisher at

the telephone number listed above.

Attachment

cc:

Assistant Superintendents for Instruction

District Language Arts Supervisors

District Assessment Coordinators

FCAT Writing Essays on Compact Disks

Questions and Answers

1. What is the FCAT Writing CD?

The 2002 FCAT Writing student responses are being returned to school districts on

compact disks (CDs).

Recent technological advances in the high-speed scanning of

student answer documents make it possible to create an electronic copy of the FCAT

Writing responses.

Imaging is a process similar to desktop scanning; however, it is

much faster. The scanned images are stored as electronic files and saved on compact

disks (CDs) for distribution.

2. Is this the first time the DOE has returned FCAT Writing responses?

Yes.

The responses from the 2002 FCAT Writing assessment are the first writing

responses returned to school districts.

3. When will the school district receive the compact disks?

The CDs were shipped to the district assessment coordinators in August 2002, along

with the shipment of FCAT Writing demographic reports. Each district will determine

local distribution procedures for the CDs.

4. What information is included on the CDs?

The FCAT Writing CDs include a copy of both pages on which a student might have

written his or her response. If the student wrote on only one page, the second page will

be blank. The student’s name, student identification number, and score are included on

both of these pages.

5. What is the format of the information stored on the CDs?

Information on the CD is stored in PDF format and can be accessed using Adobe

Acrobat Reader

�

. The CD also contains this program. Individual responses can be

easily located by searching for either the student’s name or identification number. The

file contains the name and identification number used on the student’s answer

document for the 2002 FCAT Writing administration.

The school-level CDs are

organized alphabetically by the student’s last name for the school where the test was

administered. The district-level CDs are organized alphabetically for the entire district.

6. Is the information on the CDs confidential?

Yes. Strict privacy laws protect student work, especially test scores. Information found

on the CDs should be handled with the same security precautions used to protect all

student work, especially test scores. Because the CDs contain each student’s actual

PLAZA LEVEL – 08, THE CAPITOL – TALLAHASSEE, FLORIDA 32399-0400 – (850) 413-0555 – FAX (850) 488-2869

€

http:/www.firn.edu/doe

€

An affirmative action/equal opportunity employer

€

response for the 2002 test, as well as the score he or she earned, schools and districts

must ensure that individuals who are given access to this information have a legitimate

educational interest in the child or children whose responses are retrieved. Districts and

schools should already have guidelines in place to address the storage and retrieval of

test information.

These same guidelines apply to the images of the FCAT Writing

responses contained on the CDs.

7. Can the writing responses on the CD be printed?

Yes.

The program used for reading the CD can also be used to print copies of

individual responses.

To print a student’s essay from the CD, first find the page

numbers that contain the essay. Select the print function from the “File” menu and type

in the appropriate page numbers for the essay to be printed. Providing specific page

numbers for the essay is critical in order to print only the needed essay and to eliminate

the chance of accidentally printing all of the essays contained on the CD. In addition,

access to the printer should be monitored to ensure that only authorized persons obtain

the printed copies.

8. Must we inform parents of the availability of the writing papers?

Parents have previously received a report of the FCAT Writing score. Districts are free

to decide how to communicate with parents about the availability of student essays.

Can copies of the essays be used for staff development?

After each administration of FCAT Writing, the DOE provides Grades 4, 8, and 10

Florida Writes!

publications. These documents, prepared for educators, contain detailed

information concerning FCAT Writing, along with sample student essays. Two student

responses for each of the 6 points in the scoring rubric for both prompts included in the

assessment are selected and included in

Florida Writes!.

The papers represent some

of the various writing techniques and skills students applied to respond to each prompt.

In addition to this document, the DOE provides each district with sets of annotated

student responses used during scorer training. Because these published essays have

been carefully selected to represent the FCAT scoring criteria, and have been edited to

remove all individually identifiable information, their use for staff development is

preferred.

Classroom teachers and/or school administrators may wish to examine essays from the

CD in order to improve their own instructional programs in writing. Before using essays

printed from the CD for school-wide staff development, care must be taken to protect

the confidentiality of students and other persons who might have been mentioned in the

student writing (e.g., teachers, parents, friends, or other persons). Similar precautions

must be taken if district administrators wish to use essays printed from the CDs.

Additionally, please note that papers brought to the attention of district personnel for

troubling content issues are included on the CD without any notation indicating the

PLAZA LEVEL – 08, THE CAPITOL – TALLAHASSEE, FLORIDA 32399-0400 – (850) 413-0555 – FAX (850) 488-2869

€

http:/www.firn.edu/doe

€

An affirmative action/equal opportunity employer

€

troubling content (threats or reports of harm to oneself or others).

Districts should

carefully consider whether to use such papers in any training program.

10.

How should school and district administrators respond to questions about

the score a student essay received?

Questions about the scores given to papers are the perfect opportunity to share more

information about the teaching of writing, the test, and how the essays are scored. The

following recommendations should be considered when reviewing the scores given to

essays.

Section I – Understanding the Scoring of FCAT Writing

The student essay is only one sample of student writing completed during one 45-

minute testing session. Students do not have time to revise the essay, so the writing

is scored as a draft.

Within an instructional setting, when students are given the

opportunity to plan, research, draft, revise, and edit writing, the result may be a more

polished product than the one produced for FCAT Writing. Samples of student

writing taken under other conditions and for other purposes may be different from

the writing done under testing conditions.

Each student essay is evaluated for its overall quality using focused holistic scoring

that is based on four elements of writing: focus, organization, support, and

conventions. Explaining more about these elements of writing will help to clarify the

scoring rubric. Specific scores on papers should match the descriptions found in the

scoring rubric for that score point at the appropriate grade level. Classroom scores

on student writing may differ from scores on FCAT Writing for several reasons.

Classroom writing tasks may vary from the format used on FCAT Writing. Teachers

may not use the FCAT Writing rubric in scoring classroom writing, or the rubric may

be applied differently than the way it is used for state scoring.

These and other

aspects of the scoring of FCAT Writing are described more fully in

Florida Writes!

and other FCAT publications.

At least two professional readers, who meet stringent qualification criteria, score

each essay.

The qualification criteria include a bachelor’s degree in English or a

related field (journalism, literature, communication, etc.), a demonstration of writing

proficiency, the completion of 2-3 days of training, and a passing score on a

qualifying test. Prospective readers must learn how Florida teachers want the

essays scored by reviewing papers previously scored by those teachers.

To be

hired, prospective readers must accurately score a set of papers, the qualifying set,

by matching the scores previously assigned to the papers by Florida teachers.

The accuracy of each reader’s scoring is checked daily. A reader must continue to

score consistently with papers that were scored by Florida educators (validity) and

with other readers (reliability).

If readers do not meet the rigorous performance

criteria established by the Department of Education, they must be re-trained or

dismissed. When the two scores assigned to an essay are not adjacent, the paper

PLAZA LEVEL – 08, THE CAPITOL – TALLAHASSEE, FLORIDA 32399-0400 – (850) 413-0555 – FAX (850) 488-2869

€

http:/www.firn.edu/doe

€

An affirmative action/equal opportunity employer

€

is read and scored by a third, and sometimes a fourth, experienced reader or

supervisor. The DOE writing assessment specialist must approve all scores that are

resolved by a fourth reading.

Section II – Reviewing Student Scores on FCAT Writing

To review a student’s score using the CD, print a copy of the student paper to verify

that it contains the work of the named student and that the student’s identification

number and score match the information on the student’s score report. If the work

attributed to the student does not legitimately belong to that student, the situation

should be reported immediately to the district assessment coordinator, who reports

the details to the DOE for investigation.

The school administrator or teacher responsible for writing instruction in the school

should meet with the parent or teacher of the student whose score is in question. At

this point, a more detailed examination of the various elements of writing (focus,

organization, support, and conventions) contained in the student’s paper should

occur. The student’s paper should be compared to the examples available in

Florida

Writes!

A paper similar to the student’s will likely be found in the examples for the

exact or adjacent score points as that received by the paper.

If the paper is

determined to be more like a paper for a score more than one point greater than or

less than the student’s score (a non-adjacent score), consult with the district

language arts supervisor about the next steps.

The district language arts supervisor can provide additional insight about the

elements of writing and the focused holistic scoring method.

The language arts

supervisor should review the paper based on the resources provided by the DOE

and describe to the parent or the school official how the scoring criteria align with the

paper. If the language arts supervisor determines that the paper has been assigned

a score that would be more than one point greater than or less than the score

assigned to the student (a non-adjacent score), he/she should consult with the

district assessment coordinator about the next steps.

Based on the recommendation of the district language arts supervisor and

agreement by the district assessment coordinator, the paper can be referred to the

DOE for review. A written request for review of the paper must be submitted along

with a copy of the student’s paper. The written request should include a description

of the review steps completed and a rationale that describes why it is hypothesized

that the paper should have received a score that is more than one point greater than

or less than the score given (a non-adjacent score).

Written requests for review of scores should be submitted on district letterhead to

the K-12 Assessment Section of the Florida Department of Education, 325 W.

Gaines Street, Suite 401, Tallahassee, FL

32399.

Prior to submitting a written

request to DOE, staff members of the K-12 section are available for telephone

consultations on any aspect of this review. Please call Lissa Cone (850/922-2584)

with questions about FCAT Writing, focused holistic scoring, or scoring rubric

PLAZA LEVEL – 08, THE CAPITOL – TALLAHASSEE, FLORIDA 32399-0400 – (850) 413-0555 – FAX (850) 488-2869

€

http:/www.firn.edu/doe

€

An affirmative action/equal opportunity employer

€

interpretations. Please call Jackie Mueller (850/488-8198) with questions about the

operational aspects of scoring and reporting or technical problems with the CD. All

written requests must be submitted to the DOE for review prior to January of the

year following the assessment, for example, January 2003, for the test administered

in February 2002.

Written requests for review of scores submitted to the Department of Education will

be evaluated using the same methods recommended above. Written responses will

be provided to the district assessment coordinator for each written request received.

13.

What materials are available to help teachers understand how the writing

essays are scored?

The pertinent Florida Department of Education publications about FCAT Writing listed

€

below will be helpful. These documents should be available at each school or can be

€

requested from the district assessment coordinator.

€

Florida Writes! Report on the 2002 Assessment, Grades 4, 8, and 10

€

The

Florida Writes! Reports

are designed for educators involved in teaching, planning,

€

and evaluating curriculum in the Florida public schools.

Each grade level publication

€

describes the content and application of the writing tests and offers suggestions for

€

activities that may be helpful in preparing students for the assessments.

€

2002 FCAT Writing Scoring Guide and Qualifying Set with Annotations, Grades 4, 8, 10

€

These annotated sets provide actual student responses with explanations regarding the

€

score for each paper and are released to district language arts supervisors and

€

assessment coordinators during the summer following each FCAT Writing

€

administration.

€

Florida Writes! Parents’ and Guardians’ Guide to the 2002 Results, Grade 4; Students’

€

Guide to the 2002 Results, Grades 8, 10

€

The purpose of the Parents and Guardians’, and Students’

Florida Writes! Guides

is to

€

help students, parents, and guardians understand the scores given to students on the

€

statewide writing test. The

Reports

and

Guides

are available annually each May, after

€

the FCAT results are published. The

Parents’ and Guard

ians’ Guide, Grade 4

€

translated into Spanish and Haitian Creole.

€

FCAT Performance Task Scoring—Practice for Educators Publications and Software

These materials are designed to help teachers learn to score FCAT reading, writing,

and mathematics performance tasks at grades 4, 5, 8, and 10.

Trainer’s Guide

includes instructions for using the scoring publications and software in teacher

education seminars and workshops. The publications model the scorer training

experiences by presenting samples of student work for teachers to score.

The

publications were distributed to the districts in April 2001, and the companion software

was distributed in July 2001.

PLAZA LEVEL – 08, THE CAPITOL – TALLAHASSEE, FLORIDA 32399-0400 – (850) 413-0555 – FAX (850) 488-2869

€

http:/www.firn.edu/doe

€

An affirmative action/equal opportunity employer

€

Lessons Learned

—

FCAT Sunshine State Standards and Instructional Implications

€

This document provides an analysis of previous years’ FCAT results and contains

€

analyses of state FCAT reading, writing, and mathematics data for the years since the

€

first administration. The analysis will assist educators in interpreting and understanding

€

their local FCAT scores, which will help improve instruction in the classroom.

€

Distribution to districts was in May 2002.

€

Keys to FCAT, Grades 3-5, 6-8, and 9-10

€

These booklets are distributed each January to district offices and contain information

€

for parents and students preparing for the reading, writing, and mathematics FCAT.

€

The 2002

Keys to FCAT

were translated into Spanish and Haitian Creole and are

€

available, along with the

Keys to FCAT

in English, on the FCAT home page on the

€

Department’s web site.

€

Florida Department of Education, Assessment & Evaluation

€

325 W. Gaines Street, Room 414, Tallahassee, Florida 32399-0400 (850) 488-8198

€

http://www.firn.edu/doe/sas/fcat.htm

€

PLAZA LEVEL – 08, THE CAPITOL – TALLAHASSEE, FLORIDA 32399-0400 – (850) 413-0555 – FAX (850) 488-2869

€

http:/www.firn.edu/doe

€

An affirmative action/equal opportunity employer

€