Foundational Day 1 ‐ The Brain and Learning
This workshop references the most current research on how the human brain learns. Through a simple information processing model, it focuses on function rather than structures in the brain. The workshop contains sessions on:

1. How the human brain processes information
2. Sensory preferences of teachers, professors, trainers and learners
3. Why new information needs to make sense to the learner
4. How teachers, professors, and trainers could apply what they learned in the previous three sessions to facilitate learning in their own classrooms
5. The primacy/recency effect and its use as a scientific basis for time management in the classroom
6. Purposefully designing questions for use before instruction with the specific intention of increasing learning

Among other things, in the ALTS Brain and Learning Workshop participants apply the learning benefits of: Retrieval practice and guided practice; learning and forgetting curves; prior knowledge tests, pretests and posttests; anticipatory sets including pre-questions; and Venn diagrams for analyzing reading materials.

Teachers, trainers and professors also gain an awareness of their own preferences for receiving information, how this impacts lesson planning and course development, and what they could do to give all of their students the opportunity to learn.

By the end of this exciting professional development program, even experienced teachers feel motivated and better equipped to be effective learning leaders.

Foundational Day 2 - Connecting Curriculum, Instruction and Assessment to Maximize Learning
This workshop gives participants an understanding of levels of cognitive functioning and how these are used in curriculum documents, lesson planning, instruction and assessment. To participate actively workshop participants need to bring a set of learning objectives with which they are familiar, e.g. the syllabus for one of the courses they teach.

When Bloom’s Taxonomy is used to align assessments with curriculum and instruction, it improves the content validity of assessments, provides teachers with more accurate information regarding what students really know and can do, and results in assessments students consider to be fair. Included in this workshop is a refresher course on Bloom’s Taxonomy that empowers teachers to use it for lesson planning, instruction and assessment.

In this hands-on professional development program, workshop participants write questions to assess specific learning objectives from one of their own courses. These questions are purposefully written to target the specific levels of cognitive functioning.

If teachers, trainers and professors use different taxonomies from Bloom's, they are presented with the opportunity to:

Towards the end of this comprehensive professional development experience, participants learn how to create a Table of Specifications (Test Blueprint) and how it could be employed to develop assessments of high content validity. This is a natural extension of the conversations regarding cognitive complexity initiated during the earlier sessions on Bloom’s Taxonomy, and sets the stage for Foundation Days 3, 4 and 5 which focus on creating effective assessments, question writing and the development of appropriate scoring guides.

Foundational Day 3 – Assessment: Terminology, Concepts and Effective Construction
Assessments for Different Purposes
A clear differentiation is made between questions and assessments. Then the conversation started in Day 1 regarding the similarities and differences among prior knowledge tests, pretests, posttests and prequestions is extended with additional information and discussed in greater depth. Following this, the use of test scores in a criterion-referenced manner are compared and contrasted with the use of test scores used in a norm-referenced manner. Then assessments used for formative and summative purposes are compared and contrasted.
This sets the stage for a discussion of the concepts of validity and reliability, considering the purpose of an assessment, the time available for assessment and the number of questions that can be used.

Validity and Reliability Concepts
This is an easy to follow presentation about two of the most fundamental concepts in the field of Educational Assessment – validity and reliability. By the end of this session participants have a conceptual understanding of validity and reliability and how they are related to each other in the context of an assessment.

Reliability, Sampling, Content Validity
In this session participants learn how planning and the purposeful selection of test questions are important to reliability and validity. Participants are introduced to statistical sampling concepts and the use of a table of specifications in order to achieve high content validity. Relevant connections are made to show how this process results in assessments that are well-aligned with the target curriculum and instruction.

How to develop tests that work
In this session, participants are introduced to a simple test development process. The steps in this process have been distilled from decades of practical experience in the test development industry and combined with useful tips to empower teachers, professors and trainers to create high quality criterion-referenced assessments. Issues of reliability, fairness and validity are addressed.

Numbering and sequencing items within an assessment
After all of the questions on an assessment are selected, they should be arranged in a logical order that allows test takers to turn in their best possible performance. There are ways of arranging and sequencing questions to reinforce the concepts and principles presented during instruction. To this end participants receive practical guidelines for numbering and sequencing items within an assessment. These guidelines are different for online vs. paper assessments, both are discussed.

Terminology and Statistics (Optional)
This session reviews basic assessment terminology and statistics that are often used in communicating assessment results, such as the median, range, mean, item difficulty, standard deviation, variance, correlation and reliability. By the end of this session participants will have the tools they need to be more informed consumers of assessment information.

What makes a great question?
This workshop begins by identifying the distinguishing characteristics of an effective question and provides criteria that teachers, professors, trainers and course developers can use to evaluate the quality of the questions in their own assessments.

How to write effective multiple‐choice questions
This session begins with the anatomy of a multiple‐choice question, and highlights the advantages and disadvantages of using multiple‐choice questions. Participants receive guidelines on how to write good multiple‐choice questions and what pitfalls to avoid when writing effective multiple‐choice questions.
By the end of this workshop session participants will be able to identify common cues that might inadvertently give clues to the correct answers, and explain how distractors influence question performance. Participants learn how to interpret distractor analysis results and how to write effective distractors.

How to write effective true/false, yes/no and “fill-in-the-blank (with a word bank)” questions
This session highlights the advantages and disadvantages of select-type questions. Participants receive guidelines on how to write effective questions of each type and what to avoid when writing these questions.

How to write short and long constructed-response questions that work
This session highlights the advantages and disadvantages of supply‐type questions. Participants receive guidelines on how to write effective short constructed-response questions and what to avoid when writing such questions.

How to write effective essay questions
This session highlights the advantages and disadvantages of essay questions. Participants receive guidelines on how to write effective essay questions and what to avoid when writing these questions.
The session emphasizes the need for a well‐designed rubric to score essays with consistency, and describes the purpose of inter-rater agreement analyses.
Foundational Day 5 – Scoring Guides, Rubrics and Checklists
This full day workshop introduces scoring guides, rubrics and checklists with several examples. Participants get the opportunity to develop a scoring guide for a long constructed response question. Participants also get the opportunity to develop a rubric to score one of their own essays, projects, or portfolios. It becomes abundantly clear that we cannot evaluate the quality of an essay question/project description/performance task/portfolio without also evaluating the rubric that will be used to score it.
To complete this very hands-on workshop, participants get the opportunity to develop a checklist to score one of their own performance tasks.
For this workshop to run smoothly and provide the greatest return on participants’ time investment, participants will be asked to bring a long constructed response question and a model answer that goes with it, as well as an essay question/project description/portfolio prompt, and a performance task.

Additional days and sessions can include:
Optional Day 1 – Purposeful Questions and Questioning Strategies that Increase Learning
This workshop builds upon the foundation laid in the Foundational Day 1 and Day 2 professional development experiences, and helps participants make and consolidate connections. It begins with a session on the learning benefits to be derived from “brain compatible” questioning strategies. The case is made that questions used to stimulate prior knowledge centers in the brain before instruction begins, are qualitatively different from questions used to check for understanding while each individual student is involved in the learning process, and that these questions are qualitatively different from questions used to evaluate whether the new material has been consolidated and saved in long-term storage centers in the brain. Teachers, trainers and professors are encouraged to design questions with their specific learning purposes in mind, and to use the answers to these questions as feedback when choosing new learning activities, and modifying their instructional approach to increase learning. During this workshop participants discuss different teaching/learning strategies and apply questioning strategies that make use of “wait time”; “intelligent sampling” techniques; and, conversation frames for summarizing information.

In this engaging professional development program, workshop participants have the opportunity to collaborate with colleagues to prepare a lesson plan, and used a proprietary ALTS checklist to ensure that they carefully and purposefully considered questions for use before, during and after instruction – with the goal of maximizing learning.

Optional Day 2 – The Learning Benefits of Formative Assessment: Assessment for Learning
This workshop builds on the foundation in Days 1 thru 5, and makes particularly important connections with conversations initiated in Foundation Day 3 regarding the purpose of assessment, the need for high quality feedback, and various techniques for providing high quality feedback to learners. Participants are encouraged to focus on the purpose of the assessment and participate in discussions comparing and contrasting the value of immediate feedback and delayed feedback.

It becomes very clear that questions, assessments and high quality feedback are all essential to optimizing learning. In agreement with the work of Alfie Kohn, it also becomes exceeding clear that, if the goal is to increase learning, all of these are more important than sanctions, punishment, and extrinsic rewards such as grades.

A recommendation from this workshop is that teachers, professors and trainers should invite learners to make mistakes, describe their reasoning and learn how to correct the errors in their reasoning. Then learners should be given the opportunity to use the new and improved reasoning to solve problems, analyze information and create something new. When this happens learners gain confidence with the correct reasoning and their self concept changes from one where they expect failure to one where they expect success. The priceless results are confident learners with correct reasoning, who can learn more and contribute inventions to society.

In this workshop, participants get the opportunity to develop one summative assessment and at least three formative assessments that could be used before the summative assessment (final) is administered. The value of “mock” summative assessments (parallel forms) for communication of expectations regarding levels of cognitive processing are discussed. Participants become actively engaged in making connections with the test blueprint, and discussing the pros and cons of providing feedback at the end of the “mock” summative assessments.

Optional Day 3 – Cut Scores and Performance Level Definitions
How to write Performance Level Definitions (PLDs)
This session begins with a clear definition of “proficient” performance based on the desired outcome performance described in the approved curriculum document. In this session, participants learn how to write test‐specific PLDs that clearly define what students at different proficiency levels need to demonstrate that they know and can do. Participants bring their curriculum documents, instructional materials and at least one assessment. They work in groups to develop test specific PLDs for one test. These PLDs form the basis for the cut score setting (standard setting) activities in the session which follows.

How to set cut scores using a scientific and legally acceptable method
In this session, qualified ALTS personnel lead workshop participants through a process to create cut scores for a specific assessment. ALTS recommends using the Modified Angoff technique or the Bookmark procedure, and will usually consult with the school/school district to determine which method best meets their long term needs.

Participants use the test‐specific PLDs developed in the previous session, to write descriptions of what a barely proficient student knows and can do. Other “barely” descriptions are written after the participants have mastered writing the barely proficient description. Depending on the number of cut scores needed, these may include a barely advanced description, i.e. a written description of what a barely advanced student knows and can do; as well as a barely progressing description, i.e. a written description of what a barely progressing student knows and can do.
After writing the “barely” descriptions, participants use them to set cut scores for the target assessment. Through this hands-on experiential activity participants engage in an actual cut score setting exercise, and are equipped to do their own cut score setting with other assessments in the future. This becomes invaluable when teachers, professors and trainers need to know whether students have mastered the material taught.

A single cut score (indicating whether or not mastery has been attained) is usually sufficient, for the outstanding pedagogue to answer the question: “Are the foundational building blocks in place for me to proceed to the next lesson in the instructional sequence?” Note that when scores from an assessment are used to indicate whether or not mastery has been attained, these scores are being used in a criterion-referenced manner. Also note that assessments used to produce scores for use in a criterion-referenced manner require most of the questions to provide information close to the cut score in order to clearly differentiate masters from non-masters. Thus, such assessments are designed much differently from assessments used to make norm-referenced decisions. Terms like “criterion-referenced” and “norm-referenced” decisions are defined and discussed in the Foundational Day 3 Workshop.

Establishing mastery is also essential when an elearning system needs to route learners through a system of learning modules in which some are prerequisites for others. Setting an accurate cut score prequalifies learners for the next stage in the learning sequence and sets them up for long term success! Knowing how to set accurate cut scores using a scientific and legally acceptable process is essential for outstanding and responsible performance as a teacher, professor, corporate trainer, course developer, curriculum specialist, and really for any responsible educator in any field.