Program Evaluation Final Part 2
Program Evaluation Final Part 2
- 1.
In order from first to last, list the steps that should be taken when planning an evaluation.
- A.
Determine methodology, become familiar with information needs, examine the literature, present written proposal, identify program, and stake holders, execute agreement/contract
- B.
Identify program and stake holders, examine the literature,, become familiar with information needs, determine methodology, , execute agreement contract, present written proposal
- C.
present written proposal,identify program and stake holders, become familiar with information needs, determine methodology, execute agreement contract, examine the literature
- D.
Identify program and stake holders, become familiar with information needs, examine the literature, determine methodology, present written proposal, execute agreement contract
Correct Answer
D. Identify program and stake holders, become familiar with information needs, examine the literature, determine methodology, present written proposal, execute agreement contractExplanation
The steps that should be taken when planning an evaluation are as follows: first, identify the program and stakeholders involved. Then, become familiar with the information needs related to the evaluation. Next, examine the existing literature on the topic. After that, determine the appropriate methodology for conducting the evaluation. Following the methodology, present a written proposal outlining the evaluation plan. Finally, execute an agreement or contract to formalize the evaluation process.Rate this question:
-
- 2.
____________________ refers to obtaining the agreement from people to supply data for an evaluation after they have been adequately informed about what is expected of them.
- A.
Anonymity
- B.
Confidentiality
- C.
Informed Consent
- D.
Active Consent
Correct Answer
C. Informed ConsentExplanation
Informed consent refers to obtaining the agreement from people to supply data for an evaluation after they have been adequately informed about what is expected of them. This means that individuals are fully aware of the purpose of the evaluation, the type of data that will be collected, and how it will be used. Informed consent ensures that individuals have the opportunity to make an informed decision about participating in the evaluation and have the right to withdraw their consent at any time.Rate this question:
-
- 3.
The term "stakeholder" is used to refer to
- A.
Anyone affected by the program being evaluated.
- B.
People who pay for the evaluation of a program.
- C.
People who are not directly impacted by the program being evaluated.
- D.
Neutral third party observers.
Correct Answer
A. Anyone affected by the program being evaluated.Explanation
The term "stakeholder" refers to anyone affected by the program being evaluated. This means that it includes individuals or groups who have a vested interest in the program's outcomes or may be impacted by its implementation. These stakeholders could include program beneficiaries, funders, staff members, community members, and other relevant parties. The term "stakeholder" is broad and encompasses a wide range of individuals who have a stake in the success or failure of the program being evaluated.Rate this question:
-
- 4.
A ____________________ has occurred when evaluation results find that a program is not effective, when in fact it is effective.
- A.
Type I error
- B.
Type II error
- C.
Type III error
- D.
Statistical error
Correct Answer
B. Type II errorExplanation
A Type II error has occurred when evaluation results find that a program is not effective, when in fact it is effective. This error is also known as a false negative. It means that the evaluation failed to detect the effectiveness of the program, leading to an incorrect conclusion that the program is ineffective. This can happen due to various reasons such as small sample size, measurement errors, or inadequate statistical power.Rate this question:
-
- 5.
A ______________ has occurred when evaluation results find that a program is effective, when in fact it is not effective.
- A.
Type I error
- B.
Type II error
- C.
Type III error
- D.
Statistical error
Correct Answer
A. Type I errorExplanation
A Type I error occurs when evaluation results mistakenly conclude that a program is effective, even though it is actually not effective. This error is also known as a false positive, where the evaluation falsely identifies a relationship or effect that does not exist. In the context of this question, it means that the evaluation incorrectly suggests that the program is achieving its intended goals when it is not.Rate this question:
-
- 6.
_________________ are of great concern in program evaluation because they could lead to the elimination of a program that provides beneficial services.
- A.
Informed consent and confidentiality
- B.
Stakeholders with conflicting interests
- C.
Type I errors
- D.
Type II errors
Correct Answer
D. Type II errorsExplanation
Type II errors are of great concern in program evaluation because they could lead to the elimination of a program that provides beneficial services. Type II errors occur when the null hypothesis is not rejected, even though it is false. In the context of program evaluation, this means that a program may be deemed ineffective or not providing beneficial services, when in fact it does. This can have serious consequences, as it may result in the termination of a program that is actually beneficial and needed. Therefore, it is important to minimize the occurrence of Type II errors in program evaluation to ensure accurate decision-making.Rate this question:
-
- 7.
________________ are of great concern to evaluators because they could result in the continuance of programs that do not provide any benefit to participants.
- A.
Informed consent and confidentiality
- B.
Stakeholder conflicts
- C.
Type I errors
- D.
Type II errors
Correct Answer
C. Type I errorsExplanation
Type I errors are of great concern to evaluators because they could result in the continuance of programs that do not provide any benefit to participants. Type I errors refer to false positives, where the evaluator incorrectly concludes that a program is effective when it is actually not. This can lead to the allocation of resources to ineffective programs and the perpetuation of interventions that do not deliver desired outcomes. Therefore, evaluators strive to minimize Type I errors to ensure that programs are truly beneficial to participants.Rate this question:
-
- 8.
When establishing evaluation criteria, an evaluator does not have to be concerned about ___________________________.
- A.
Selecting criteria that reflect the program's intent.
- B.
Selecting criteria that staff can influence.
- C.
Selecting criteria that can be measure realiably.
- D.
Selecting only criteria that stakeholders want them to use.
Correct Answer
D. Selecting only criteria that stakeholders want them to use.Explanation
When establishing evaluation criteria, an evaluator does not have to be concerned about selecting only criteria that stakeholders want them to use. While it is important to consider stakeholders' perspectives and input, the evaluator's primary responsibility is to ensure that the selected criteria accurately reflect the program's intent, can be reliably measured, and are not influenced solely by staff preferences. The evaluator should prioritize objectivity and the overall effectiveness of the evaluation process rather than solely catering to stakeholders' desires.Rate this question:
-
- 9.
Which of the following is NOT a typical evaluation question asked in a program evaluation?
- A.
Are the resources devoted to the program being expended appropriately?
- B.
Does the program or plan match the needs of the people to be served?
- C.
Do program outcomes achieved match the program goals?
- D.
Does the program plan match the values of the evaluator?
Correct Answer
D. Does the program plan match the values of the evaluator?Explanation
This question is asking about a typical evaluation question asked in a program evaluation. The other three options are all common evaluation questions that are typically asked in program evaluations. However, the question about whether the program plan matches the values of the evaluator is not a typical evaluation question. Evaluations should be objective and focus on the program's effectiveness, outcomes, and alignment with the needs of the people being served, rather than the personal values of the evaluator.Rate this question:
-
- 10.
__________________ goals refer to how and when program activities are established and carried out.
- A.
Implementation
- B.
Intermediate
- C.
Outcome
- D.
Activity
Correct Answer
A. ImplementationExplanation
Implementation goals refer to how and when program activities are established and carried out. This means that they focus on the practical aspects of executing a program, such as setting timelines, determining the order of activities, and ensuring that tasks are completed according to plan. Implementation goals are essential for ensuring that program activities are effectively implemented and that desired outcomes are achieved.Rate this question:
-
- 11.
Intermediate goals are the things that are expected to occur to participants as a result of the proper implementation of the program activities, and _________________ goals constitute the final goals (or impact) intended for the program.
- A.
Implementation
- B.
Output
- C.
Short-term
- D.
Long-term
Correct Answer
D. Long-termExplanation
Long-term goals are the final goals or impacts intended for the program. These goals are expected to be achieved as a result of the proper implementation of the program activities. Intermediate goals, on the other hand, are the things that participants are expected to experience along the way as a result of the program implementation.Rate this question:
-
- 12.
One of the advantages of using program participants as a source of evaluation data is that
- A.
It is a relatively inexpensive way to gather data.
- B.
Measurement is non-reactive.
- C.
Participants can provide data on nearly all aspects of the program.
- D.
Data collection for evaluation is the highest priority for participants
Correct Answer
C. Participants can provide data on nearly all aspects of the program.Explanation
Using program participants as a source of evaluation data is advantageous because they have firsthand experience and knowledge of the program. They can provide data on various aspects of the program, including their own experiences, outcomes, challenges, and suggestions for improvement. This comprehensive data can give a holistic view of the program's effectiveness and help identify areas that need attention. Additionally, involving participants in the evaluation process can enhance their sense of ownership and engagement with the program.Rate this question:
-
- 13.
Evaluations are more likely to be effective if the list of variables to be measured
- A.
Is lengthy and includes all variables that could possibly be measured.
- B.
Is limited to the most important variables.
- C.
Includes some personality characteristics.
- D.
Includes the most expensive variables to measure.
Correct Answer
B. Is limited to the most important variables.Explanation
Evaluations are more likely to be effective if the list of variables to be measured is limited to the most important variables. This is because focusing on the most important variables allows for a more targeted and efficient evaluation process. Including all variables that could possibly be measured would be time-consuming and may dilute the focus of the evaluation. Additionally, including some personality characteristics or the most expensive variables to measure may not necessarily contribute significantly to the effectiveness of the evaluation.Rate this question:
-
- 14.
In selecting variables that are to reflect program success, it is useful to choose variables that are
- A.
Historical, instrumental, and well-known.
- B.
Hidden, covert, and implicit.
- C.
Summative, formative, and sensitive.
- D.
Important, sensitive to change, and cost-effective.
Correct Answer
D. Important, sensitive to change, and cost-effective.Explanation
In order to accurately reflect program success, it is important to choose variables that are important, sensitive to change, and cost-effective. Important variables are those that directly impact the success of the program and are relevant to its goals. Sensitive to change variables are able to detect even small changes or improvements in the program's outcomes. Cost-effective variables are those that can be measured and monitored without excessive time, effort, or resources. By selecting variables that possess these qualities, the program can effectively evaluate its success and make informed decisions for improvement.Rate this question:
-
- 15.
The most widely used type of measurement procedure in program evaluation is
- A.
A community level index.
- B.
The written survey completed by program participants.
- C.
A rating of the program by a significant other.
- D.
An observation made by an outside expert.
Correct Answer
B. The written survey completed by program participants.Explanation
The most widely used type of measurement procedure in program evaluation is the written survey completed by program participants. This method allows for gathering quantitative data from a large number of individuals, providing valuable insights into their perceptions, experiences, and outcomes related to the program. Surveys are cost-effective, easily administered, and can be standardized to ensure consistency in data collection. Additionally, they allow for anonymity, which may encourage participants to provide honest and unbiased feedback. Overall, surveys are a versatile and reliable tool for evaluating program effectiveness and identifying areas for improvement.Rate this question:
-
- 16.
_______________________ is the procedure of gathering information that does not prompt or influence the respondent in providing information.
- A.
Multiple measurement
- B.
Valid measurement
- C.
Relevant measurement
- D.
Non-reactive measurement
Correct Answer
D. Non-reactive measurementExplanation
Non-reactive measurement is the procedure of gathering information without influencing or prompting the respondent. This means that the measurement does not cause the respondent to alter their responses in any way. It ensures that the data collected is unbiased and accurate, as the respondent's behavior or answers are not affected by the measurement process. Non-reactive measurement is important in research and surveys to obtain reliable and valid data.Rate this question:
-
- 17.
If different observers reporting on the same person report similar levels of a variable, we can say that the observation procedure is ______________.
- A.
Reliable
- B.
Reactive
- C.
Redundant
- D.
Rectified
Correct Answer
A. ReliableExplanation
If different observers reporting on the same person report similar levels of a variable, we can say that the observation procedure is reliable. This means that the procedure consistently produces accurate and consistent results, indicating that it can be trusted to provide reliable information.Rate this question:
-
- 18.
It is difficult to detect improvements among program participants if most of them are already in a desirable condition when the program begins. This problem is called a _______________.
- A.
Detection effect
- B.
Deterioation effect
- C.
Ceiling effect
- D.
Reactive effect
Correct Answer
C. Ceiling effectExplanation
The correct answer is "Ceiling effect." The ceiling effect refers to a situation where a measurement tool or assessment is unable to accurately detect improvements or changes in individuals who are already performing at a high level or have reached a desirable condition. This occurs because the measurement tool has reached its maximum limit or ceiling, making it difficult to differentiate further improvements.Rate this question:
-
- 19.
Which of the following is the best worded survey question to use in a teacher evaluation?
- A.
The teacher was not unorganized.
- B.
The teacher was organized and helpful to students.
- C.
The teacher was organized.
- D.
The teacher were organized.
Correct Answer
C. The teacher was organized.Explanation
The best worded survey question to use in a teacher evaluation is "The teacher was organized." This question is clear and concise, focusing on a specific attribute of the teacher. It does not include any negative or contradictory statements, making it easier for respondents to provide an accurate evaluation of the teacher's organizational skills.Rate this question:
-
- 20.
_______________ are not considered observers for the purpose of program evaluation.
- A.
Program participants
- B.
Evaluators
- C.
Significant Others
- D.
Experts
Correct Answer
A. Program participantsExplanation
Program participants are not considered observers for the purpose of program evaluation because they are directly involved in the program and may have biased perspectives. Their experiences and opinions are valuable for evaluating the program's impact on them, but they are not objective observers. Evaluators, significant others, and experts, on the other hand, can provide more impartial and objective observations and assessments of the program's effectiveness.Rate this question:
-
- 21.
Letters written by participants and graffiti on buildings are examples of __________ that may be a valuable source of information for a program evaluation.
- A.
Community indexes
- B.
Artifacts
- C.
Nonreactive measures
- D.
Reactive measures
Correct Answer
B. ArtifactsExplanation
Letters written by participants and graffiti on buildings are examples of artifacts that may be a valuable source of information for a program evaluation. Artifacts refer to physical or tangible items that are produced or left behind by individuals or groups. In the context of program evaluation, artifacts can provide insights into the experiences, perspectives, and behaviors of participants. They can offer a unique and authentic glimpse into the lived experiences and impact of a program, making them valuable sources of information for evaluators.Rate this question:
-
- 22.
The threat to internal validity called "maturation" refers to
- A.
Events happening in the community that will change the behavior of the program participants.
- B.
The improved skill of the program staff as they become more experienced.
- C.
Predictable changes in people that can be expected solely due to the passage of time.
- D.
The changing nature of an evolving program.
Correct Answer
C. Predictable changes in people that can be expected solely due to the passage of time.Explanation
The threat to internal validity called "maturation" refers to predictable changes in people that can be expected solely due to the passage of time. This means that as time goes on, individuals naturally grow and change, which can impact the results of a study or program. It is important to consider maturation as a potential confounding factor and account for it in order to accurately interpret the effects of the program or intervention being studied.Rate this question:
-
- 23.
The threat to internal validity called "history" refers to
- A.
Predictable changes in people that can be expected solely due to the passage of time.
- B.
Events happening in the community that will change the behavior of the program participants.
- C.
The improved skill of the program staff as they come to understand the history of the program.
- D.
Events that occur in the life of a program, such as a change in program direction.
Correct Answer
B. Events happening in the community that will change the behavior of the program participants.Explanation
The threat to internal validity called "history" refers to events happening in the community that will change the behavior of the program participants. This means that external factors or events occurring outside of the program can influence the behavior or outcomes of the participants, making it difficult to determine whether the program itself is responsible for any observed changes.Rate this question:
-
- 24.
Events that occur in the life of a program, such as a change in program direction.
- A.
History
- B.
Maturation
- C.
Regression to mean
- D.
Selection
Correct Answer
D. SelectionExplanation
The events that occur in the life of a program, such as a change in program direction, can be categorized as "selection". This term refers to the process of making a choice or decision based on certain criteria. In the context of a program, selection refers to the ability of the program to make decisions and choose different paths or directions based on certain conditions or inputs. Therefore, "selection" is the most appropriate answer to describe the events mentioned in the question.Rate this question:
-
- 25.
If an evaluator judges the rate of success of a program only on the basis of those who complete the program, with no consideration to who left the program, the evaluator is failing to deal with the threat to internal validity called
- A.
Selection
- B.
Attrition
- C.
Regression to mean
- D.
Maturation
Correct Answer
A. SelectionExplanation
The correct answer is selection. When an evaluator judges the success of a program solely based on those who complete it, without considering those who dropped out, they are failing to address the threat to internal validity called selection. This means that the evaluator is not taking into account the potential bias that can occur if there are systematic differences between those who complete the program and those who do not. By only focusing on the completers, the evaluator may not be getting an accurate representation of the program's effectiveness.Rate this question:
-
- 26.
The threat to internal validity called "testing"
- A.
Refers to errors made in calculating test scores
- B.
Refers to school-based program evaluations that rely on standardized tests.
- C.
Refers to changes in the meaning of the scores of the measures used
- D.
Refers to changes in behavior due to the observation techniques.
Correct Answer
D. Refers to changes in behavior due to the observation techniques.Explanation
The threat to internal validity called "testing" refers to changes in behavior due to the observation techniques. This means that when participants are aware that they are being observed or tested, they may alter their behavior in response. This can lead to inaccurate results and affect the validity of the study. It is important to minimize this threat by using appropriate observation techniques and ensuring that participants are not influenced by the testing process.Rate this question:
-
- 27.
The threat to internal validity called "instrumentation"
- A.
Refers to the people who are instrumental in creating changes in the program participants.
- B.
Refers to events that happen in a community that will change the behavior of program participants.
- C.
Refers to changes in the meaning of the scores of the measures used to collect data.
- D.
Refers to evaluations that use electroninc eqiupment to collect data.
Correct Answer
C. Refers to changes in the meaning of the scores of the measures used to collect data.Explanation
The threat to internal validity called "instrumentation" refers to changes in the meaning of the scores of the measures used to collect data. This means that over time, the way in which the measurements are taken or interpreted may change, leading to potential inaccuracies or inconsistencies in the data collected. This can introduce bias and affect the validity of the study's findings.Rate this question:
-
- 28.
Internal validity refers to whether
- A.
The findings of a program evaluation can be generalized to other similar group a statistically significant change occured.
- B.
A statistically significant change occured.
- C.
A research design provides sufficient control so that the evaluator knows if the program actually caused a significant change in a dependent variable.
- D.
A research design provides sufficient control so that the evaluator knows if the program actually caused a significant change in a dependent variable.
Correct Answer
B. A statistically significant change occured.Explanation
Internal validity refers to the extent to which a research study provides accurate and reliable results. It focuses on whether the observed changes or effects in the dependent variable can be attributed to the program being evaluated rather than other factors. Therefore, the correct answer is that internal validity refers to whether a research design provides sufficient control so that the evaluator knows if the program actually caused a statistically significant change in a dependent variable.Rate this question:
-
- 29.
An evaluation that uses a pre-test/post-test single group design cannot answer the question
- A.
"Did participants improve"?
- B.
Did participants reach criterion?"
- C.
"Did participants improve more that people who were not participating in the program?"
- D.
"Did participants improve more that people who were not participating in the program?"
Correct Answer
C. "Did participants improve more that people who were not participating in the program?"Explanation
A pre-test/post-test single group design can only evaluate whether participants improved or not, but it cannot compare their improvement to those who did not participate in the program. This design does not include a control group, which is necessary to make such a comparison. Therefore, it cannot answer the question of whether participants improved more than those who did not participate in the program.Rate this question:
-
- 30.
An evaluation that uses a single group design with post-test only collects data related to outcome variables
- A.
After participants have completed the program.
- B.
Before participants have entered the program.
- C.
Before participants enter the program and after participants have completed the program.
- D.
After participants have completed the program and at that time both the participants and comparison group members are tested.
Correct Answer
A. After participants have completed the program.Explanation
This evaluation design collects data related to outcome variables after participants have completed the program. This means that the data is collected at a single point in time, specifically after the program has ended. This design does not collect data before participants enter the program or during the program. Additionally, there is no mention of a comparison group in this design, so the data is only collected from the participants who completed the program.Rate this question:
-
- 31.
_________________ is a threat to internal valididy of the pretest/posttest design whenever people are chosen for a program because initial observations showed that they needed the program more than others did.
- A.
Repression to the mean
- B.
Attrition
- C.
Selection
- D.
Regression to the mean
Correct Answer
D. Regression to the meanExplanation
Regression to the mean is a threat to internal validity in the pretest/posttest design when individuals are selected for a program based on initial observations that they needed the program more than others. This occurs because extreme scores on the pretest are likely to be less extreme on the posttest due to random variation. Therefore, the observed improvement in scores may be falsely attributed to the program, when in reality it is just a natural tendency for extreme scores to regress toward the mean.Rate this question:
-
- 32.
A time series evaluation design
- A.
Requires repeated observations of the same group at many points in time.
- B.
Requires observations that take more time to conduct.
- C.
Requires at least three times as many groups to observe
- D.
Should only be used if there is not a lot of time to conduct the evaluation
Correct Answer
A. Requires repeated observations of the same group at many points in time.Explanation
A time series evaluation design requires repeated observations of the same group at many points in time. This means that the same group or sample is observed multiple times over a period of time, allowing for the analysis of trends and patterns over time. This design is useful for studying the effects of interventions or changes over time and can provide valuable insights into the dynamics of the variables being measured. It allows for the identification of temporal relationships and the assessment of the impact of time on the outcome of interest.Rate this question:
-
- 33.
The major weakness of the nonequivalent, control group design concerns
- A.
Instrumentation effects
- B.
Selection effects
- C.
Attrition effects
- D.
Testing effects
Correct Answer
B. Selection effectsExplanation
The major weakness of the nonequivalent, control group design concerns selection effects. This means that the groups being compared in the study may not be truly comparable or representative of the population, leading to biased results. Selection effects can occur when participants self-select into groups or when the researcher assigns participants to groups based on non-random criteria. This can introduce confounding variables and make it difficult to determine the true cause-and-effect relationship between the independent and dependent variables.Rate this question:
-
- 34.
An especialy strong quasi-experimental design combines the features of
- A.
Posttest only design with experimental design
- B.
Time series design with a non-equivalent control group design
- C.
Time series design with a posttest only single group design
- D.
Regression design with time series design.
Correct Answer
B. Time series design with a non-equivalent control group designExplanation
This answer is correct because a time series design involves collecting data over multiple time points to observe changes over time, while a non-equivalent control group design involves comparing a treatment group to a control group that is not equivalent. By combining these two designs, researchers can observe changes over time and compare them between a treatment group and a non-equivalent control group, allowing for a stronger quasi-experimental design.Rate this question:
-
- 35.
The ___________ design includes observations of additional dependent variables that would not be expected to be changed by the program of interest.
- A.
Pretest-posttest
- B.
Regression-discontinuity
- C.
Selective control
- D.
Control construct
Correct Answer
D. Control constructExplanation
A control construct design includes observations of additional dependent variables that would not be expected to be changed by the program of interest. This design allows researchers to compare the effects of the program while controlling for other factors that may influence the outcomes. It helps to establish a baseline and determine the true impact of the program by accounting for other variables that could potentially confound the results.Rate this question:
-
- 36.
The ___________ design includes a comparison group that was included in the evaluation to control for a specific threat to internal validity.
- A.
Selective control
- B.
Regression -discontinuity
- C.
Control construct
- D.
Pre-test postest
Correct Answer
A. Selective controlExplanation
Selective control design refers to a research design that includes a comparison group specifically to control for a particular threat to internal validity. This means that the comparison group is chosen strategically to ensure that any observed differences between the groups can be attributed to the independent variable being studied, rather than any other factors. By using selective control, researchers can minimize the potential influence of confounding variables and increase the internal validity of their study.Rate this question:
-
- 37.
Which of the following is a threat to internal validity for a quasi-experimental design but not a threat to the internal validity of an experimental design?
- A.
Maturation
- B.
Selection
- C.
History
- D.
All of these
Correct Answer
B. SelectionExplanation
Selection is a threat to internal validity for a quasi-experimental design but not for an experimental design. In a quasi-experimental design, participants are not randomly assigned to groups, which can introduce selection bias. This means that the groups may differ in important ways at the start of the study, making it difficult to determine if any observed effects are due to the treatment or the pre-existing differences between the groups. In an experimental design, random assignment helps to minimize selection bias and ensure that the groups are comparable at the start of the study. Therefore, selection is a threat to internal validity for a quasi-experimental design but not for an experimental design.Rate this question:
-
- Calculus Quizzes
- Data Handling Quizzes
- Decimal Quizzes
- Discrete Mathematics Quizzes
- Equation Quizzes
- Fraction Quizzes
- Function Quizzes
- Geometry Quizzes
- Graph Quizzes
- Measurement Quizzes
- Multiplication Quizzes
- Number Quizzes
- Percentage Quizzes
- Place Value Quizzes
- Probability Quizzes
- Problem Solving Quizzes
- Ratio Quizzes
- Shape Quizzes
- Statistics Quizzes
- Trigonometry Quizzes
Back to top