Grantee Resources for Data Collection and Management
This webpage provides a series of eight workshops designed to provide an overview of the basic principles and methodologies for program monitoring and evaluation with emphasis on practical applications for researchers and practitioners. These resources were developed through a contract with RDA Consulting to provide technical assistance to grantees of the California Violence Intervention and Prevention Grant Program (CalVIP). However, the materials offered are highly beneficial for all grantees.
The primary audience for these workshops is individuals who play a role in quality assurance and/or the evaluation of grant programs. Other individuals interested in data and program evaluation might also benefit from this workshop series. The workshop series will be particularly helpful to individuals responsible for:
- quality assurance practices.
- designing or carrying out evaluations of program implementation and outcomes.
- collecting and analyzing data about program implementation and outcomes.
- writing reports or disseminating information about program implementation and outcomes.
To attend future workshops please see the CalVIP webpage.
Workshop #1: Best Practices in Data Entry, Collection, and Administration – Part I
September 2023
This workshop will provide an overview of data quality considerations, with a focus on aligning data to grantees’ evaluation questions. This is part 1 of a two-part workshop. By the end of this workshop, participants will be able to:
- Learn the difference between process and outcome evaluation questions.
- Identify the two major types of data.
- Understand strategies to evaluate the quality of their data.
- Consider different data collection strategies to address their evaluation questions.
Workshop #2: Best Practices in Data Entry, Collection, and Administration – Part 2
October 2023
This workshop focuses on best practices in developing data collection instruments and describes how to create quality instruments to meet data collection needs. This workshop builds off Workshop #1: Best Practices in Data Entry, Collection, and Administration – Part 1. By the end of this workshop, participants will be able to:
- Consider several questions to select the best data collection method(s) for their program.
- Understand the purpose of different data collection instruments (e.g., existing records, interviews and focus groups, observations, and surveys) and best practices for using them.
- Describe the advantages and disadvantages of common data collection methods.
Worksheets:
Workshop #3: Measuring Client Progress through Outcome Data
December 2023
This workshop emphasizes the relationship between your program’s mission and data collection processes. By highlighting the link between client outcomes and program outputs, the workshop looks to instill further confidence in grantees’ capacity to measure program efficacy. By the end of this workshop, participants will be able to:
- Incorporate the notion of baseline data collection to motivate program evaluation.
- Understand the benefits of pre-/post-tests, retrospective tests, and post-tests only in alignment with program goals and resources.
- Understand the difference between outcomes and outputs.
- Consider appropriate timing and be able to weigh the tradeoffs between measuring short-, medium-, and long-term outcomes.
Worksheets:
Workshop #4: Streamlining and Managing Data Collection Processes – Part 1
January 2024
This workshop shares information on how to improve data reliability throughout the data collection pipeline. Part 1 will focus on data accuracy, particularly at the point of entry. Together we will review validation techniques and document standardization guidelines, processes for managing qualitative data collection, and recommendations for data codebooks and storage conventions. By the end of this workshop, participants will:
- Have a clear set of actionable quantitative data validation and document standardization recommendations.
- Collect and manage qualitative data so that they are usable for their program evaluation.
- Be able to draft essential documentation for data entry and retrieval.
Worksheets:
Workshop #5: Streamlining and Managing Data Collection Processes – Part 2
February 28, 2024, 11:30 AM to 1:30 PM
This workshop continues the discussion on how to improve data accuracy and reliability throughout the data collection and management pipeline. Part 2 will focus on steps for reviewing data for accuracy and quality after it has been collected. By the end of this workshop, participants will:
- Understand measures of data quality and recommended quality assurance steps.
- Have practiced conducting quality assurance processes and discussing potential causes of inaccurate data.
- Have reviewed recommended practices for data retention and destruction
Webinar
Presentation
Worksheets:
Workshop #6: Monitoring Data Collection and Establishing Continuous Quality Improvement Processes
April 3, 2024, 10:00 AM to 12:00 PM
This workshop builds on data collection monitoring to program data usage. Specifically, we emphasize continuous quality improvement of program implementation as an interrelated component of data collection. By the end of this workshop, participants will be able to:
- Identify data quality monitoring processes that suits your organization’s needs and capacity, including roles and responsibilities for data quality monitoring.
- Consider approaches to staff training and building a data culture.
- Use data to inform continuous quality improvement efforts and practice Plan, Do, Study, Act (PDSA) cycles.
Worksheets:
Workshop #7: Using Secondary Data to Support BSCC – Part 1
April 24, 2024, 11:30 AM to 1:30 PM
This workshop discusses how to leverage publicly available data to support program evaluation efforts. Together we will discuss what qualities to look for in secondary data sources as well as how to search for data and conduct cursory literature reviews. We will also review publicly available datasets that could be relevant to CalVIP grantees. At the end of this workshop, participants will be able to:
- Build a repository of work done by other researchers or programs that informs program evaluation efforts.
- Review and retrieve publicly available datasets that could be relevant for program implementation and/or evaluation.
Workshop #8: Using Secondary Data to Support BSCC – Part 2
May 23, 2024, 11:00 AM to 1:00 PM
This workshop discusses data-sharing processes after data needs have been identified, including outreach to partner organizations, making data requests, and establishing data-sharing agreements. By reviewing the desirable characteristics from the publicly available datasets discussed in Part 1, participants will identify potential organizations or institutions in which collaboration could be useful for program evaluation. By the end of this workshop, participants will be able to:
- Address broad considerations related to data sharing, including legality, consent, and foundational agreements/documents that govern data-sharing practices.
- Draft communications related to data sharing and collaboration, including specific data requests and data sharing agreements.