Family First Prevention Services Act Grantee FAQ

The evaluation includes survey measures of child well-being, parenting skills, caregiver mental health, and caregiver substance use. Each agency will only collect the measures that align with the goals of the Family First program they are delivering. Each measure will be collected twice; at the beginning and the end of the family’s services in the Family First program.

Family First Prevention Services Act Grantee Forms

No. Families who started services before the beginning of survey data collection at your agency will not be asked to complete data collection. This includes people who are still being served at your agency. Only new enrollees from the launch of data collection on will be asked to complete the data collection form.

No. The KU Research Team is managing the licensing and permissions for all data collection instruments.

The launch of survey data collection began in September 2020, post human subjects research approval from the University of Kansas Institutional Review Board (IRB). With the approval being received and your agency’s individual data system being ready, data collection can begin at your site.

No. Though you may provide services to whole families with multiple children and multiple caregivers, you will only collect survey data from one primary caregiver about themselves and one target child. The primary caregiver is defined as the person who identifies themselves as providing the primary day-to-day care of the child. If multiple caregivers share care responsibilities, the primary caregiver for data collection purposes is the person who will be available to complete the surveys at both timepoints. The target child is defined as the child named on the referral who falls within the age range of the Family First service your agency is providing. If more than one child in a family meet these criteria, select the child whose first name begins with the letter highest in the alphabet. For example, if brothers Aaron and Bob are both named in the referral and fall within the service eligible age-range, Aaron should be selected as the target child and child well-being surveys should reference him.

Survey measures will be administered at two timepoints; at the beginning of Family First services (Time 1) and at the end of Family First services (Time 2), with the exception noted in #7 below.

• At Time 1, survey measures should be completed with the caregiver within the two weeks (10 business days) before or after the start of services.
• At Time 2, survey measures should be completed with the caregiver upon the conclusion of services, and should be collected in 2 weeks (10 business days) following the completion of services. For Family First services lasting longer than 12 months, the Time 2 measures should be completed once the family reaches 12 months of services, even if services continue.

The Demographics Form (PDF) requests the number of adults in the home and the number of children (including target child) in the home. This information will be used to calculate family size and proportion of families living above and below poverty levels. We recognize that family and household makeup may take many forms. To answer these questions, please coach the family to answer with the number of caregivers and children, either related or not related, who live in the household for the majority of their time. This may include step-siblings or other children who spend time in multiple households, or other adult caregivers.

Our goal is to collect as much information as possible to assess the impact of all programs. To achieve this goal, we will adapt data collection procedures when possible to accommodate prenatal service delivery. First, we will ask programs serving families prenatally to collect the Time 1 Parenting Sense of Competence scale (PSOC) as a retrospective pre-test. This means that both the Time 1 and Time 2 parenting survey measures will be administered at the end of Family First services. You will begin with the Time 1 survey measure, asking the parent to reflect back on how they felt before services started. Then you will complete the Time 2 PSOC survey as usual. Unfortunately, this procedure does not apply to the Ages and Stages Questionnaire: Social-Emotional –2 (ASQ:SE-2) as social-emotional developmental cannot be measured prenatally. If services begin prior to the child’s birth, the ASQ:SE-2 will only be administered at the end of Family First services (Time 2).

Parenting Sense of Competence scale (PSOC)Forms

Although the ASQ:SE guidance dictates that use of ASQ:SE should start after 4 weeks, please go ahead and collect the data on the child at the latest possible opportunity within the data collection window, even if it is prior to 4 weeks of age.

If a family withdraws or discontinues Family First services, Time 2 survey measures do not need to be administered for the evaluation. The end of services will be reflected in the administrative data received from DCF. If a family wants to discontinue participating in the evaluation but wishes to continue Family First services, Time 2 survey measures do not need to be administered for the evaluation. You will indicate the family’s withdrawal from the evaluation in the monthly case tracking reporting.

It is important to keep completed survey measures (or copies of survey measures) for a brief period in case we need to verify any of the data. For purposes of the study, paper or PDF copies of completed survey measures should be retained for at least six months.

Our research procedures are flexible and allow for data collection to take place remotely as all surveys may be administered orally. How you deliver services and administer survey measures will depend on your agency’s policies and guidelines and your method of routine service provision.

Data collection should be integrated into the flow of your enrollment and service provision. The KU Research Team will make survey forms available to you on the FFPSA Evaluation website. You will collect data from families within the flow of your services, while adhering to the guidance related to timing of survey administration.

Family First Prevention Services Act Grantee Forms

All Family First service providers at agencies using DAISEY for data management will have login access to DAISEY to enter their data. Individual agencies may decide whether staff enter their own measures or whether data entry is centralized within the staff. For monitoring and supervision, DAISEY allows agencies to view and export their organization’s data submitted in DAISEY.

DAISEY Meetings 2020

The KU Research Team will regularly make data summaries and data visualizations available to you/your agency to monitor the impact of your program and Family First overall. The team will also report on aggregate progress during all future grantee meetings.

Grantee Meetings 2020

It is important for the evaluation that the same adult completes the surveys for Time 1 and Time 2. If the primary caregiver for the child changes during services (without removal from the home), we will need the new primary caregiver to complete all the Time 1 and Time 2 surveys at the end of services. For this case, use the specific forms for Time 1 designated for a change of caregiver, located on the FFPSA website. These forms are identified as the Change in Caregiver forms. The Time 1 Change in Caregiver forms should be administered to the new caregiver at Time 2 as retrospective surveys, reflecting back to how they felt before services began. You will immediately follow this by administering the usual Time 2 surveys to the new caregiver, based on their current thoughts, feelings, and functioning. We suggest reviewing the sample script for recommendations on introducing the new caregiver to this adapted data collection process. 

Although the question content in the change of caregiver forms remains the same as the standard Time 1 forms, these forms have different instructions and wording to ask the new caregiver to answer questions about the past. The change of caregiver forms direct the new caregiver to think back to how they felt or what they observed at the time services began. The new primary caregiver’s Time 1 information (including the demographic survey) should be entered into DAISEY/your database system and designated as the new primary caregiver for the child. For instructions on how to enter a new primary caregiver into DAISEY, see the Change of Caregiver Instructional video on the FFPSA website.  

Changing the Primary Caregiver Forms

Changing the Caregiver in DAISEY Tutorial

17.1. Why is the monthly tracking report important?

The monthly tracking report is an important tool for the evaluation team to monitor implementation of Family First across many sites. This essential tool provides routine feedback about Family First staffing, referrals, uptake, evaluation participation, and model fidelity. Routine completion of this survey helps improve consistency in data collection, provides required outcome information for Federal reporting, and also helps the Family First Data evaluation team validate the data received from multiple sources.

17.2. When is the deadline? If I skipped one monthly tracking survey, what should I do?

The monthly tracking survey will be distributed on the first day of each month, beginning on November 1, 2020. The deadline for submitting this survey is the 10th of each month. However, if you miss the deadline, please do not hesitate to finish your submission to the system ASAP. Please submit the missing report, and then proceed to the current monthly report.

17.3. Where can I find the Monthly Tracking Survey?

The monthly tracking survey will be sent via email through a secure, online survey platform called REDCap. This link will be sent only to the primary contact for each grantee program. If you did not receive the monthly report, please first check your ‘Junk Email’ box, if you still cannot find it, please contact the Family First Evaluation Team ASAP to receive a new link.

17.4. How can I check my previously submitted surveys?

Upon completion of each survey, the REDCap survey will provide an option to download a PDF copy of your responses for your records. Additionally, you may contact the KU evaluation team at any time to receive a site-specific report for any given month. Finally, the evaluation team will regularly provide summary reports across sites for your information.

17.5. How do I know whether I have submitted the survey successfully?

After submitting the report, you will be redirected to a confirmation page thanking you for your participation.

17.6. Who should fill the Monthly Tracking Survey?

The survey is sent only to the staff member designated as the primary contact for the agency and they are responsible for completing the survey. However, if there is somebody who is better suited for this job or the primary contact changes, please update that person’s information with the evaluation team by notifying Kaela Byers (kaela@ku.edu)

17.7. Monthly Tracking Survey-Content Definitions

17.7.1. How many new staff members were hired to deliver Family First services?

“New staff” refers to the new employee(s) newly hired in your agency during the reporting period, including full-time staff, part-time staff, and volunteers who are responsible for delivering the Family First service at your agency.

17.7.2. How many staff members received training related to Family First services?

This refers to training provided during the reporting period to both new and existing staff related to delivery of Family First services. For example, if you are an Multisystemic Therapy (MST) provider, it refers to the number of staff who received any training on MST services during the reportin period to support Family First program delivery.

17.7.3. How many Family First referrals did you receive from DCF?

The number of new referrals received from DCF during the reporting period, regardless of whether they ultimately received services.

17.7.4. How many Family First referrals did you accept?

The number of DCF referrals accepted by your agency during the reporting period for outreach and enrollment, regardless of when the referral was received.

17.7.5. How many Family First referrals did you reject this month only due to a lack of capacity to accept and serve them?

The purpose of this question is to assess overall capacity across the state. Report any referrals rejected during the month due to issues with agency/program/staff capacity, regardless of when the referral was received.  Lack of capacity may be for any reason such as low staffing, staff already working at capacity, staff in the family's area already working at capacity, etc.

17.7.6. How many Family First referrals did you reject this month only because your program or service would not be able to meet the needs of the family (e.g., child age ranges do not align with the program model, family resides outside of the service area, the service is not the right fit for the family's needs, another service would be more appropriate for the family, etc.)?

The purpose of this question is to assess the overal quality of referrals by capturing referrals rejected because they do not align with the agency's program or service.  Report any referrals rejected during the month because they were not a fit for your program/service, regardless of when the referral was received.  Do not include referrals rejected/retracted for other reasons such the family declined services or lack of capacity.

17.7.7. Of the referrals you accepted, how many have declined services?

The number of accepted referrals who declined to participate in your agency's Family First service during the reporting period, regardless of when the referral was received or accepted.  This includes families/referrals who previously accepted - and perhaps even engaged in - services but declined services during the reporting period.  This includes retracted referrals if the retraction is made because the family declined services.

17.7.8. Of the referrals you accepted, how many declined to complete the evaluation surveys?

The number of families who accepted enrollment in your agency's Family First service who declined to complete the evaluation surveys during the reporting period, regardless of when the referral was received. 

17.7.9. At the current time, how many staff delivering Family First services have met the fidelity requirements of the model your agency is delivering?

In order to assess compliance to fidelity of your model, we are relying on your existing, internal practices rather than introducing new monitoring procedures.  Please report the total number of staff who are meeting the fidelity requirements of your Family First program model according to the model developers and your agency's internal procedures during the reporting period, regardless of when the referral was received.

17.7.10. At the current time, how many staff delivering Family First services are not meeting fidelity requirements of the model your agency is delivering?

In order to assess compliance to fidelity of your model, we are relying on your existing, internal practices rather than introducing new monitoring procedures.  Please report the total number of staff who are not meeting the fidelity requirements of your Family First program model according to the model developers and your agency's internal procedures during the reporting period.