Uncovering Actionable Intelligence Behind Big Data

AAFCPAs is excited to present NTI’s remarkable journey as a prime example of an organization leveraging data to solve real world challenges. NTI is an organization dedicated to supporting individuals facing physical or mental health challenges in securing gainful employment. Despite a sophisticated AI-driven platform, they were confronted with a challenge: many potential candidates were dropping out of the hiring process. To solve this, NTI partnered with AAFCPAs’ Data Analytics practice.

The primary objective of this partnership was to leverage data analytics to discern the reasons behind candidate attrition and devise strategies to improve retention rates. This success story delves into the innovative application of data analytics in enhancing NTI’s candidate placement process.

About NTI@Home

NTI@Home is a job placement program led by National Telecommuting Institute (NTI), a 501(c)(3) nonprofit organization. Founded in 1995, NTI places individuals with disabilities in home-based employment positions. This talented workforce acts as the backbone to commercial and government call centers, where sales, customer service, and technical support agents assist on a remote basis.

The organization has played a pivotal role in the Social Security Administration’s Ticket to Work program since 2004 and has trained and placed more than 10,000 highly motivated candidates to date. Clients have included recognized brands such as Amazon, the Internal Revenue Service, Gerber Life, and John Hancock.

Obstacles and Challenges

NTI’s robust candidate placement process began with outreach to candidates with physical disabilities or mental health impairments that often act as a barrier to employment. Reliant on a hiring platform powered by AI automation, the organization then allowed candidates to advance autonomously through each stage toward employment.

Once they completed screening, they were placed into various tiers dependent on experience, qualifications, and capabilities. Candidates were next assigned a coach, who could help with job training as needed. In the end, candidates entered the applicant pool, where they could apply for jobs, conduct interviews, and get hired.

NTI tracked status along its six-point process:

NTI tracked actions and knew when applicants abandoned the process prematurely; however, they did not know the reason for abandonment.

“I saw that we had literally a couple thousand people registering a month,” explained Alan Hubbard, Chief Operating Officer for NTI@Home. “But we only saw a small talent pool we could use to fill job requisites.” NTI first needed to understand their rationale so they could increase placements and broaden the program. “I spent several months just trying to figure that out,” he said. “It was tough to get to the root cause of the problem. Did we need less information? More information?”

The Solution

NTI engaged AAFCPAs’ Data Analytics practice to see if they could use their data in a way that might pinpoint the root cause of abandonment and help them achieve their goals. We first worked to define the Key Business Question (KBQ); specifically, how can NTI place more candidates in jobs and better advance its mission? We then defined Key Performance Indicators (KPIs) that would help us answer the KBQ. For example, a KPI might be a coach vs. candidate ratio, while a KBQ might be: Is there a particular coach that is overloaded?

AAFCPAs started from scratch, building a map of NTI’s current process using Microsoft Visio (snapshot captured in the following image). This was an important first step, helping us better understand NTI’s candidate recruitment process and making a direct relationship between the data points collected and the observed candidate behavior. It also helped NTI better understand its own process and how that relates to our data analysis. Identified in this map were the various systems NTI used during candidate recruitment. This visualization offered transparency into each step a candidate took prior to exiting the process.

“Step one was to understand the process,” explained Vassilis Kontoglis, AAFCPAs’ Director, IT Security & Data Analytics. “This is a critical part of our solution because we need to understand how it happens, what happens, and when it happens. Then, as we look at the data, we can begin to understand what that data means at any given point in the journey.”

AAFCPAs used Microsoft Power BI, a leading data analysis and visualization platform, for data analysis. This started with a data modeling exercise where we imported data from NTI’s multiple systems, conducting data mapping and allowing for the continual import of new data into the same Power BI model for ongoing consideration. Next, we built visualizations using Power BI to support the defined KPIs and translate NTI’s data into meaningful business behaviors, such as the number of candidates placed annually.

As we sought clues at each drop-off point, it grew evident that NTI had lost a large segment of candidates between outreach and registration. We also identified significant drop-offs after the initial contact was made as the AI system took candidates to a questionnaire to see whether or not they matched. Here we formed a hypothesis—based on our experience working with clients, our understanding of their operations, and our ability to look at the process visually—that an AI-powered automated system might be particularly challenging and thus discouraging. This hypothesis rung true, which led to further investigation into factors like timeframes. Lengthy processes might lose a candidate’s interest.

As our analysis progressed, more metrics around candidate abandonment surfaced. Specifically, candidates not central to its core mission were more likely to use NTI’s services, advance in the process, apply for postings, and earn placement. Meanwhile, those more central to its mission were abandoning the process in greater volume. In other words, NTI was losing its most qualified candidates along the way.

While NTI knew candidate abandonment was an issue, they now learned that their most relevant candidates were far more likely to drop out prior to job application than were their one in four counterparts.

This data visualization assisted well in NTI’s forecasting. For instance, NTI might have previously set a goal to increase placements year over year. But it had been tracking that progress manually in table format, which never delivered real-time actionable insight to, say, establish whether a placement of seven people in May was a concern or if June numbers were a windfall.

Now NTI’s historical data is pulled into a meaningful visual forecast, providing a tool to plan expected placements throughout the year. This gives leadership a concrete projection to keep everyone on track.

With easy-to-read charts and reports now automatically populated with incoming data, NTI saves hours of manual labor and gains better business insight they can use to assist in their strategic vision. They also freed time to focus on more important things like candidate communication, client partners, and core business competencies. “We’re now able to step back, take a better look at the data, and continually improve the candidate’s experience over time,” Alan explained.

The Result

Results and key findings were presented to NTI’s executive team who, in turn, incorporated feedback into executive planning processes. They also performed corrective actions. Most notably, they chose to restructure the organization based on findings.

Restructure meant uniting a once segmented team into one seamless and coherent engine. So as opposed to working with candidates across six silos, each of which supporting one aspect of the six-point process, candidates could now work collaboratively and concurrently with staff from contact through placement. For example, job coaches now engage with candidates immediately upon registration and into application, interviewing, and placement. By working as one informed and collective team, they are now able to address and eliminate some of those attrition points candidates faced along the way.

“We were amazed, to be honest, that the team picked up so quickly,” Alan said of the experience. “Once we explained what was going on, AAFCPAs came back and presented what they saw and were spot on. It was good to see that the team understood the issues so quickly and were able to point to the places where candidates had been dropping off. It also highlighted the fact that, internally, our communication was not as good as it could be.”

In addition to their restructuring, leadership decided to focus on updating their communication strategy. Taking a step back, they asked themselves: What do we tell candidates? How do we reach out? And how can we better meet candidates where they are? While NTI has traditionally focused on phone communication, they are now exploring new avenues, such as text messaging along with other tools and channels today’s job seekers favor most.

As the organization takes a step back, they’re also looking to see how they might tailor a training plan to not only ready candidates with the right skills for the job but also teach in the format that best suits each person’s learning preference. This means taking a closer look at how a candidate learns, how long they’ve been out of the workforce, and the exact training they need to land their ideal role.

“We want to give candidates the skills they need to be successful in the workplace,” he said. In turn, NTI is better equipped to provide employer partners with candidates who are ready from day one.

As analysis continues, as more data is collected, and as hiring cycles are introduced, the team is excited to have the tools they need to spot and resolve bottlenecks faster and to best align their objectives with KPIs.

“People think they know what the process is,” Vassilis said. “But until you put it down on paper and identify all the stakeholders involved, they can’t really. It’s only once you’ve mapped it out that realization comes. Equally important is having a means to translate your data into scenarios and relationships along that journey. That’s when you may begin to understand the story behind it.”

In the end, the process was an overwhelming success.

“The final cherry on the top is that we blew our goals away in 2022,” Alan said. “The reorganization, the changes we made in our recruiting, and increased visibility into where each candidate was throughout the process made a real difference.”

Alan Hubbard, Chief Operating Officer | NTI@Home

Contact AAFCPAs

Do You Want to Improve Your Data Usage?