Big Data – the collection of larger than average datasets that require unconventional storage, processing, and analysis methods, has revolutionized nearly every field of business, from marketing to manufacturing. Big Data can provide those firms that develop the infrastructure to analyze and act on the patterns and insights contained in these datasets, with a source of competitive advantage in any industry.

This infrastructure includes the technology to aggregate, process, and analyze various datasets, and the personnel to perform these operations, which marketing research firm Gartner estimates will be a $232 billion dollar industry by 2016.

As more and more firms invest in Big Data infrastructure and integrate it into their existing internal operations, such personnel are in high demand these days. Firms often find them with the help of Big Data-driven recruiting procedures. Indeed, Big Data has transformed the world of recruiting; and it may help you find the talent you need, in each area of your business.

Big Data, or people analytics, as it is known when applied to recruiting, provides recruiters with more data to analyze. Social media networks have become the first stop for many recruiters after receipt of a resume. However, people analytics encompasses more than just social media data mining. Indeed, it encompasses even more than just back-end software or personnel. People analytics is also an orientation – an attempt to create a complete picture of a candidate long before they step foot in an office for an interview. An applicant’s entire online presence, their use of a firm’s recruiting database, their customer or non-customer status, their political affiliations, their smoking preferences, and other characteristics can all taken into consideration in this era of Big Data.

Best Uses of Big Data in Recruiting

© |

In this article, we will cover 1) the benefits of recruiting using Big Data; 2) recruiting using Big Data; 3) the risks of using Big Data for recruiting; 4) the future of hiring with Big Data; and 5) a case study of a business using Big Data.


The people analytics approach has tremendous advantages for recruiters. The proliferation of available information about candidates has made it possible for recruiters and human resources professionals to match an employee’s professional and personal fit with their firm more closely to the firm’s opening and corporate culture respectively. People analytics’ tools and techniques allow firms to develop a much more complete profile of a candidate – far beyond a one-page cover letter and accompanying resume.

People analytics allows firms to move away from hiring based on subjective factors that may have very little to do with an employee’s chances of success at that particular firm. The Big Data approach involves first determining what existing factors lead to employee success and retention, and hiring candidates who fall within those parameters. This approach makes it easier for recruiters and managers to justify new hires as well. And it works. Xerox recently used algorithm-driven recruiting techniques to reduce the attrition in its call centers by 20%.

Further, analyses of one’s internal HR database, its strategic sales plan, and its accounts receivable, can yield insights about where a firm needs to hire to stay on top of existing orders. This insight allows firms to recruit proactively, rather than when they face a talent shortfall. Hiring proactively allows firms to spend the time necessary to select the right candidate, and avoid paying a premium for talent in moments of extreme organizational need. It also allows firms to develop strategic recruitment plans that incorporate a firm’s broader hiring goals, such as building a diverse workforce.

People analytics can reduce your cost per hire, and your average time needed to fill open positions by making the recruiting process more efficient. Lastly, hiring using people analytics can align your compensation packages more closely with real market averages, by conducting analyses of publicly available salary information.


Big Data has given rise to a number of recruiting techniques designed to make recruiting efforts more precise and accurate. While these techniques predate the rise of Big Data, the explosion of available information has led to the development of algorithm-driven recruiting software solutions (as well as firms that specialize in algorithm-driven recruiting); and helped refine the tools and techniques used specifically for recruiting. These tools and techniques include data mining, keyword filtering, and testing.

Data mining

Data mining is a technique used by firms to aggregate data for a variety of different business purposes, including recruiting. Data mining can be used to analyze the internal data created by high-performing and/or longstanding candidates to search for insights into their performance and/or longevity. Data-driven firms like IBM, along with standalone data analysis firms like the California-based Cataphora, specialize in such statistical analyses, which can be used for internal recruiting and/or retention. By analyzing from where successful candidates have been hired can simplify the recruiting process as well. For example, a firm whose internal analyses have revealed that 49% of their top performers had their initial contact with a recruiter from Viadeo, may lead the firm to reduce advertising on LinkedIn, and instead ramp up recruitment efforts on the French social networking site.

Recruiters and human resources professional can also combine data mining with predictive analytics – the use of statistical methods and techniques to forecast the probability of a likelihood occurrence using historical data, to generate predictions about a candidate’s likely tenure with the firm should they be hired. These insights can also be used to provide parameters for the recruiting of external candidates.

Data mining, or as some recruiters call it “talent mining” can be done manually or automatically online. Individual recruiters and/or software can search online resume databases (internal or external), professional social network profiles, or other websites of interest for personnel who might be a match for an opening.

Social networks, in particular, capture significant information about an individual. Recruiters can determine not only whether a candidate might be a good fit for the culture of the firm, but also whether they might be successful there, by assessing this information against internal profiles of high performing candidates. For example, a firm’s highest performers may spend a small amount of time on a single social network. A candidate who spends considerable time on multiple social networks might raise some flags. Alternatively, a social network might indicate that the candidate is engaged in activities that might impair their productivity, such as excessive drinking or high-risk hobbies, such as extreme sports. These insights can be helpful to the diligent recruiter.

Keyword filtering

Using desired skills and other characteristics as keywords, recruiters can run searches in popular search engines, on professional and non-professional search engines, in public or private online communities, and on other online properties. This can yield promising leads, who recruiters can contact for an informational or formal interview.

Keyword filtering is also helpful when screening out applicants who have applied for a position through a web-based talent management application (either proprietary or from a third-party recruiter). Recruiting software automatically scans submitted resumes and cover letters for specific keywords, rejecting those without them, and returning to recruiters only the candidates who fit the job description on paper.


More and more, testing is used in the hiring process. Usually, pre-screened applicants are invited to take a skills test, a personality test, or both. Skills tests are used to authenticate the skills listed in one’s job application, but also can be used to test those not listed, such as soft skills. Personality tests are used to assess a candidate’s fit with the firm’s culture, as well as soft skills. Personality tests have been around for a long time, but the combination of computer-assisted testing, and data-driven approached to psychology, make these tests much more sophisticated and precise.

Increasingly, both skills and personality tests are assessed against internal analyses of high performing employees. For example, an advertising firm may find success with candidates who work well in a team and possess a high degree of digital fluency, regardless of the job opening. They may in turn offer measure all candidates for an opening against skills and personality tests they mandate during the hiring process.

It is not uncommon for candidates for senior positions in all industries (and even some junior level positions in industries such as finance) to be given one or multiple, skills tests, and a personality test, during multiple interview rounds. These tests provide hiring managers with more data points, alongside the job application, the interview(s), online data, and other publicly available information, against which to measure candidates.


Big Data may yield tremendous potential for recruiting, and indeed, for many firms, some big results. However, there are risks to using algorithm-driven recruiting tactics that can lead to some substantial consequences. These risks stem from overreliance on algorithm-driven recruiting. People analytics should, ideally, supplant human recruiting efforts, rather than supplant them.

Incorrect forecasting assumptions

Using profiles developed by analyses of internal data can yield promising candidates. Nevertheless, this assumes that the analyses provide accurate insights into what it takes to perform at the firm. The best analyses have a degree of uncertainty, and employee performance standards constantly change along with the demands of the markets. Historical performance is no guarantee of future performance. Further, Big Data is notoriously messy. It is critical that you invest in either a data-driven recruiting firm or the personnel to analyze recruitment-relevant data. Your HR people, even the best intentioned, may not be able to properly analyze multiple datasets to generate actionable recruitment insights. You will need statisticians.

It is also very important to understand that the models predicting candidate success are based on how strongly a particular candidate’s characteristics are correlated with the characteristics of a hypothetical high performer employed by the firm. Correlation does not imply causality, which means, in this case, that just because a candidate is identical to the hypothetical performer on all levels those data do not mean that he or she will definitely succeed. If you approach people analytics expecting to find set parameters for candidates that will always result in success, you are likely to be disappointed.

Moreover, assessing candidates individually, pre-employment (and even post-hire) rarely yields insights into what said individuals can contribute to a team. For example, designing an algorithm that predicts whether an individual whose performance may be average but who may provide rousing pep talks to team members outside of work, is challenging at best.

It should be noted that privacy is also a big concern, particularly when it comes to internal data gathering. Employees, using resources belonging to an employer, should expect a certain amount of data gathering. But how much is too much? At what point does data-gathering feel invasive and undermine productivity – the very thing the firm seeks to measure?

Human beings design the recruitment algorithms so, in addition to the possibility of human error in the design and/or implementation, of the algorithms, there is the likelihood of bias, especially when it comes to less objective measures of success. People tend to associate with people like themselves. Using algorithms that use historical data to predict employee performance as hiring parameters may yield candidates who are similar to existing candidates, but neglect those who are different enough to innovate. As innovation is a key source of not just competitive advantage, but ability to adapt to market dynamics, over time, a homogenous staff may hurt your firm.

Violations of equal opportunity laws

Homogenous employees may hurt your firm in ways other than reduced innovation. Recruiters must make sure that their hiring algorithms do not systematically exclude classes of protected employees. The fact that an algorithm repeatedly returned a single ethnic group or gender is not an excuse in the eyes of the U.S. Equal Employment Opportunity Commission, and other foreign counterparts. Recruiters must take pains to ensure that the algorithm takes a country’s diversity laws, both through automatic and manual review of the candidates returned by software at every stage of the interview process. Failure to do so can be time-consuming and costly in the case of either resulting civil litigation, or damage to your firm’s brand.

Even entire industries are not immune from the need to vary their recruiting approaches. In August of 2014, a flurry of reports highlighted the lack of diversity in Silicon Valley. This was at least partially attributed to algorithm-driven recruiting practices.


Despite the risks, algorithm-driven approaches to recruiting are here to stay. Big Data has accelerated the rate of advancement in machine learning. Machine learning is the design and study of learning algorithms that, essentially, help a computer process and understand data better. As the field of machine learning advances, so too will the sophistication and accuracy of algorithm-based recruitment tools.

Moreover, the cost of hiring the wrong candidate is high. Training, salary and benefits, and search related costs just scratch the surface if the new hire’s mistakes cost revenue. The potential benefit of algorithm-driven recruitment methods, in the estimation of most firms, outweighs the growing pains associated with new approaches. Moreover, algorithm-driven approaches already have worked well for a number of firms.



© Wikimedia commons | Google

Case in point: Google. It is expected that a firm as data-driven as Google would be a pioneer in data-driven approaches to recruiting. Google has dedicated resources to building a hiring algorithm, which predicts a candidate’s probability for success if hired. They also developed a separate algorithm designed to backstop its initial screening of candidate resumes, which indicated that its primary algorithm had missed 1.5% percent of the time.

Their dedication to research-driven approach also informed their application process. After much research, they determined that four interviews provided the maximum amount of insight. They make considerable use of behavioral interviews. Further, they employed group hiring to reduce bias dramatically in hiring decisions.

Google has also developed retention algorithm, using predictive modeling methods that predict the probability of success of employees, post-hire, and applying them to the dynamic data that is reflective of a growing and changing workforce. This has informed their hiring practices by allowing Google to refine the parameters of their hiring algorithm. They have also used analytics to increase the hiring of underrepresented groups, such as women.

As a result, Google has been able to fill vacancies rapidly, and enjoyed both low turnover and a reputation for being very selective, enhancing its prestige as a top employer brand for years.

Big Data for Recruitment | SourceIn London 2013

Comments are closed.