The job market may rise and fall with the economy, but the hunger for skilled software engineers and developers never stops. In your pursuit to fortify your business with aptitude and mastery, embracing technical assessments is paramount.
In this post, we’ll discuss a few key trends in technical assessment tools to consider if your organization is thinking about how to modernize and optimize your technical talent hiring process.
Non-traditional assessments are innovating the technical hiring process.
Talent assessments, including technical skill assessments, are a finely calibrated and sensitive tool, and therefore novel technology-driven assessment tools are taking time to build credibility and acceptance. In our research, we’ve come across two fairly novel types of technical assessment tools that we believe have earned the credibility for widespread use.
Automated Technical Interview Assessments
A relatively new type of assessment product for technical skills hiring is a fully (or mostly) automated hybrid of a coding test and coding interview, primarily designed to replace the pre-onsite technical interview part of a company’s hiring process. Byteboard, a technical interview platform that spun out of Google’s Area 120 in 2021, is perhaps the most notable in this category. The Byteboard assessment is a project-based take-home interview that evaluates candidates across 20+ software engineering skills as they work through a small time-boxed project that simulates real-life asynchronous work. Established technical assessment providers such as CodeSignal have also developed similar products. Currently, this type of assessment is getting the most traction in software companies such as Lyft and Adobe, but given the challenge that most organizations have with finding the hiring resources required for live technical interviews, we believe this type of assessment has growth potential.
At TTL, one of our common findings is that organizations using technical skills assessments as a candidate screening tool sometimes find that they aren’t getting enough “signals,” or communications representing potential job fitness, from these assessments to make an informed hiring decision. For example, candidates need to be able to show critical thinking: to know how to ask questions, know where the problem is heading, and how to think about the solution in order to perform strongly in a software engineering role.
To get a read on these types of signals, organizations will sometimes incorporate yet another step in the hiring process, whether it’s a behavioral assessment or an additional interview, which may increase quality of hire, but it also decreases candidate experience and overall hiring efficiency. With an interview-style assessment like Byteboard, these companies are able to combine the first two steps of the hiring process and still get a holistic understanding of the candidate’s capabilities before inviting them to complete an on-site interview.
Conversational Chat-based Assessments
A chat-based assessment is a text-based interview-style assessment that allows prospective employers to screen every applicant with a conversational AI-powered chatbot. This type of tool delivers a candidate experience that simulates a live interview, yet is fully automated in its delivery and evaluation.
Currently, AI chat-based assessments are most commonly used to measure psychometric and behavioral traits. A key example in the market is Sapia (formerly PredictiveHire), a conversational, Natural Language Processing (NLP) based chat AI interview delivered via an online messaging platform. However, chat-based assessments can also be used for hiring technical talent: the Singapore-based company Adaface has created a conversational, chat-based coding assessment as a way to screen candidates for programming skills using non-googleable MCQs and coding questions. The reasoning behind the use of this format is that traditional assessment tools focus on theoretical questions or puzzles. However, the questions are not always completely relevant to the fundamentals of the skills you are testing for, and therefore, do not elicit all of the signals required to confidently move a candidate on to the next stage.
Chat-based assessments are gaining acceptance in mostly high-volume hiring for customer-facing roles such as retail and call center workers, but we think this type of assessment has potential to add value in screening for technical roles due to its moderate length (~40 minute coding skills test) and its interactive and candidate-friendly experience.
Biometric Identity Verification Is Falling Out of Fashion.
In the past, we have seen a lot of interest in biometrics for test-taker identification, and to reduce candidate assessment fraud, but biometrics may be falling out of favor. One prominent concern is that facial recognition technology may be unfair to some demographics. Other forms of biometrics (e.g. voice recognition, retina) may have similar concerns and also may be restricted by various laws and regulations.
The sentiment we’re now hearing from both assessment vendors and practitioners is that some candidates will always be smart enough and motivated enough to find their way around identity verification measures. Using an assessment tool that is inherently difficult to “cheat” may be the better option going forward.
As an alternative, providers of non-traditional assessment tools such as Byteboard have invested a lot of effort into making their assessments harder to cheat. Since Byteboard invests so much time in designing “cheat-proof” interviews, they have decided to lessen their focus on security and identity verification (e.g. biometric scans, facial and voice recognition, etc.).
For example, many automated take-home technical assessments have predictable questions that candidates can use outside resources like ChatGPT with which to generate answers. According to Byteboard, their interviews are structured in a way that is much more difficult to generate a passable answer through outside means.For example, the candidate can’t easily copy paste a four-page design document and full code base into ChatGPT and have it solve the problem, because they are required to show their critical thinking skills along the way.
Conclusion
Overall, market adoption of innovative new assessment methods and the technologies used to administer them is slower than other areas of talent acquisition, as it should. For as long as pre-employment assessments have been around, it can take years for novel testing methods to achieve a high enough level of credibility and predictive validity – as it well should. These tests provide data that is factored directly into hiring selection, and therefore determine the future livelihood of millions of people. Sensitivity to regulatory compliance (AI, EEOC, etc.) plays a big hand in slower adoption of non-traditional automated assessment tools, especially those that leverage AI analysis.
We think interview-style assessments like Byteboard have proven their credibility, safety and validity as hiring tools, particularly for experienced senior-level candidates. Meanwhile, chat-based assessments like Adaface are more appropriate for entry-level high-volume roles, yet may offer a solution to the perennial issue of poor test-taking experience and high candidate drop-out rates.
Talent Tech Labs stands ready to advise on technological solutions tailored to meet your specific needs. Contact us today!