Do any of you use work sample testing in order to determine the top candidates to interview? I'm interested in collaborating with experts to develop work-sample tests for use in corporate finance hiring, and am looking for insights around current use of these types of tests. Our approach is to focus on higher-order skills (application of knowledge, analysis, evaluation, creation) and I'd be interested in hearing which topics you find are most significant/useful for initial screening tests. If you are also of the strong opinion that work-sample testing is not capable of providing anything of value to your hiring process, I'd be interested in hearing your thoughts as well.
Pre-interview screening tests based on samples of actual work?
Answers
You talk with a candidate. That person makes an impression. Then you give a test "create something".
How long are you giving them? Are they creating a spreadsheet. What type of sheet. Are you asking them to create something they've done a thousand times or something new?
Seems sort of foolish to me. So I create they great spreadsheet. I'm a genius on spreadsheets. But then you put me into a meeting where not only do I have to explain my work, but defend it. I fail pitifully.
There are different levels of application of knowledge. Academic tests don't test the application of knowledge that is important, creativity, ability to learn, the ability to ask the right questions and more importantly the ability to know when you need to ask those questions.
1
Thanks for your feedback, Wayne. I very much agree that it is the higher levels of application of knowledge that are important to assess, rather than simple knowledge assessments. For instance, if you're familiar with Bloom's taxonomy, the ability to defend one's work would fall under "Evaluation", which is classified as a higher order thinking skill. Interestingly, studies show that work-sample testing is better than any single other type of assessment, including interviews, although using a combined approach is also shown to be better than any single assessment. Our aim is to provide companies with meaningful work sample assessments they can use in combination with other types of hiring assessments.
How do you currently go about assessing whether a potential candidate will be able to explain and defend their genius spreadsheet in a meeting?
Good Grief,
Shannon Glass is selling Testing Services -- her question was not even a legitimate quandary - it is schill to access user names for solicitation. Go to www.testdome.com to confirm. Flagged as Offensive/Promotional!
C'mon Proformative --
3
I am absolutely not here to try to sell testing services, nor to sell anything at all. Nor do we currently offer any tests that would be of interest to this group. As I said in my post, I'm looking to collaborate with experts to develop the use of work-sample testing in corporate finance hiring. Hence, my request for insights on this topic.
Oh really, you are not selling testing services?
This is directly from your company's URL:
"Automated testing of programming skills"
"Ask candidates to write real code, before calling them for an interview."
"Practice interview questions "and get certified for free."
"Work Sample Testing -To test if a candidate will be good at work, we give them a sample of the actual work. Science has proved that this has a better correlation with work performance than indirect measures such as IQ, personality or education level."
"Superior Screening - Use our online tests and the platform to detect A+ programmers. Our tests can measure code correctness, memory consumption and execution speed automatically. And they are customized for every language and technology."
Please tell us that you did not solicit Jim Rogers --
Mark, my company sells programming tests for companies in need of such things. I do not sell testing services or any other services or goods for my company or any other company. I reached out here, both in general and to specific people, to discuss the use of work-sample tests in hiring for finance jobs precisely because this is an active community for people in that field.
It's all right there in my original question: "I'm interested in collaborating with experts to develop work-sample tests for use in corporate finance hiring, and am looking for insights around current use of these types of tests." Talking about the development and current use of such tests is quite different from selling tests.
As a recruiter for accounting talent, this discusssion piqued my interest. Accounting is a technical field, and it's essential that candidates be technically astute and those skills be tested, particularly at the staff levels. It goes beyond personality fit.
As a controller & CPA in a previous life, I was brought into clean up many messy balance sheets, a direct result of staff lacking basic understanding of fundamental accounting concepts. So, have to disagree with those adamantly opposed to testing. I employ it in my own practice, and it has served me well. Albeit, it's a baseline for determining who I want to further pursue. It's not the end all be all, but sure streamlines the recruiting process when one is inundated with hundreds of resumes. Also, it shows a level of professionalism when one is willing to take that extra step to verify their qualifications. I know there is quite a bit of push back, but folks it's becoming mainstream. The tech companies such as Uber are making it a part of the hiring process.
4
Some counter-arguments to consider...
1. Companies already have ERPs with entries already mapped out. Even the "lowly" Quickbooks can be learned quickly and even designed for non-accountants. If it was an old school manual (I am betting even some newer CPAs would have difficulty with T accounts..do you still remember columnar sheets?) system, I might agree with you. Depending on the ERP, systems nowadays take out the "technicality" in accounting.
2. Companies already have established and written down step by step procedures. For day to day transactions.
3. In the course of a year, how many "new" (or unfamiliar) transactions do companies experience? And how established are one's procedures in case employees encounter such "new" transactions.
I for one advocate for a procedure to document the treatment of new transactions (which means it goes through vetting and approval) to be included in company documentation and for some, approval by the CFO and/or CEO/Board.
4. How many accounting/finance staff or even mid level managers does one hire coming from an unrelated field? Point is, most if not all come from business (if not accounting) understudies. If not, CPAs already....and on that point, do you also "test" CPAs?
My main point is that the arguments are for unestablished or MISestablished (is that even a word?) systems. It is geared more for realities of the PAST and NOT the current state of accounting and finance. The UNavailability of information/knowledge on accounting and finance is no longer like it used to be (nor is it monopolized by CPAs or to that extent, those with accounting backgrounds). The internet changed all that.
I understand that my points are generalized and does not consider state and lifecycle stage of a company. I just wanted to post the counter arguments to the points raised.
Oh, and BTW, with UBER's (personnel/culture) problems, I dont think it is the appropriate example to cite.
Patricia, it's very interesting that you've found that staff sometimes lack basic understanding of fundamental concepts. Tech companies, as you mentioned, are employing testing as part of the hiring process largely for that reason: the majority of applicants for programmer positions have been found to not be able to write any code at all (for more on this, check out:https://blog.codinghorror.com/why-cant-programmers-program/). As such, a traditional interview in that field might find great personality fits and wonderful communicators, but fail to discern the lack of necessary skills to perform the core functions of the job.
Emerson, thank you for pointing out specific elements that run counter to this idea. Do you feel that these conditions, including established procedures, certifications, and the availability of information, allow you to adequately ensure that new hires are able to perform, for instance, analytical evaluations, or plan/devise procedures for new or unfamiliar encounters?
Shannon,
Hiring is dependent on a matrix of factors where one factor may make up for another. That is one reason why I disagree about "baselines". A company should evaluate candidates on their "totality" and decide if the candidate can be an asset to the organization.
As a hiring concept, I always have the INCLUSION perspective rather than an EXCLUSION one. Meaning, one candidate may NOT have a certain quality/qualification (you can say that this is what people mean by baselines), but what other qualities does he/she have that can make up for it.
When hiring, I always have this Couch K in mind - " I'd take a 2-Star recruit with a 5-Star work ethic over a 5-Star recruit with a 2-Star work ethic any day"
Another counter argument for the "technical" criteria. Show me a candidate that claims he knows ALL the technical aspect of finance and accounting and I will show you a candidate that you should run away from. Another is, does one test for 100% of subjects?
This forum is proof that even CPAs either are unfamiliar with some subjects/issues or they have different interpretations on technical issues. This is not a put down on the certification...just a reality.
I also take exception to the programming and accounting comparison (Tech company "technical" tests) as I think they are apples and oranges.
Emerson,
Tell the small business entrepreneur trying to survive that his books are in disarray because he failed to have documented procedures for transactional posting. He's got more pressing issues like selling. Isn't it understood that when you hire a bookkeeper they come with a certain level of skill set to perform the job?! Herein lie the problem trying to, as you voiced above "....take out the "technicality" in accounting". You cannot take out the technical in accounting, it's technical. We all have benefited exponentially from the advances in technology, no disagreement here. However, as technology automates away mundane tasks which it should, let's make sure the user understands the concepts behind a process, like how to do a bank reconciliation. Real life example here: While hiring for a staff person, I asked candidates to perform a simple bank rec. Had one candidate who labored for more than an hour, until she finally said "well, I could do this in QuickBooks". And that's the point, could click the "button" to have QB spit out a bank rec but no idea of the process behind it. Do you really want to hire an accountant who has not mastered basic concepts from which they can draw upon when encountering new issues or complications that require reasoning and analytical skills? If you understand the process, then you know how to fix it.
I think we do a disservice to future generations if we give them a pass and say no problem, they can Google it or we got it covered with recurring JE's, and don't get me wrong, I use them all, but come on, if we don't demand a certain level of knowledge, we might as well hire monkeys.
AI (artificial intelligence) is on the verge of disrupting every industry. As this happens, there will be an increasing need for higher level skilled workers as jobs disappear. We should be preparing the next generation of workers for what I believe will be an even more competitive marketplace and arming them with an arsenal of skills to be successful. Short sighted to think everybody can be a generalist and no need for specialists.
6
Tell that entrepreneur that he won't get an investor unless he does a-b-c, and that same entrepreneur will document and sing Ave Maria in Macy*s window to attract investors.
I just took a on-line course created for DOD. I passed the myriad of tests. I stopped listening after the third module.
What have I learned?
By the way, life is a test...
As a P.S.
"Do you really want to hire an accountant who has not mastered basic concepts from which they can draw upon when encountering new issues"
This is where we differ in our thinking and I believe the point of my objection to testing.......The knowledge itself is NOT important but the traits of the candidate/employee to learn, understand and seek knowledge.
oops...sorry, I meant...."LESS important" not "NOT" important.
I'm sorry, guys. I had written a long response in this thread last week that seems to have disappeared. My apologies if you saw these details before it vanished, but I want to reiterate in case it was never published.
To clarify, our approach is to optimize the process from application to interview, not to replace the interview. The component of the hiring process that we aim to replace is the one consisting of resume screening and/or phone calls by recruiters who have no knowledge of the field and rely on scripted assessment questions.
Furthermore, technical skills are not necessarily the point of such tests. The goal is to test whatever basics (skills, knowledge, or traits) allow for discernment between a good candidate and a bad candidate. For instance, if you get 500 applications for a job, how do you decide who to interview? Do you interview all of those candidates? Do you run their resumes through a keyword filter and interview only those who are best at writing resumes? Do you have someone from HR call all 500 applicants and ask them scripted questions?
What kinds of things would you want to know at *that* stage, prior to the interview, that would be meaningful in deciding which candidates (plural) to interview?
Again, the idea is for testing to be just a piece in the hiring process, not exclusive of other types of assessments, including interviews.
However, our approach is quite different in regards to the testing than those used in some companies. We focus on short tests, say 15 min., that can be completed online at the candidate's convenience, so that this is not an onerous step inciting flashbacks of final exams. Quick and easy, but still relevant.
Many of you are execs, so I should also clarify that these kinds of work sample tests are not designed for your job level. Think entry to mid-level positions.
Given all of that, Wayne and Emerson, do you still think this kind of testing would run contrary to your hiring aims?
Believe it or not, I look at resumes. Keywords/acronyms change with region/industry/sector.
I end up creating a glossary of terms (at least verbally) with clients so we're all on the same page because of differences in understanding, even with concepts (budget/forecast). So filtering is out.
I'm not a University snob, so I really don't care where you went (and sometimes I question why you sent a resume in if you went to a top-flight school).
Test, well I've made my opinion clear on that.
Contrary to practice, you don't need to do a full on-boarding day 1. You can hire and PAY someone for a probationary period and give them actual work to perform. If they fail, they are let go. If they provide a work product that is acceptable, you can start to full on-boarding toward the end of month 1.
You'll know within a week or so whether you made the right choice.
Shannon,
"Given all of that, Wayne and Emerson, do you still think this kind of testing would run contrary to your hiring aims?"
I do NOT have anything against "testing". However, it should be viewed as a GAUGE (where the candidate is at in terms of technical knowledge) and NOT a qualifying criteria or hurdle.
My hiring aim is to hire the most persistent, curious, resourceful candidate I can afford.
Google's Eric Schmidt in finding a "smart creative" who could "thrive in chaos," -
"...persistence is the single biggest predictor of future success. ... And the second thing was curiosity. What do you care about? The combination of PERSISTENCE and CURIOSITY is a very good predictor of employee success in a KNOWLEDGE ECONOMY...it matters less how you discover those traits in the hiring process than the actual discovery of them."
The knowledge itself is LESS important. If we are testing for knowledge, then we are testing for the wrong things. You test for knowledge you are testing for a ceiling...you look for traits, then the future for an ever changing environment is endless.