11 Questions I Keep in My Notes App for Every AI Vendor Meeting

Caroline Kiesler, Director of Human Resources, Workwear Outfitters

Artificial Intelligence is here in the human resources and talent acquisition space. From resume screening and candidate matching to workforce planning and onboarding, AI promises increased efficiency, faster decisions, and more data-driven everything. I’m genuinely excited about where AI is headed and how it will shape the next 20 years of my career. It fascinates me and pushes me to think bigger.

But excitement doesn’t mean abandoning caution.

As an HR/TA leader, my job is to ensure that the innovation I’m willing to invest in doesn’t outpace ethics or the human experience. I’m always asking: Is this right for my company? Does it make hiring better, more efficient, fair, inclusive, and transparent? Worst-case scenario—if I’m sitting in a deposition because something went wrong, where is my confidence and who is sitting beside me?

That’s why I keep a running list of questions on my phone that I ask every AI vendor I meet with. These help me cut through the hype and lead deeper conversations focused on what matters most: safety, fairness, risk, long-term trust, in addition to what the tool does and how it will function for my business.

And I know I'm not alone. According to the 2025 Axios Harris 100 poll, 77% of Americans say they want companies to develop AI more slowly, so they get it right the first time, even if it delays new breakthroughs.

Here are the 11 questions (and counting) I have saved in my phone:

1. What controls are in place to detect and correct errors or inconsistencies in candidate or employer data?

2. How is data tracked? Can we trace insights or scores back to specific data inputs?

3. How frequently is the underlying model trained? Does it pull in new data to stay relevant and accurate? Are some inputs weighed more heavily than others?

4. What processes are in place to mitigate algorithmic bias across age, gender, race, and background? Can the system flag unconscious bias introduced by a human user?

5. Do you have documentation or audit trails to demonstrate fairness in predictions and decision-making?

6. What role does your AI play in supporting compliance with EEOC, GDPR, and the EU AI Act in terms of bias and fairness?

7. Are there tools that help hiring managers and recruiters understand why a candidate is prioritized or scored higher?

8. Who is responsible for monitoring and validating the AI models—your team or mine?

9. Have your algorithms been independently audited for bias and compliance in the U.S. and other markets where you operate? How often?

10. Can the software flag disparities in outcomes before they escalate into legal or ethical issues?

11. Are candidates/employees informed when AI is used in their evaluation? Can they opt out or appeal a decision?

I’m all in on the future of AI in talent. But I’m equally committed to ensuring that the future is safe, fair, and human-centered. These 11 questions aren’t meant to stall progress—they’re meant to spark deeper, more meaningful conversations.

If you’re evaluating AI for your HR/TA stack, bring deep curiosity along with some healthy caution. The right vendor won’t just tolerate these questions—they’ll welcome them.

AI

Artificial Intelligence

Legal