Which AI Auto-Scoring Options Are Offered by Canditech?

Canditech’s AI auto-scoring capabilities help you evaluate open-ended candidate responses faster, more consistently, and with greater objectivity. By leveraging AI, you can reduce manual review time while ensuring that every candidate is assessed according to the same clear standards.

 

Which Question Types Support AI Auto-Scoring?

AI Auto-scoring is available for both open-text and one-way video questions.

For video responses, Canditech automatically converts the candidate’s speech into text and evaluates only the written transcription. Factors such as accent, tone of voice, appearance, background, or recording environment are never taken into account, allowing you to focus purely on what the candidate says rather than how they say it.

 

The AI Auto-Scoring Methods Available in Canditech

Canditech currently offers three auto-scoring approaches, so you can choose the level of structure and customization that fits the role:

1. Example answers
2. Pre-Built AI Agents
3. Custom AI Auto-Scoring (Custom Agents)

1. How Does “Example-Answer” AI Scoring Work?

In this approach, you provide example responses along with the scores they should receive. The AI compares each candidate’s answer to those examples and assigns a score based on similarity. The closer the candidate’s answer is to the examples, the closer the score will be.

This method is especially useful for questions with relatively clear or expected answers, where you already know what a strong response looks like.

 

2. What Are Canditech’s Pre-Built AI Scoring Agents?

Canditech includes several pre-built AI agents, each designed to evaluate a specific competency. These agents are ready to use and require no configuration.

• Language Agent – evaluates grammar, spelling, and overall language clarity in any language.
• Objection Handling Agent – evaluates how candidates respond to objections in a sales context. This agent focuses on how well the candidate acknowledges objections, responds professionally, and moves the conversation forward. It is particularly relevant for sales, business development, and account management roles.
• Conflict Resolution Agent – evaluates how candidates handle frustrated or dissatisfied customers. This agent focuses on empathy, communication, and problem ownership, making it especially suitable for customer support and customer success roles.

3. What Are Custom AI Auto-Scoring Agents?

Canditech’s Custom Agents are the most flexible and powerful AI auto-scoring option.

This option allows hiring teams to decide exactly what they want to evaluate in a candidate’s answer by defining their own scoring rules. Instead of relying on generic criteria, you can translate your expectations, priorities, and evaluation standards directly into measurable checks.

Custom agents are especially useful when you want to align evaluation with your company culture and preferred ways of thinking or working, ensuring that candidate responses reflect your organization’s values. They are also ideal for addressing unique business challenges, allowing you to assess how candidates would approach scenarios that are directly relevant to your company’s day-to-day operations.

How to Build Your Own Custom AI Auto-Scoring Agents

Canditech’s custom auto-scoring agents allow you to automatically evaluate open-text and video questions using a scoring rubric you define. This enables you to consistently assess responses without relying solely on manual review.

Using the “Build custom agent” option, you can create a custom scorecard that clearly defines what the agent should look for in a response. This may include specific concepts that must be mentioned, required skills or knowledge areas, expected depth of explanation, or other criteria tied to the question itself.

Custom agents can be applied across a wide range of use cases, from more general responses to highly specific or niche skill sets. They are particularly useful when you want scoring to reflect your internal evaluation framework, rather than a generic assessment approach.

Once configured, the agent applies your custom rubric consistently across all candidate responses, making it easier to compare results at scale while maintaining alignment with your scoring standards.

 

What is a “Custom Agent” (in Canditech terms)?

When we talk about building a “custom agent” for AI auto-scoring on Canditech, you’re not coding an AI from scratch. Instead, you’re defining a set of clear, objective rules that our underlying AI agent will follow to evaluate candidate responses. Each rule acts as an independent “check” for a specific aspect of the candidate’s answer, resulting in a simple “Pass” or “Fail” for that particular point.

 

How is the Score Calculated?

The final score represents the percentage of rules the answer successfully satisfied. For example, if you defined 5 rules and a candidate met 3 of them, they will receive a score of 3 stars.

This means you set the evaluation standards, and our AI ensures they’re applied consistently and fairly to every candidate.

 

Why Should Hiring Teams Use Custom AI Auto-Scoring?

While our pre-built agents cover a wide range of roles and skills, building your own custom agents allows you to:

1. Save Valuable Time with Auto-Scoring: Automatically evaluate candidate responses, freeing hiring managers from manual reviews and accelerating decision-making.
2. Address Unique Business Challenges: Evaluate how candidates would tackle scenarios directly relevant to your company’s operations.
3. Boost Accuracy & Fairness: Minimize human bias and guarantee every candidate is scored against the same precise, objective criteria.
4. Align with Company Culture: Ensure candidates’ responses reflect your organization’s values or preferred approaches.

These benefits highlight how advanced AI recruiting tools are enabling more consistent, scalable, and objective hiring processes.

Curious how AI auto-scoring works in practice? Book a demo to see how Canditech lets you create custom agents that score candidate responses based on your exact criteria.

 

How Do You Create Effective AI Auto-Scoring Rules?

Creating effective scoring rules is straightforward. Follow these principles to build a custom agent that delivers accurate, insightful scores:

Principle 1: One Rule, One Clear Check

• The Goal: Each rule should measure a single, distinct aspect of performance. Avoid combining multiple ideas into one statement.
• Why it Matters: Our AI agent evaluates each rule independently. If a rule has two parts, and a candidate partially meets it, the AI might struggle to give a definitive “Pass” or “Fail.”
• Good Example ✅
  • “The candidate identified at least one negative trend, efficiency gap, or campaign outcome that fails to meet the target KPIs.”
  • “The candidate acknowledges potential risks in their solution.”
• Avoid ❌
  • “The candidate suggests an innovative solution and considers all its potential impacts.” (This is two checks: innovation + impact assessment.)

 

Principle 2: Focus on Observable Behaviors or Content

• The Goal: Rules should check for things the AI can actually “see” or “read” in the candidate’s written response (specific words, phrases, concepts, or structural elements).
• Why it Matters: The AI interprets text. The more concrete and explicit your rule, the better the AI can apply it consistently. It can’t infer emotions or intentions not directly expressed.
• Good Example ✅
  • “The response includes a specific target audience for the marketing campaign.”
  • “The candidate uses professional and respectful language throughout the response.”
• Avoid ❌
  • “The candidate demonstrates motivation.” (“motivation” is too abstract).

 

Principle 3: Include All Necessary Context

• The Goal: Ensure the rule contains all the information the AI needs to make a judgment, without requiring it to “remember” details from the original question prompt or previous rules.
• Why it Matters: Each rule is evaluated independently. The AI doesn’t carry context from outside the specific rule text. If your rule references something, it must be explicitly included or fully described within that rule.
• Good Example ✅
  • (If the question mentioned a client named “Acme Corp”): “The candidate addresses ‘Acme Corp’ by name in their email draft.”
• Avoid ❌
  • “The answer gives 3 reasons for the chrome indicators presented above.” (The AI won’t know what “chrome indicators presented above” refers to because that context isn’t within this specific rule.)

 

Principle 4: Align with Competencies & Question Intent

• The Goal: Each rule should connect directly to the specific skills, knowledge, or judgment you intend to assess with that open-text question.
• Why It Matters: Well-aligned rules help ensure that you’re evaluating what truly matters for the role, and nothing else. When scoring criteria drift away from the question’s original intent, you might unintentionally measure unrelated traits, which can make results less reliable and less consistent.
• Example: If your question is designed to assess prioritization skills, a rule like “The candidate identifies the most critical task first” keeps the focus on the intended competency. Adding unrelated elements (like writing style or tone) could unintentionally skew results and reduce the accuracy of your evaluation.

 

Principle 5: Include Examples for Clarity

• The Goal: Provide illustrative examples of what a “Pass” looks like, so any grader (human or AI) understands exactly what constitutes meeting the rule.
• Why it Matters: This is crucial for consistent, objective, and fair scoring. Your examples help the AI understand the intent of the rule, not just the literal phrasing.
• Example: “The candidate clearly explains why their chosen approach is superior to alternatives. (e.g., ‘This method is faster because…’, ‘It’s more cost-effective due to…’).”

 

Principle 6: Avoid Subjectivity and Vague Terms

• The Goal: Be precise. Replace general or interpretive terms like “good,” “strong,” “appropriate,” or “effective” with specific, measurable indicators.
• Why it Matters: Subjective terms are open to different interpretations, making consistent AI scoring difficult. Define what you expect in concrete terms.
• Good Example ✅
  • “The candidate provides at least three distinct steps in their proposed action plan.”
• Avoid ❌
  • “The candidate gives a good answer.”

 

Principle 7: Calibrate for Proficiency, Not Perfection

• The Goal: Write rules that a solid, qualified candidate will pass. A score of 100% (5 stars) should indicate a “Strong Hire,” not necessarily a rare, one-of-a-kind genius.
• Why it Matters: If your rules are too strict or look for “extraordinary” nuances only, excellent candidates might receive average scores. The rules should cover the essentials of a good answer, ensuring that anyone who meets your standard for the role receives a high score.
• Good Example ✅
  • “The candidate mentions at least one risk mitigation strategy.” (A standard expectation).
• Avoid ❌
  • “The candidate provides a highly unique and never-before-seen mitigation strategy.” (Too high a bar; implies only outliers get full points).

 

By following these guidelines, you’re not just creating a checklist; you’re building a sophisticated, tailored evaluation framework that ensures your skill assessments collect deep, objective insights you need to pinpoint and hire the best talent for your organization. 

Ready to start building your custom rules? Book a meeting with an expert to empower your hiring process with AI precision!

 

How Do You Review AI Auto-Scoring Results in Canditech?

The candidate assessment report is your primary tool for making data-driven hiring decisions. While manual review is often a key part of the process, Canditech’s AI Auto-Scoring provides an immediate layer of insight, offering a standardized evaluation of candidate responses for Open-Text and Video questions.

Understanding how to read these results, explore the underlying logic, and apply your own judgment is essential for a seamless grading experience.

 

Understanding Your AI Auto-Scoring Options

Before diving into the results, it’s helpful to know which “lens” the AI is using to evaluate your candidates. Canditech offers three distinct auto-scoring modes: Pre-built agent for specific skills (e.g. Conflict Resolution, Language Proficiency), Example Answers, and Custom Agents, which allows you to define your own specific rubric.

 

Reviewing AI Auto-Scoring Results on a Candidates’ Session

When you open a candidate’s assessment report, scroll to the specific question you wish to review. If AI Auto-Scoring was enabled, you will see a dedicated AI Results section below the candidate’s answer.

If you find a question with only the standard 5-star rating and no AI Results row, it simply means the question was not set for auto-scoring and requires your manual input.

A single question can be evaluated by multiple AI models simultaneously (e.g., Conflict Resolution and Language Score). In these cases, the system distributes the weight equally: if two methods are used, each accounts for 50% of the question’s total grade. If three are used, each contributes a third.

 

Expanding the Logic: Why the AI Scored This Way

Transparency is at the heart of our AI. By clicking on any of the scoring methods, you can expand it to see a detailed 1-5 star breakdown and a written justification. This rationale explains the “why” behind the grade. For example, if a candidate receives a lower score on a language model, the AI will specify whether it found significant grammatical errors or if the tone was simply inconsistent with professional standards. This allows you to understand the AI’s perspective before you decide to accept or challenge it.

Interpreting the Custom AI Agent

For those using the Custom Agent options, the interface becomes even more granular. Instead of just a general star rating, you will see a list of specific, binary rules that you defined for the role.

Next to each rule, a green checkmark (✅) or a red X (❌) will clearly indicate whether the candidate met that specific requirement. Just like the general models, each rule includes a detailed explanation of the rationale used to reach that conclusion. This format makes it incredibly easy to see exactly where a candidate excelled and where they fell short of your specific benchmarks.

The Power to Override: Human-in-the-Loop

While the AI provides a strong baseline, we believe the final hiring decision should always remain in human hands. If you disagree with an AI determination, overriding it is simple and intuitive.

For the Custom AI Agent: Because the score is built on specific rules, overrides are done at the rule level. If you disagree with the AI’s “Pass” or “Fail” for a specific point, click the Thumbs Down icon. This will flip the result (e.g., changing a “Fail” to a “Pass”) and automatically recalculate the total weighted score. You can always click the Thumbs Up icon to revert to the AI’s original logic.

For general star ratings: You can simply click on the number of stars (1–5) you believe is correct. This will immediately update the score to reflect your judgment.

How Does the Star Rating System Convert Scores Into Points?

The star rating system converts your rating into a fixed percentage of the question’s total points. Each star corresponds to a percentage of the score, and that percentage is applied to the total points allotted for the question.

For example, if a question is worth 20 points:

• 1 star = 0% → 0 points
• 2 stars = 25% → 5 points
• 3 stars = 50% → 10 points
• 4 stars = 75% → 15 points
• 5 stars = 100% → 20 points

This ensures consistent scoring across all manually reviewed questions, such as open-text and video responses.

You’ve scrolled this far, so you might be serious about improving how you evaluate candidates. Book a demo to see how Canditech’s AI auto-scoring can help you assess skills faster and more consistently.