Testing AI for Boolean Search Strings: Which is Best?

Discover which AI tool reigns supreme for crafting complex Boolean search strings to find the perfect candidates. I put ChatGPT, Gemini, Perplexity, Claude, Mistral, and others to the test.

The rise of artificial intelligence has led many to believe that the days of Boolean search strings are numbered. Why bother to understand or learn complex queries and operators when AI can understand natural language and find the perfect candidates for you?

But not so fast. Despite the incredible capabilities of today's AI, Boolean search remains a powerful tool in a recruiter's arsenal for a few reasons:

1. Specificity: Boolean strings allow you to zero in on exact skills, qualifications, and experiences in a way that AI may struggle with. You can fine-tune your search to find needles in the candidate haystack.

2. Control: With Boolean search, you, as the recruiter, remain in the driver's seat. You can iterate on your search, adding and excluding terms as you refine your requirements. AI may make assumptions or miss important criteria.

3. Transparency: Boolean strings make your search logic explicit. You know exactly why a candidate was included or excluded. AI can sometimes feel like a mysterious black box.

Instead of declaring Boolean dead, a better approach is to see how AI can take our searches to the next level. I’ll test some of the top AI tools to see how they handle building a complex recruiter search.

My mission in this article will be:

• Find an accountant based in Prague, Czech Republic with knowledge of US GAAP and experience conducting financial analysis.

I'll be testing the Boolean chops of ChatGPT, Gemini, Perplexity, Claude, Mistral, and others. For each tool, I'll give an overview, share the Boolean string it generated, and evaluate its performance. Finally, I'll crown a winner in this battle of the Booleans.

You’ll find the results and the top 10 most effective tools at the end of this article.

---

Boolean Search String Test

To keep this test fair, I asked a recruiter friend who just got started to share the prompt he would use. This way, I’m not creating expert-level prompts that could influence the results. The idea is to see what everyday users would actually ask AI tools.

To test ChatGPT's Boolean skills, we gave it the following prompt:

"Create a Boolean search string to find candidates who meet these criteria:
- Accountant
- Based in Prague, Czech Republic
- Knowledge of US GAAP
- Experience conducting financial analysis"

---

ChatGPT

• URL: https://chatgpt.com/

• ChatGPT is a conversational AI model developed by OpenAI. It's been trained on a vast corpus of text data, allowing it to engage in dialogue and assist with a variety of tasks, including coding, analysis, and creative writing.

Boolean String Test

Here is the Boolean string that ChatGPT generated:

(Accountant OR "Financial Accountant" OR "Senior Accountant") AND ("Prague" OR "Czech Republic") AND ("US GAAP" OR "Generally Accepted Accounting Principles") AND ("financial analysis" OR "financial reporting" OR "financial modeling")

Analysis

This Boolean search string isn't incorrect per se, but it's too broad and potentially inefficient for finding an accountant specifically with knowledge of US GAAP and experience conducting financial analysis. Here's why:

• Overly Inclusive "OR" in the Skills Section: The biggest issue is the final "OR" grouping: ("financial analysis" OR "financial reporting" OR "financial modeling"). This means the search will return candidates who have experience in any one of these areas, not necessarily someone with specific experience in financial analysis. You'll likely get many results for financial reporting or modeling specialists who may not have the financial analysis skills you're looking for.

• Overly Broad Location: Using "Prague" OR "Czech Republic" is redundant and inefficient. Someone living anywhere in the Czech Republic will be included. The intent is likely to find someone in Prague, so simply searching for "Prague" would be more targeted. Including the broader "Czech Republic" widens the search significantly and introduces irrelevant candidates from other cities.

ChatGPT produced a solid first attempt at the Boolean string, demonstrating a grasp of the basic concepts. However, the string would likely require some manual refinement to improve its precision and recall. The main problem is with "Czech Republic"—this is something that’ll give you the wrong results.

Perplexity

• URL: https://www.perplexity.ai/

• Perplexity is an AI-powered search engine that provides direct answers to queries. It leverages large language models and knowledge retrieval to understand the intent behind a search and surface the most relevant information.

Boolean String Test

Here is the Boolean string that Perplexity generated:

("Accountant" OR "Certified Accountant" OR "CA") AND ("Prague" OR "Czech Republic") AND ("US GAAP" OR "Generally Accepted Accounting Principles") AND ("financial analysis" OR "financial analyst")

Analysis

This Boolean search string looks a little bit better, but it still suffers from some of the same issues regarding accuracy. Here's why:

• Still Broad Location: The use of ("Prague" OR "Czech Republic") remains a primary issue. It's still unnecessarily broad and will include candidates from all over the Czech Republic, not just Prague. This creates noise in your results.

• Improved Title Specificity: Adding "Certified Accountant" and "CA" is a positive change. It helps target candidates with professional accounting qualifications. However, remember that different countries have different certifications, so "CA" might mean something different depending on the context of the profile.

• More Focused Financial Skills: Changing ("financial analysis" OR "financial reporting" OR "financial modeling") to ("financial analysis" OR "financial analyst") is a significant improvement in focusing on candidates with direct experience in financial analysis. This reduces the potential for irrelevant results who primarily focus on reporting or modeling.

---

---

Anthropic Claude

• URL: https://claude.ai/

• Claude is an AI assistant created by Anthropic, an AI research company. It's built using "constitutional AI" principles which aim to make AI systems more steerable, interpretable, and transparent. Claude excels at tasks requiring analysis, coding, and written communication.

Boolean String Test

Here is the Boolean string that Claude generated:

Results from Haiku 3.5 included in the FREE version:

(Accountant OR "Accounting Professional" OR "Finance Professional") AND ("Prague" OR "Czech Republic") AND ("US GAAP" OR "United States Generally Accepted Accounting Principles") AND ("financial analysis" OR "financial modeling" OR "financial reporting" OR "financial analytics")

Analysis

• Overly Broad "Professional" Titles: Including "Accounting Professional" and "Finance Professional" significantly broadens the search beyond just accountants. This could pull in individuals in various finance roles that don't possess core accounting skills or US GAAP expertise.

• Location Redundancy: Similar to the original, using ("Prague" OR "Czech Republic") is inefficient and will return candidates from anywhere in the Czech Republic, not just Prague.

• Broad Financial Skill Terms: While including "financial analytics" is relevant, it adds another layer of potential noise. Someone doing data analysis related to finance might be included even if they lack the deep financial analysis experience of an accountant.

Results from Sonnet 3.5 included in the paid version:

("accountant" OR "accountancy" OR "accounting") AND ("Prague" OR "Praha") AND "Czech Republic" AND ("US GAAP" OR "USGAAP" OR "U.S. GAAP" OR "United States GAAP") AND ("financial analysis" OR "financial analyst" OR "financial reporting" OR "financial modeling")

Analysis

• Location Inefficiency: While including "Praha" (the Czech spelling of Prague) is a good idea, having both ("Prague" OR "Praha") AND "Czech Republic" is a bit redundant. If someone mentions "Praha," it's highly likely they are in the Czech Republic. This structure doesn't necessarily hurt the search, but it's not the most efficient way to phrase it.

• Overly Broad Initial Terms: Using ("accountant" OR "accountancy" OR "accounting") is a bit too broad. While related, "accountancy" and "accounting" could refer to academic fields or general departments, not necessarily individuals holding the title of "accountant."

Mistral

• URL: https://mistral.ai/

• Mistral AI is a French artificial intelligence startup founded in 2023, specializing in the development of open-source large language models (LLMs). The company, based in Paris, was established by former researchers from Google DeepMind and Meta, and aims to provide accessible, customizable, and cost-effective AI solutions that require fewer computational resources compared to other leading models.

Boolean String Test

Here is the Boolean string that Mistral AI generated:

("Accountant" OR "Financial Analyst" OR "CPA" OR "Chartered Accountant" OR "Accounting Manager" OR "Senior Accountant") AND ("Prague" OR "Czech Republic") AND ("US GAAP" OR "GAAP" OR "Generally Accepted Accounting Principles") AND ("Financial Analysis" OR "Financial Analyst" OR "Financial Reporting" OR "Budgeting" OR "Forecasting" OR "Variance Analysis")

Analysis

This string shows some improvements but still has accuracy issues, particularly in overbreadth. Here's a breakdown:

• Job Titles: ("Accountant" OR "Financial Analyst" OR "CPA" OR "Chartered Accountant" OR "Accounting Manager" OR "Senior Accountant") - This section is quite comprehensive and includes relevant job titles and professional certifications. It's a strong point of the string. However, the inclusion of "Financial Analyst" can broaden the search slightly beyond purely accounting-focused roles. While there's overlap, a financial analyst's primary focus might not be core accounting and US GAAP.

• Location: ("Prague" OR "Czech Republic") - This is still the primary source of inaccuracy and inefficiency. As pointed out in previous analyses, using OR in this way is redundant and overly broad. You will get candidates located anywhere in the Czech Republic, significantly widening your search and requiring you to filter out those not in Prague.

• US GAAP Knowledge: ("US GAAP" OR "GAAP" OR "Generally Accepted Accounting Principles") - This section has a significant flaw. While "US GAAP" and "Generally Accepted Accounting Principles" (when used in a US context) are relevant, including just "GAAP" is far too broad. "GAAP" stands for Generally Accepted Accounting Principles, but every country has its own set of GAAP. This will pull in accountants with knowledge of any country's GAAP, making the search significantly less accurate for finding someone with US GAAP experience.

• Financial Skills: ("Financial Analysis" OR "Financial Analyst" OR "Financial Reporting" OR "Budgeting" OR "Forecasting" OR "Variance Analysis") - This section is quite broad, covering a range of related but distinct skills. While these skills are often found in accounting roles, someone heavily focused on budgeting or forecasting might be included even if their financial analysis experience is limited. The inclusion of "Financial Analyst" again contributes to this broadening, as their skill set could be different from a core accountant doing financial analysis.

Google Gemini

• URL: https://gemini.google.com/

• Google Gemini, formerly known as Bard, is a state-of-the-art multimodal AI model developed by Google DeepMind, announced on December 6, 2023. Designed to simulate human-like conversations and interactions, Gemini integrates natural language processing with the ability to understand and generate responses based on various data types, including text, images, audio, and video. This capability allows it to perform complex tasks such as image recognition, audio processing, and code generation seamlessly.

Boolean String Test

Here is the Boolean string that Gemini AI generated:

Results from Gemini 1.5 included in the FREE version:

("Accountant" OR "Accounting") AND ("Prague" OR "Czech Republic" OR "Czechia") AND ("US GAAP" OR "Generally Accepted Accounting Principles (US)") AND ("Financial Analysis" OR "Financial Analyst" OR "Financial Planning & Analysis" OR "FP&A")

Analysis

• Job Titles: ("Accountant" OR "Accounting") - This is a bit broad. While "Accountant" is a direct hit, simply using "Accounting" could pull in results related to academic programs, accounting software, or departments, rather than individuals with the specific job title of "accountant."

• Location: ("Prague" OR "Czech Republic" OR "Czechia") - This is better than previous examples in terms of coverage, acknowledging the alternative name "Czechia." However, the fundamental issue of using OR between Prague and the country still exists. It will still return candidates located anywhere in the Czech Republic, not just Prague. While comprehensive in terms of names, it's still inefficient for finding someone specifically in Prague.

• US GAAP Knowledge: ("US GAAP" OR "Generally Accepted Accounting Principles (US)") - This is a good improvement. It explicitly specifies "Generally Accepted Accounting Principles (US)" which helps clarify the intent and reduces ambiguity compared to just "GAAP."

• Financial Skills: ("Financial Analysis" OR "Financial Analyst" OR "Financial Planning & Analysis" OR "FP&A") - This is a decent range, covering both the activity ("Financial Analysis") and related roles/department names ("Financial Analyst," "Financial Planning & Analysis," "FP&A"). It targets the desired skillset effectively.

Results from Gemini 2.0 included in the PAID version:

("accountant" OR "accounting" OR "CPA" OR "chartered accountant") AND ("Prague" OR "Praha") AND ("Czech Republic" OR "Czechia") AND ("US GAAP" OR "Generally Accepted Accounting Principles (US)") AND ("financial analysis" OR "financial analyst" OR "financial modeling" OR "financial reporting")

Analysis

• Job Titles: ("accountant" OR "accounting" OR "CPA" OR "chartered accountant") - Similar to Gemini 1.5, "accounting" is still a broad term. However, the addition of "CPA" and "chartered accountant" significantly improves the targeting of qualified professionals.

• Location: ("Prague" OR "Praha") AND ("Czech Republic" OR "Czechia") - This is an interesting structure. Using AND between the city and country terms is a step in the right direction to narrow the location. However, within each set, the OR still exists. This means it will capture anyone who mentions either "Prague" or "Praha" and anyone who mentions either "Czech Republic" or "Czechia." While it might seem like it narrows it, it's still not perfectly precise. It's better than just OR for the whole location, but not ideal.

• US GAAP Knowledge: ("US GAAP" OR "Generally Accepted Accounting Principles (US)") - This is the same improvement as in Gemini 1.5, clearly specifying US GAAP.

• Financial Skills: ("financial analysis" OR "financial analyst" OR "financial modeling" OR "financial reporting") - This is a good, relevant set of skills for an accountant with financial analysis experience. It's similar to the financial skills in Gemini 1.5, but replaces the more specific "Financial Planning & Analysis" and "FP&A" with "financial modeling" and "financial reporting," which are also relevant.

Head-to-Head Comparison

Looking at this comparison, a few key observations stand out:

1. Location Targeting: Most of the AI tools used an overly broad location search, including both "Prague" and "Czech Republic." This is likely to return many irrelevant candidates from outside Prague. The human-generated string and a few of the more advanced AI models (like Google Gemini Paid) narrowed this down more effectively.

2. Job Title Specificity: The AI-generated strings varied widely in their job title targeting. Some, like ChatGPT, kept it fairly narrow to core accounting roles. Others, like Mistral, included a broader range like "Financial Analyst" which could dilute the results. The human string struck a balance by including both key accounting titles and relevant certifications.

3. US GAAP Phrasing: Most of the AI tools included "US GAAP" and "Generally Accepted Accounting Principles," which is good. However, some, like Mistral, also included just "GAAP" which is too broad and could include any country's GAAP. The human string and a few AI ones (like Google Gemini) were more precise by specifying "US" or "United States."

4. Financial Skill Breadth: The AI strings ranged from relatively narrow (Perplexity) to quite broad (Mistral). Broader strings could return candidates with less directly relevant experience. The human string focused on core financial analysis skills and the closely related FP&A.

5. Exclusions: Only the human string and LLaMA included a NOT clause to exclude auditors, which can be a major source of irrelevant results when searching for financial analysts.

So, what can we conclude from this experiment? First and foremost, while AI has made remarkable strides in natural language understanding, it's not a magic bullet for complex Boolean searches. The AI-generated strings, while mostly sensible, often suffered from subtle but significant issues like location overbreadth, job title dilution, and lack of exclusions.

Interestingly, the performance did vary quite a bit between AI models. ChatGPT—currently the most famous and widely used tool—actually produced one of the weaker strings. Meanwhile, some of the more advanced and specialized models, generated more sophisticated and targeted searches.

However, even the best AI strings didn't quite match the precision and thoughtfulness of the human-crafted one. This doesn't mean AI has no place in Boolean search—far from it. AI tools show exciting potential. But at this stage, AI is best used as a complement to, not a replacement for, human expertise. An experienced recruiter can use AI to generate an initial string, then refine it based on their knowledge of the role, industry, and search best practices.

---

---

What is the Best AI for Recruiters?

Want to know who outsmarted the AI and which AI tool is best for creating Boolean search strings? Keep reading!

Top 10 Best Boolean Search Strings

1. Behind Paywall

2. Behind Paywall

3. Google Gemini 2.0 (Paid)

4. Perplexity

5. Google Gemini 1.5 (Free)

6. Behind Paywall - My favorite tool

7. Mistral

8. Anthropic Claude (Sonnet 3.5 - Paid)

9. ChatGPT

10. Anthropic Claude (Haiku 3.5 - Free)

Let’s check out who’s in first and second place and see how this Top 10 list was put together: