Skip to main content
SmartRecruiters

Frequently Asked Questions Setup

 

Your FAQ Dataset is the base for answering your user´s questions. The dataset is divided into categories with unique answers and a set of questions attached to it.

To help you with this important step we have pre-filled your account with a Starter Set. It is a collection of more than 3.000 HR-related questions in 68 categories. These categories were build to reflect patterns in 130,000 questions asked by real candidates. The Starter Set has been anonymized and is available in English and German. You can read more about how this process looked like here.

You will need to take a look at this Starter Set and adjust it for your specific needs. Please consider:

  • Which categories are not suited for your company? 

  • Which categories might be missing but are important for your company? 

You can simply go to the Category view in your SmartPal dashboard and see the whole Starter Set. Go through all categories and evaluate if they are useful for your case or not. If not, simply delete the category. If you feel that some common topics, that are unique to your business, are missing, you can add them with the blue New Category button. You will be able to find those topics usually in your current FAQ page, user-facing inbox, previous chat solutions, phone logs, social media channels (e.g., Facebook Career Page) and so on. It helps to involve the department(s) which already are dealing with answering user queries on a daily basis.

For the categories that you have decided to keep, please do not forget to add your answers since the Starter Set has only placeholders.

How to review and confirm my starter set?

  1. You will get access to your account in SmartPal's dashboard with Starter Set, in a selected language, already in it.

  2. Adjust the Starter Set:

    1. Delete irrelevant categories.

    2. Add your own categories with answers and at least 15 questions. Read more about how to create good categories here.

  3. Replace placeholder answers in Starter Set categories.

  4. Data Review (dataset performance check performed by SmartPal).

  5. Data Review implementation (depending on the support level, done by SmartPal, or your team).

Categories

The FAQ dataset is divided into categories and a set of, attached to it, questions. Our NLP engine compares every incoming question with all queries available in the dataset. When the similarity is detected the incoming question is matched to the appropriate category and the category’s answer is given to the user by the chatbot.

For this process to function as well as possible it is important to have a clean FAQ dataset with well-defined categories. The categories and questions within the set should not overlap. This means very similar questions should not belong to different categories.

How to create good categories?

  • Do not create overlaps.
    The questions in two different categories should be clearly distinct from another. This means avoiding the same question being located in two different categories.

  • Enrich your categories.
    Add if possible add at least 15 questions to each category to avoid weak and worse performing categories.

  • Have fewer categories with more questions in them.
    It is useful to combine all questions for a topic to one category and have a nice well-rounded answer for that category instead of having one topic spread into several smaller categories. For example: Combine questions for parking into one parking category (instead of having three categories: Parking allowed, Parking costs, Parking limit).

 

Keep in mind that with bigger topics it can make sense to split them into several categories, for example:

  • Benefits: Can be divided into Benefits Health insurance, Benefits general, Benefits internship etc.

  • Salary: Can be divided into Salary general, Salary internship, Salary dual studies etc.

It is important in this case to have the category-defining keywords (like an internship, insurance etc.) in the questions. In that way the categories will stay clean and distinctive like in the examples below:

  • “Do you offer healthcare” → Benefits Health insurance category

  • “What benefits do you have” → Benefits general category

  • “What benefits will I have as an intern?” → Benefits internship category

Keep also in mind that for job-specific answers you can use contexts and you should not create separate categories.

Questions

Great chatbot performance depends on questions you create and approve in your dataset. They are the base on which chatbot calculates similarity and provides automatic responses to other, incoming questions. Please find here a few tips on what makes a good (and bad) question:

  • Questions should be short & precise

Too long questions might confuse the NLP engine in picking the correct category since there might be a lot of keywords and irrelevant topics in the query.

Short and precise questions allow the NLP engine to better process the meaning behind them and provide, in the future, better responses to the incoming questions. 

  • Questions should not be too general

Too general questions might confuse the NLP engine in picking the correct category since the category-defining keywords are missing. A general question can possibly belong to several categories so please avoid adding those to any category.

  • Questions should not be just single words or commands

Single words or commands are not questions and since a particular word can belong to several categories it should never be trained to just one - it gives that category too much “power” and may lead to category overpowering other, similar ones. 

  • Questions should be unique for your category

The exact same or very similar questions trained into different categories may cause confusion, meaning that the chatbot will not be able to decide where, similar questions, should belong. This way, it may start providing wrong answers to new incoming questions. If you notice such an overlap it is good to review the categories and shift similar questions into one category. 

  • Questions should not contain personal information

Adding personal information will not only confuse the chatbot but it is against privacy guidelines. Please always avoid adding questions with any kind of personal information like names, contact information, addresses, payment information, and so on.

  • Questions should be in chatbot’s language

Questions in foreign to your chatbot languages should not be trained or approved since every dataset works on just a singular language framework (meaning - a chatbot can fully understand just one selected language). Chatbot has limited knowledge of other languages, this is why you may sometimes see it providing correct categories for foreign questions. But in no case, you should approve those questions into your dataset since it may have a really bad impact on the chatbot’s performance.

  • Questions should not be small talk or insults

Small talk questions and insults will be handled before they arrive to your dashboard so creating small talk categories and adding small talk questions is unnecessary. We currently handle several small talk categories, you can read more about them here

As a Chatbot Trainer, you will spend most of your time in the Training section of your dashboard. Here you will see all incoming questions (with few exceptions, like small talks). 

Incoming questions are a great opportunity to train your chatbot and possibly avoid mistakes in category detection for future similar questions! For that purpose we have built a Training decision tree to help you with your decision-making process:

When managing your dataset follow these steps to create a well-performing chatbot:

  1. Does it look like a relevant and good question?

    1. If No, provide a direct answer or delete it.
      Examples: commands like “internship”, small talks like “what is your name?”, emojis, question marks, etc. should be deleted. Valid questions with personal information should receive a direct answer.

    2. If Yes, move to the next step.

  2. Does the incoming question fit into an existing category?

    1. If Yes, then assign it to the fitting category.

    2. If No, then move to the next step.

  3. Is it likely that a similar question will appear again?

    1. If No, provide a direct answer.

    2. If Yes, create a new category and answer for it. Do not forget to enrich that newly created category with at least 14 other questions.

Answers

The answers to the incoming questions are defined in every category. You can set up two different types of categories: In the case of context-independent categories you will need to create just one answer for your category. In the case of context-dependent categories it is recommended to provide answers for your contexts to make sure that your user will get a response from the chatbot. Read more about context-dependency here

When writing your answers there are few things to keep in mind:

  • Keep them short

The answers should not be too long - ideally 300 to 600 characters. Longer answers do not lead to the best user experience due to the nature of messaging platforms that will divide them into few pieces or will force your user to scroll to see the whole message. You can add a link (please use full links like https:// or http://) to redirect the user for more information if necessary or create a content flow around that topic.

  • Do not start with “Yes!”, “Unfortunately” or similar

The answers should not start with “Yes!”, “Unfortunately” or similar because, as you can see in the example, not all questions that users ask require a “Yes” or “No” response from the chatbot and such answers would not be ideal UX.

  • Be informative

The answers should be informative and not only consist of links redirecting them to the content. Of course you can include links leading the user to more information if necessary but the link should not be the only part of the answer.

  • Cover all questions 

It is important that all questions in a category are covered by the answer otherwise it could happen that, even though the correct category for an incoming user question was detected, the answer does not fit or the information given is insufficient.

  • Other things to keep in mind:

    • All links in the answers should start with http:// or https:// otherwise they will not be clickable in the chat window later on.

    • Feel free to be less formal, use emojis and exclamation marks to give your chatbot a character.

    • Do not use bullet points because they are not rendered properly in the messaging platforms. 

Common questions:

  1. Can I create hyperlinks?
    Unfortunately, it is not possible. Please use full links (starting with http:// or https://) instead to make sure that they are clickable for the user. You can of course use link shorteners like bit.ly to provide better UX in a case of long links. 

  2. Can I make text bold, italic, or apply any other formatting to it?
    Unfortunately, it is not possible.

  3. Can I add emojis to my answers? How to do it?
    Yes, you can! Please copy and paste the image of your emoji from any database available online (for example https://emojipedia.org/)

  4. Can I add images or videos to my answers?
    Unfortunately, it is not possible. If your chosen platform supports it, you can direct your questions into the Content Flow that can handle images and videos. 

  5. Can I provide an answer with follow-up buttons or quick replies?
    You cannot do it in your dataset but you can, with support from your Implementation Manager, create a category that will lead to the Content Flow block. That block can contain buttons or quick replies that will lead to more content.

  6. How can I create a fallback answer in my context-dependent categories? 
    Unfortunately, it is not possible. Please make sure that all possible contexts have an answer before going live. The general answer will only be given when a question is not asked in the context of a job.