NLU module does not detect slots correctly

I am using the latest version of botpress i.e. 12.1.1. For the user input in the emulator that exactly matches with one of the utterances for the intent I could see that the intent is detected correctly but all the slots are not detected.
My intent has the utterance - tell me the status of objectives regression that ran on latest branch
Here “objectives” and “latest” are marked as two different slots. “latest” gets detected by “objectives” doesn’t. From the debugger I could see that the slots array is just 1 instead of 2. I can’t understand what could be reason.

For another utterance - “what is the status of crediting regression running on master branch” both the slots do get detected.

For this utterance which has only one slot - “what is the status of icm regression” again botpress does not detect “icm” as the slot.

I do not understand the difference between the test outputs since all of them are exact matches in the intent.

Any suggestions on what could be going wrong would be really helpful.

Thanks

Hi Shahamit,

It’s a bit difficult helping with specific slot extraction without seeing the whole picture of the utterances, intents and entities.
Have you tried adding training utterances?
Are these “any” slots or are they linked to an entity?

Regards,
JB

If possible, please send us your intent(s) & entities in private message and we’ll have a look.

Thanks for the reply.

Yes I do have 10 utterances for this intent but note that I am passing one of the exact utterances as user input.

Also the slots are linked with entities (they are not ‘any’)

Thanks.

Ok sure. I will share them by tomorrow

OK thanks! I did some testing with my own entities and utterances, which seems to work well – so having your dataset will really help us identify the problem.

2 Likes

Appreciate your efforts in troubleshooting the issue.
After this discussion, I realized that every possible slot value needs to be added as an “occurrence” in the entity. The earlier user inputs that I reported weren’t all as an occurrence in the entity hence the slot wasn’t getting detected.

Does that mean that I should have all possible values for the entity as an occurrence ? That sounds impossible for my use case since the product and branch names (entities) could be any alphanumeric string.

any suggestions?

Thanks.

@sylvain - kindly share your inputs to the my query on how to configure the entities such that it understand any alphanumerics strings.

Thanks.

Hi Shahamit,

Is there a pattern to these alphanumeric strings, could you use a pattern extraction (regex).

If there is no pattern, and they could be literally anything, you have to use they “any” type.

You don’t need to add every single possibility, you just need to add multiple utterances with different syntaxes of where it should expect that alphanumeric string, making sure to tag the slot every time.

Also, I recommend populating your intent with the widest array possible of different examples of what that “any” slot could look like.

Regards,
JB

2 Likes

Sounds good. I added more utterances and the output looks better.

Does it help if the slot is configured with both an entity of type list and also system.any?

Thanks

Hi Shahamit,

Funny you would ask that question, I had a conversation with the engineers about this subject, just last week.

Unfortunately, I don’t remember the conclusion of that conversation. @EFF @sylvain might be able to enlighten us about this.

Regards,
JB

@shahamit Depending on your use case, it could (i.e if your slot can be anything).

Keep in mind that if you set your slot to any, Slot Extractor will be looking for words(or group of words) not matching entities. Meaning that if you want your slot to always respect specified entities, then don’t mark it as any.

Sorry for the fuzzy answer but that really depends on your use case.

We ran a couple of tests on this just yesterday too. Based on our experience, system.any can actually mix things up a bit. So probably best to try to use more specific entities. Either system entities, or if all else fails then create a pattern based custom entity and add that.

In our case we were detecting weight and height (of the person chatting) (within the text I mean, not actually detecting the person’s weight :slight_smile:) , which has the added twist of many different formats (78kg, 78 kilograms, 198 pounds, or, even better, 178cm versus 5feet 9inches or 5’9" or even “five feet”, etc.) embedded into many different sentce structures (“I am 178cm”, “My height is 5’9” ", etc.). So we found that system.quantity and system.distance worked best as the entity for the slots height and weight. With system.any we got all sorts of weird results, which was understandable.

Something else that also helps I think is if you use contexts. Define a context for your intent, remove global and add a custom, more specific context, and within your flow switch to that context where you want to detect that intent and its slots. It helped us avoid false slot detections.

2 Likes

We are trying to build a bot for development and testing operations. The use case that we are implementing right now is to check the status of the last test run given the git branch name. Now this branch name could be any alphanumeric string. Some examples of these branch names are - spring-upgrade, OF-49759, jackson_bug_fix_module, di_rety_OF-39579 etc.
I don’t see any of the existing system entities matching this case.

If @system.any entity seems like a problem, what would be a better suggestion for this?

If you can’t find matching system entities or don’t find any relevant pattern and can’t define a finite list of things that this slot can be then any is the way to go. any type slots rely a lot on the content of the slot itself and the surrounding words so try to give it as many training examples as you can think with different variation in the nearby wording and different branch name length and patterns.

That being said, it seems like your branch names can indeed be any alphanum string but in your examples, seems like there’s always multiple group of chars joined either by a dash (-) or low dash (_). Try to build a pattern entity (regex) and see if that helps.

1 Like

Hi, it has been some time. A question regarding the above and contexts:

If I remove “global” and add a custom context to my intent definition like this:
image

Then this intent (which I called “weight” in this case) should only be detected by a node in the dialogue flow if the context has been set to “bmi” using appendContext. Is that not a correct asumption?

For some reason my “weight” intent is gettign detected even when context is “global” or something else, and not “bmi”.

1 Like

Also interesting that the entity type assigned to a slot does not seem to make any difference.

For example, for my “weight” intent (pictured above) I have a slot called “weight” (selected and shown with the blue highlights) that has a custom entity “weight” assigned to it. (Sorry for identical names, hope not too confusing…)

Now, if the entity is pattern based and changed to something silly such as \d{30,} (=at least 30 digits), my slot skill in the dialogue flow still says the utterance is of intent type “weight”, even though I am saying “I am 4 years old” - where there is clearly no match for the “at least 30 digits” entity.

Ultimately what we are trying to achieve is to have an “age” intent and a “weight” intent and be able to differentiate if the user is tellign us their weight or their age. Both can have similar semantics: “I am 81 kilograms” and “I am 81 years old”. So we tried to help the NLU engine by using contexts appropriately (see above), and/or by defining very specific entitites for the slots containing the age or the weight. But either way the NLU engine seems to just be going with one of the two intents, and never recognizing the other.

Is there any special trick to usign context? Is the slot / intent mechanism using that to filter amongst defined intents when matching a user’s utterance to an intent?

Thanks for any thoughts from anyone who ran into similar.

1 Like

Yes, you’re right and this is a known issue, it has been fixed in the upcoming patch 12.2.2. Thank you for sharing this.

Furthemore, for your information, the slots currently do not impact the intent “election process” but they will once we have “data augmentation” sorted out properly. Basically at the moment the intent election process is completed before the slot extraction meaning that assigning a very specific entity to the slot expecting it to help is a correct assumption but is not taken in to account.

Release is coming in the next couple of hours.

1 Like

Cool, much appreciated, thank you for your response!

Botpress continues to grow and imporve! :+1: :slightly_smiling_face:

2 Likes