Are there any downsides to using these?

So I’ve noticed that at times during dialogue you’re able to get boosts to your rolls from other members of your party if they’re proficient in a field that would help your response.

I’ve noticed that these aren’t added by default to boost your chances of a success, so it had me wondering if there was any down side to them that I should be thinking about before using them or if it’s just not auto selected for players that want to do a harder run?