When we have few values, we may have several labels with the same value,
which is meaningless. In such a case, we lower the number of labels to
prevent such an issue.