If either the front tires are more worn that the rear tires, or vice versa, the small difference in diameter can increase wear and possible destroy the center diff. It’s even worse with performance tires because they wear faster, therefore, more room for error. It’s the same concept as when you wanted to run different width wheels with tires that had almost the same diameter.
Will it break right away? Probably not, but it’s just added wear and tear that is easily avoided.