Voldermort famously avoided Cornwall during both the first and second wizards wars. While he would never admit this, was he worried about the pixies? The Good Guys™️ clearly knew of Voldemort’s fear of pixies, that’s why Shell cottage was on the Cornish coast.

While accessing Harry’s memories, no doubt Voldemort saw Gilderoy Lockhart attempt to cast the spell only to have the Pixie disarm him. There’s also a non zero chance that pixies have their own form of love magic and not only would Voldemort lose his body again if he tried to defend himself from a pixie, he would also create another unintended horcrux in the pixie that defeats him.