He looks a little perplexed by that, but it only lasts a few seconds before he's standing up and clicking his tongue. "Geez, you're so annoying! I won't make you take down those castles, so just stay there for a second!"

This annoyance is so over the top that it's clear he's hiding something with it. It's a little similar to how he acts with Gon. Raising a hand, he waves off Hisoka, stepping away. "I'll be right back!"

He's going to go buy some cards and other things himself.