A researcher that I know organizes a symposium within a strange conference and has asked me to go. I do not know how to answer

I have been invited to a symposium related with my research field (Statistics) within a conference devoted to general Applied Mathematics. This symposium is organized by a researcher that I met in previous, well-established and respectable conferences.

The problem is that I have never heard before about that conference to any colleague in my field, if we exclude the emails sent by the conference organizers asking researchers to attend or to organize a symposium within the conference (I have received myself a lot of them in my email spam folder). Indeed, I am really confused about the reputability of the conference itself, and I tried to find it out without too much success in this closed question.

I have decided not to attend the conference given the information that I have at hand, since as a young researcher I want to stay as far as I can from potential predatory conferences. Also the funding money for attending conferences is limited and I think I will get a better value for it on another conference.

Now, my question is regarding the way and etiquette for rejecting the invitation. My initial idea was to be honest and tell the symposium organizer about my reason to not accept the invitation (not feeling confortable in that conference), without making up any excuse such as problem in the dates or whatever. But maybe in that way I will hurt his/her feelings, something that obviously I do not want.

What will be a honest and polite way of rejecting this invitation? Anyone has experienced the same situation? Besides, secondary comments and feedback related with the whole issue are more than welcomed!