How much will a U affect my grad school chances?

I have received a U as a final grade in a single credit course that is part of an honors college program during my first semester in college.

Will this affect my chances of getting into graduate school.

I'm studying Computer Science (with a double major in Maths) at a public university. I hope to attend a private school like Vanderbilt to pursue an MS or PhD.

I also had one terrible (D) score in a 200 level maths course during the same semester.

My GPA is still above a 3.