You should only be careful because he doesn't know what goes on in your mind unless you tell him. So when you said he "asked if you still had feelings for him" did you ever? And why does he seem to feel as if you did? It's likely he wants more and he's trying for more, but he only knows as much as you tell him, and I'm assuming that isn't very much. Consequently, you could be leading him on. A lot. If you only want to be friends and if you don't like physical contact/aren't ready for it/whatever, you need to tell him.. and so long as you've been honest about how you feel it's up to him to decide if he can withhold feelings and be friends. With that said, if you aren't sure what you like/don't like it isn't fair to both of you to keep him strung along as anything other than a platonic friend.
_________________
If Jesus died for my sins, then I should sin as much as possible, so he didn't die for nothing.