To say is to utter, to pronounce. You can say something even when you're alone, lost in the Sahara desert. You can say, "Help!" That's why when you want to communicate (not just pronounce) you should use "to say to." 

•  What did you say to him? / What did you tell him?
• Say us the time. (l'interlocuteur: "Us the time."
• Tell us the time. (l'interlocuteur: "It's nine o'clock p.m.")