If your truly lost, you have two options, to stay put so someone has a chance of finding you, or try to find your way out. Obvious, if your lost, unless you know there is something in a specific direction, it isnt going to matter what direction you walk in, you just pick the most common sense direction and walk.
Obvious you dont want to keep walking in a circle, and if you actually want to survive, you DONT use most of the methods people have pointed to in this thread. I would never suggest someone use a stick and try to go by the shadow or to try to go by the sun or stars. Those things are just silly when you have a more powerful tool to use, your brain.
The most important thing to use is land features. Around here we have these things around here called mountains and streams. Mountain ranges and streams run in one direction, so if your in the mountains, and dont know what way to go, follow the mountain ridges for a better view, but if you come across a creek, you have a food source, so choose carefully what one you pick.
While your walking, look for land features to walk to, or ones that might work to your advantage, and always keep stopping and turning around to keep a eye on were your walking from.
Use any method you can find to mark were you have been, and make darn sure that the distance between markings are close enough to be findable. This way, if while walking you lose sight of the feature your walking to, or lose your sense of direction, you can backtrack some to get your bearings again.
Use common sense of were houses, roads, and food sources are going to be. Unless your lost at sea, following the stars at night isn't a very good idea!