To edit your public site menu, go to: Settings and admin > site settings > menus. The menu you want to edit is the "main menu".  

You can also add menu links on the pages themselves - it's at the bottom, just above the save button. This is often easier than creating main menu links manually.

When adding menu items, please note that you will always want to create your content first before creating the menu item.