While holding the nose while jumping into the water is normal for small children, most people learn to release the nose while underwater without getting water up the nose. If you are grown and still haven't learned to stop holding your nose, don't worry. You are not the only one. A few simple tricks can help you break the habit.
Start by simply blowing air out of your nose for the duration you are underwater. While this is a great trick to stop holding your nose, it should not be relied upon, as it greatly decreases the amount of time you can spend underwater. Once you get comfortable, you can use this trick just when you are jumping into the water or turning flips underwater.
Practice humming while you are underwater. While humming does not release a great amount of air out your nose, it prevents air from entering the sinuses. Once you become comfortable with the hum, you should be able to swim underwater without holding your nose.
Place your tongue on the top of your mouth and allow pressure to build in your sinuses, restricting water from entering. You can get a feel for the muscles needed for this trick by paying close attention to the muscles in your nose that work when you swallow water or food. You can practice holding the pressure against your sinus while out of the water and then take it underwater.
Use a nose clip as a last resort or while you are practicing hands-free underwater techniques so that you can overcome the fear of getting water up your nose.