The normal distribution, also called the Gaussian distribution, is the commonest of the many probability distributions that describe the pattern of future probabilities of some value.
The normal distribution's bell shaped curve is also familiar. It is often used in financial economics, even though it is often a simplifying assumption rather than the most accurate description of the probabilities. This is because it is (comparatively) easy to manipulate mathematically to derive useful results.
The cumulative normal distribution is the area under the curve of the normal distribution up to a particular value. In mathematical terms, the integral of the normal distribution.
The nature of the cumulative standard normal distribution (as used in Black-Scholes) should now be self-explanatory.
Advantages of the normal distribution
The normal distribution is widely used partly because it does genuinely often occur. It is also often used even when it just a rough approximation because it is easy to handle. The normal distribution can be manipulated algebraically much more easily than alternatives, so it can be used to derive formulae. This means that it is possible to derive results that can easily be applied (although computers have made this less important). Academics also like to derive formulae (or “closed form solutions”) for their own sake.
Problems and limitations
A number of valuation and risk models assume that the future price of a security is normally distributed. This is clearly false as a normal distribution function has a positive value for any value of the future price, whereas the price of a security cannot fall below zero.
A particularly important weakness, in the context of risk models, is that real distributions are fat-tailed. Their extremes are more probable than those of the standard distribution, because of the risk of crashes and booms.
The normal distribution is easy to use and the assumption that prices are normally distributed is sufficiently accurate in many circumstances.
There are many more mathematical descriptions of the normal distribution easily available in text books and on the web, such as this.