The most important aspect of improvisation is creativity. This is the most vital concept for an improviser to understand. The goal is to hear something interesting in your head and be able to play it immediately. Your understanding of music fundamentals is one ally in this endeavor. It can help you interpret the sounds you hear in your head by relating them to sounds you know and understand. Your technical proficiency on your instrument is another ally. It can help you accurately execute what you conceive. Inspiration, however, is what enables you to hear interesting ideas to begin with. That creative spark is what distinguishes the true artist from the mere craftsman. While no primer can show you how to be creative, I can try to shed a little light on creativity as it pertains to improvisation.
Trumpet player Clark Terry summarizes the creative process as "imitate, assimilate, innovate". Listening to other musicians can give you ideas you may wish to develop further, and being able to successfully duplicate what they are doing is one step toward being able to express yourself. Next, you must understand why the things you are playing sound the way they do, so that when you want to create a particular sound, you will know how to achieve it. The theory presented in the following sections can help you structure your thoughts, and can also help you identify the sounds you hear. However, analytic processes are an aid to the creative process, not a replacement for it. Two analogies, one with language and one with mathematics, should help make this clear.
When you began to speak, you learned at first by listening to others and imitating them. Gradually, you became aware of grammar, and eventually the grammar was codified for you in English classes. Your vocabulary has probably been growing ever since you spoke your first word. In both writing and conversation, your tools are your knowledge of grammar, vocabulary, and appropriate subject matter. To write or say anything interesting, however, you must have a certain amount of inspiration. It is not sufficient to merely string together grammatically correct phrases of words. What you have to say is generally more important than how you say it, although proper use of the language can help to get your point across. Similarly, in music, knowledge of theory and fundamentals are the tools of composition and improvisation, but inspiration plays the most important role in determining your success. It is not enough to merely play the "right" notes; you must also play interesting music. Jazz improvisation is often likened to "telling a story", and, like a good story, should be well structured and also convey something interesting to the listener.
In mathematics, creativity can often be crucial as well. Learning the various axioms, formulas, and equations normally does not tell you how to solve a particular word problem, integrate a certain function, or prove a new theorem. Some ingenuity is required to be able to apply your knowledge to the problem at hand. Often, knowing how similar problems have been solved in the past can give you an idea of where to start, and experience working with a particular type of problem can help direct you. In all but the simplest of math problems, however, some original thinking is required. Similarly, in jazz, your familiarity with the works of other musicians can help you get started, and your knowledge of theory can help direct you, but in order to be a successful improviser, you will need to be creative. Just as long columns of numbers are not particularly interesting, even if they add up correctly, neither is an improvisation that consists of nothing but scales and patterns based on those scales.
Your listening experience, your knowledge of music theory, and experimentation on your instrument will define the musical context in which you are able to express yourself. You should continually strive to expand that context by listening to many different musicians, analyzing what you hear, and practicing as much as possible. Still, the final ingredient, the inspiration, you will have to find on your own.
You should by now, if you have not already, be starting to improvise. You should start the same way you began to practice swing: alone and unaccompanied at first, with a tape recorder if possible, and then with some sort of rhythm section accompaniment. Again, Band-In-A-Box, Aebersold records, or do-it-yourself accompaniment will be invaluable.
For your first attempts at improvisation, pick a key with which you are comfortable and then start to play whatever comes into your head. Invent little melodies that use mainly notes from the selected scale. Do not try to fill all available space with notes. Instead, concentrate on hearing a short phrase in your head, and then try to play that phrase. Do not worry if this means there are breaks of several seconds or more between phrases. Miles Davis used this style of phrasing all the time.
At some point while improvising in a given key, try playing notes that are not in that key. Playing notes that are not in the current key is sometimes called playing outside. You will find that in many cases, it sounds very natural, while in other cases, it sounds dissonant, or harsh. The later sections on theory may help you understand why this is so, but your ear is the ultimate judge. When you finally run out of ideas in one key, you may wish to switch to another. You may also wish to try improvising without any key center at all. I believe this should be just as natural as improvising within a key.
Transcribing solos played by other musicians is one way to get some ideas of what to play. You can examine the structure of the solo, see how they use the various chord/scale relationships discussed later in this primer, and try to apply what you learn to your own playing. One of the best solos for a beginner to study is Miles Davis' solo on "So What" from the album Kind Of Blue. The chord structure is simple: sixteen bars of D minor, followed by 8 bars of Eb minor, and then 8 bars of D minor again. Miles' lines are easy enough to transcribe note for note. The theory sections below will help you understand the framework in which Miles was working, but transcribing his solo will help you see what he was doing within that framework.
Another way to get ideas for soloing is by using patterns, or short phrases that you have practiced beforehand and know will fit the chord changes at a particular point. In general, improvising is much more than simply stringing together patterns, but pattern practicing can be a good way to develop your technique as well as your ear, particularly if you practice your patterns in all twelve keys. There are several books, including Jerry Coker's Patterns For Jazz, that give some useful patterns.
A technique used often in the bebop era and since is quoting, or using a recognizable phrase from another composition or well-known recorded improvisation as part of one's own improvisation. This is also sometimes called interpolation. You may have noticed this taking place in solos you have heard. There is usually some humor value in quoting, particularly if the interpolated work is something silly like "Pop Goes The Weasel".
The most important obstacles for a beginning improviser to overcome are his or her own inhibitions. At first, when practicing improvisation by yourself, you may feel you have no idea what to play. Once you have reached the point where you feel comfortable in the practice room and decide it is time to play with other musicians, you may feel self-conscious about playing in front of your peers. Finally, when you can play with other musicians in private, you may feel nervous when you first perform in public. I have no miracle cures for these problems. I can only suggest you play as much as possible at each stage, and continually push yourself to take chances.