I am practicing chords and scales according to the chord progression and I find it difficult to make the transfer from technical exercises to creative improvisation.
Is there any method or idea that could help me make the shift?
I will be happy and grateful even with very small steps at the beginning.

Try writing out a solo, taking the time to make sure it has all the elements of a good jazz solo (structure, note choice, rhythmic interest, etc., etc.)

Also try playing the melody and using it as a spring board for improvisation. Does the tune have any interesting harmonic, melodic, or rhythmic devices that you could use in a improvised solo?

Listen to a recording of someone playing the tune and transcribe some of his/her musical ideas that you really like.

It sounds to me that you mainly need to listen to more jazz. Perhaps you are having problems because you lack a basic jazz vocabulary. Also, be patient. John Coltrane didn`t record a Giant Steps without decades of practicing and listening behind him.

My ideas...

Listen to more Jazz:

Miles Davis

John Coltrane

Benny G.

Duke Ellington

Herp Albert (w00t, just for the fun of it!)

Oh, and Chris Botti, he`s pretty good.

Try writing out a simple, simple solo consisting of mostly whole notes (O), play it over and over, but each time add a bit of your own twists in-between the whole or halfs notes, whatever you chose.