How to make Paintings out of your images: Neural Style Transfer

Neural Style Transfer on one of my image, combining the style from “The Starry Night” by Vincent Van Gogh. The first image shows the original image, and the corresponding images show style transfer after additional 500 iterations.

Link to the notebook : https://www.kaggle.com/code/virajkadam/neuralstyletransfer-for-creating-paintings

What is Neural Style Transfer ?

* It is the technique of generating a Generated Image by blending the content of the Content Image (which is our images in this case), with the style of the Style Image (which will be some famous painting in this case).
* The Generated Image is similar in style to the style Image and similar in the content to the Content Image.
* These Images can be generated using style and content features obtained by passing the style and content images to a pre-trained Convolutional Neural Net.

What we will need for style transfer : 
1) Pre-trained CNN Model 
2) Content and Style Image 
3) We can then generate the stylized image.

How Style transfer works?

* Generated Image : Start with a  image with dimensions similar to our Style and Content Images.We then optimize this Generated Image.
* Loss Function: We define a Loss Function that helps guide our Generated Image to be similar to Content and Style Images. We define separate loss function for a style features and content features.
* Using a Pretrained - CNN , we extract the style features from the Style Image.
* Similarly, we extract the Content features from  the deeper layer of the CNN.
* We then compare the Generated Image with Style Features and Content Features (using Loss Function), and use that to guide optimization of  Generated Image.
* This process is repeated until a desired image is Achieved.

Results

Here are some of the results from my experiments. You can go notebook link and try style transfer on your images.

Style Transfer on a landscape image. The desired style was achieved in a few iterations.

Picture of a tree in spring, stylized using painting by Van Rossum.

Image Stylized using a painting by Van Rossum

References and Resources

1) Link to the Neural Style Transfer Notebook : https://www.kaggle.com/code/virajkadam/neuralstyletransfer-for-creating-paintings

2) Tensorflow Tutorial : https://www.tensorflow.org/tutorials/generative/style_transfer#:~:text=Neural%20style%20transfer%20is%20an%20optimization%20technique%20used,in%20the%20style%20of%20the%20style%20reference%20image