Stable Diffusion offers a way to generate image from text using AI or machine learning. The SDK is available only in Python at the time of this writing.

To use Stable Diffusion API to generate image from text, all you’ll need to do is first install the following python package using pip.

pip3 install stability-sdk

1. Modify the Stable Diffusion example code and turn it into a reusable function

In our example here, we will need to include the very basic function found on the the example code base on Stable Diffusion. We put it into a Python async function and call it gen_image() in our module funcs.py so it can be reused anywhere, as follows:

from stability_sdk import client
import stability_sdk.interfaces.gooseai.generation.generation_pb2 as generation
from PIL import Image
import warnings
import os 
import io
from dotenv import load_dotenv

load_dotenv()

async def gen_image( prompt, seed = 1002992 ) :
    os.environ['STABILITY_HOST'] = 'grpc.stability.ai:443'
    os.environ['STABILITY_KEY'] = os.getenv('STABLE_DIFF_API_KEY')

    stability_api = client.StabilityInference(
        key=os.environ['STABILITY_KEY'], # API Key reference.
        verbose=True, # Print debug messages.
        engine="stable-diffusion-v1-5", # Set the engine to use for generation. 
        # Available engines: stable-diffusion-v1 stable-diffusion-v1-5 stable-diffusion-512-v2-0 stable-diffusion-768-v2-0 
        # stable-diffusion-512-v2-1 stable-diffusion-768-v2-1 stable-inpainting-v1-0 stable-inpainting-512-v2-0
    )

    answers = stability_api.generate(
        prompt= prompt, 
        seed= seed , # If a seed is provided, the resulting generated image will be deterministic.
                        # What this means is that as long as all generation parameters remain the same, you can always recall the same image simply by generating it again.
                        # Note: This isn't quite the case for Clip Guided generations, which we'll tackle in a future example notebook.
        steps=30, # Amount of inference steps performed on image generation. Defaults to 30. 
        cfg_scale=8.0, # Influences how strongly your generation is guided to match your prompt.
                    # Setting this value higher increases the strength in which it tries to match your prompt.
                    # Defaults to 7.0 if not specified.
        width=512, # Generation width, defaults to 512 if not included.
        height=512, # Generation height, defaults to 512 if not included.
        samples=1, # Number of images to generate, defaults to 1 if not included.
        sampler=generation.SAMPLER_K_DPMPP_2M # Choose which sampler we want to denoise our generation with.
                                                    # Defaults to k_dpmpp_2m if not specified. Clip Guidance only supports ancestral samplers.
                                                    # (Available Samplers: ddim, plms, k_euler, k_euler_ancestral, k_heun, k_dpm_2, k_dpm_2_ancestral, k_dpmpp_2s_ancestral, k_lms, k_dpmpp_2m)
    )


    for resp in answers:
        for artifact in resp.artifacts:
            if artifact.finish_reason == generation.FILTER:
                warnings.warn(
                    "Your request activated the API's safety filters and could not be processed."
                    "Please modify the prompt and try again.")
            if artifact.type == generation.ARTIFACT_IMAGE:
                img_bytes = io.BytesIO(artifact.binary)
                return img_bytes

Please note the above gen_image() has been modified based on the Stable Diffusion example code base, which now has two parameters, which are the text that needs to be provided to generate the image and the seed when provided allowing the generated image to be deterministic i.e. the same image will be generated again when the same text and seed are provided. And it returns an in-memory binary stream of the generated image.

And please note that, we have our API key stored in an .env file, so you’ll need to import load_dotenv and call this function at the beginning of funcs.py. And the API key is obtained by using os.getenv(‘STABLE_DIFF_API_KEY’) as shown in line 9. Storing API KEY on .env file is the recommended way, so you can have .gitignore to exclude your API KEY from publishing on a GitHub repo.

2. Build an REST API by using Fast API.

In order to have our iOS app to call Stable Diffusion to have the image generated by the user’s input text, we’ll need to have our iOS client to call a REST API that will use the gen_image() function to generate the image by the user’s input text and returns the image to the iOS client for display.

In Python, the easier way is to use Fast API. First of all, for a quick guide, to install Fast API, just do as follows:

pip3 install fastapi

And install uvicorn:

pip3 install "uvicorn[standard]"

So, we quickly code up a main.py which has one end point only as follows, and put in the src folder of our project, please also put the above funcs.py file into the src folder as well:

from fastapi import FastAPI
from src.funcs import gen_image
from fastapi import Response

app = FastAPI()

@app.get("/get_image/{text}/{seed}")
async def get_image(text : str, seed : int):
    img =  await gen_image(text,seed)
    return Response(content=img.getvalue(), media_type="image/png")

So in the above code, we need to import the gen_image() function from the “funcs” module. And the REST API has only one end-point /get_image/{text}/{seed}, which the client just needs to provide the text and the seed to generate the image.

Before we proceed to test our REST API, make sure to get an API key from Stable Diffusion’s DreamStudio and create an .env file in the root folder of your FAST API project, which contains the API key that you’ve obtained from DreamStudio, for example as follows:

STABLE_DIFF_API_KEY=sk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Let’s test the REST API to generate some images by text. You can start your REST API now at your console with uvicorn with the following command, such as make it listen to port 8500. Please note our code is in the src folder of the project root, therefore, you’ll need to specify src.main:app with unvicorn.

uvicorn src.main:app --reload --port 8500

So, to test our REST API, we can simply open the browser to provide it a text and a seed e.g. “Dinosaur with Rainbow” and one of my results is as follows:

3. Now let’s build our iOS app with SwiftUI

In our tutorial, we are going to build a simple iOS app which presents a ContentView that provides a TextEditor and TextField for the user to input some text and the seed respectively and to be constructed as an image url to send to the REST API.

And the image is displayed by using AsyncImage, that has been available since iOS 15.

The ContentView

The code for the ContentView is as follows, which basically has a VStack that vertically stacks the imageView(), the TextEditor for inputing the text, the TextField for inputing the seed and a button when tapped, will construct an image URL set to the @State imageUrl variable to be consumed by the AsyncImageView.

We also have a @State loading variable which is set to true when the “Generate Image” button is tapped. When it is set to true, the Button will be disabled to prevent the user from tapping the Button again. And this @State loading variable is passed to the AsyncImageView as a Binding variable, which will be set to false upon the completion of the AsyncImage loading, thus re-enabling the Button.

struct ContentView: View {
    
    @State private var text : String = ""
     
    @State private var seed : String = ""
    
    @State private var loading : Bool = false 

    @State private var imageUrl : URL?
    
    var body: some View {
        
          VStack(alignment: .leading, spacing:10) {
            imageView()
            
            VStack(alignment: .leading,  spacing:2) {
                Text("Your text here:")
                TextEditor(text: $text)
                .lineLimit(3)
                .frame(height:50)
            }
            
            VStack(alignment: .leading,  spacing:2) {
                Text("Seed :")
                TextField("Seed", text: $seed)
                .keyboardType(.numberPad)
            }
            
            Button(action: {
                genImage()
            }){
                Text("Generate Image")
            }
            .disabled(self.processing)

            Spacer()
        }
        .padding()
    }
}

The imageView() is a function that has a HStack which aligns the AsyncImageView at the center with Spacer at both right and left.

extension ContentView {
    
    private func imageView () -> some View {
        HStack {
            Spacer()
             AsyncImageView(imageUrl: imageUrl, loading: $loading)
            .frame(width:250, height:250)
            Spacer()
        }     
    }
}

We build an AsyncImageView as follows, which can be reused when needed. The AsyncImageView is provided a @Binding loading variable that can be set to false when the AsyncImage has completed loading and an optional imageUrl property.

struct AsyncImageView : View {
    var imageUrl : URL?
    @Binding var loading: Bool
   
    var body: some View {
        imageView()
    }
}

extension AsyncImageView {
    
    @ViewBuilder
    private func imageView () -> some View {
        
        if let imgUrl = self.imageUrl {
       
            AsyncImage(url:imgUrl ) { phase in
             
                switch phase {
                    case .empty:
                        ProgressView()
                    case .success(let image):
                        image.resizable()
                        .aspectRatio(contentMode: .fit)
                        .onAppear{
                            self.loading = false
                        }
                    case .failure(let err):
                        VStack(spacing:4) {
                            Image(systemName: "exclamationmark.circle.fill")
                            .foregroundColor(.red)
                            Text("Error : \(err.localizedDescription)")
                        }
                        .onAppear{
                            self.loading = false
                        }
                    @unknown default:
                        Text("Unknown state!")
                        .foregroundColor(.blue)
                        .onAppear{
                            self.loading = false
                        }
                }
            }
        }
        else {
            
            Image(systemName: "photo.fill")
            .resizable()
            .aspectRatio(contentMode: .fill)
            .opacity(0.3)
           
        }
    }
   
}

The AsyncImageView has a @ViewBuilder function, imageView() which returns a static SwiftUI Image view only if the URL provided is nil or invalid, just to fill up the space else it’ll return the SwiftUI AsyncImage. The AsyncImage is created by the 3rd initializer which provides a content closure of phase value (AsyncImagePhase) at different point.

A switch statement is used to return different SwiftUI View at different state of phase, such as a ProgressView is returned when its state is .empty, an error view when it’s on error and also the SwiftUI Image view with the loaded image on success. Please note that, we set the @Binding loading variable passed by the parent view to false on each state when the View has successfully appears except the .empty while it’s still loading.

Finally the genImage() function, which generally sets the loading variable to true and set the image URL by concatenating with the text and seed is as follows:

extension ContentView {
    private func genImage(){
        
        if (text != "" && seed != ""){
            self.imageUrl = nil
            DispatchQueue.main.asyncAfter(deadline: .now() + 0.5) {
           
                if let imgUrl = URL(string: "http://localhost:8500/get_image/\(text.urlEncoded ?? "")/\(seed.urlEncoded ?? "")") {
                    self.loading = true
                    self.imageUrl = imgUrl                    
                }
            }
        }
    }
}

Please also note that the above text and seed must be url encoded, which can be done by a String extension as follows:

private let urlAllowed: CharacterSet =
    .alphanumerics.union(.init(charactersIn: "-._~")) // as per RFC 3986

extension String {
    var urlEncoded: String? {
        return addingPercentEncoding(withAllowedCharacters:urlAllowed )
    }
}

The result is as follows, when run on an iOS simulator. (Click the image below to view the animated GIF on how it works.)

It’s often that I need to convert videos of screen recordings to GIFs, while authoring this blog. Therefore, I’d like to explore how to convert video to gif with Python. Python comes with lots of libraries. To convert video to gif, the MoviePy simply comes in handy, which you can have it done in just few lines of code.

To get started with MoviePy, first of all, you’ll need to install it as follows:

python -m pip install moviepy

To convert a video to gif basically, you just need to import MoviePy and create an instance of MoviePy and supply it with the path to your video clip and call the method write_gif() and provide it a name of the output file:

from moviepy.editor import VideoFileClip
videoclip = VideoFileClip("Path to your video clip")
videoclip.write_gif("output.gif")

The above code will run as follows:

By simply doing the above, it’ll produce a gif that is sometimes too big for the web. In order to reduce the size, you can do the followings such as :

videoclip = VideoFileClip("Path-to-video-clip")
videoclip = videoclip.resize(0.8).subclip(1.0, 1.5)

videoclip.write_gif("output.gif",fps=10)

Let’s build a simple command line tool for video to gif conversion

It’s a good idea to build a simple command line tool for video to gif conversion, so it’ll come in handy when needed.

The simple command line tool will be supplied with various optional arguments for its frame rate, start and end times in seconds and the fraction of its size to reduce to etc. And it shall be run as follows:

python videotogif_cml.py [video_path] [fps] [time_start] [time_end] [resize_to] [output_path]

We need Sys module to read the arguments from the command line

from moviepy.editor import VideoFileClip
import sys

if len(sys.argv) < 2 :
    print("Usage : python videotogif_cml.py [video_path] [fps] [time_start] [time_end] [resize_to] [ouput_path]")
    sys.exit()
else:
    print("Start converting...")
    videoclip = VideoFileClip( sys.argv[1])
    # the following code will handle the optional arguments
    # and convert video to gif accordingly here...  

The above code checks if the user has supplied at least two arguments, will display a usage instruction and exit if not.

Please note that argument zero is the python script itself, argument one is the video path and from two onwards are only the arguments for the options of conversion accordingly.

The rest of the "else" block continues as follows which checks if each optional argument is provided and is given a default value if it's not. And finally, call the write_gif() method for the conversion.

fps = 10
if len(sys.argv) > 2:
   fps = int(sys.argv[2])

if len(sys.argv) > 4:
   time_start = float(sys.argv[3])
   time_end = float(sys.argv[4])
   videoclip = videoclip.subclip(time_start,time_end)		
elif len(sys.argv) > 3:
   time_start = float(sys.argv[3])
   videoclip = videoclip.subclip(time_start) # if time_end is not provided, 
                                             # it is assumed to be the duration of the clip
if len(sys.argv) > 5:
   videoclip = videoclip.resize(float(sys.argv[5]))
		
output_path = "output.gif"
if len(sys.argv) > 6:
   output_path = sys.argv[6]

videoclip.write_gif(output_path, fps=fps)

The complete source code can be found on GitHub

Thread lets different parts of program run concurrently. Usually when you have a part of execution in your program that takes longer than usual to run, it’s better let it run in a thread without blocking the main program to handle the user’s interactions or other executions etc.

Thread in Python does not mean multi-processing, it still runs on a single processor. Functions run in different threads are simply taking turns to run.

import threading
import time
def my_func1():
   counter = 0 
   while counter < 100 :
      counter += 1
      print("counter is :"+str(counter))
      time.sleep(1)
   print("bye bye now")

def my_func2():
	txt = input() 
	while txt != 'x' :
		print("You've entered :" + txt +"\n")
		txt = input()
	print("bye, you've entered x")

In the above code, we have two functions my_func1 and my_func2.

Function my_func1 takes long time to execute, as it only exits when it's increased the counter to 100 and each time it'll take a one-second sleep; meaning you'll have to wait up to about 100 seconds then the following lines of instructions to be executed.

The function my_func2 keeps looping for the user's input at the command line and exits only when an "x" is entered.

thread1 = threading.Thread(target=my_func1)
thread1.start()

thread2 = threading.Thread(target=my_func2)
thread2.start()

In the above example we run the two functions in two separate threads, thus my_func1 and my_func2 will run concurrently, the output is as follows:

If my_func1 and my_func2 are not run in threads, then the program has to wait for my_func1 to finish first then only to run my_func2.

There are at least two reasons that you should develop GUI-based application with Python. GUI applications developed with Python are cross-platform, which you can run them on Mac, Linux and Windows and Python is easy to learn with much simpler syntax than any other programming languages.

Tkinter is the standard GUI library for Python and it is easy to use. Basically, to create a GUI application main window, just need a few lines as follows

from tkinter import Tk
window = Tk()
window.mainloop()

Here is an example of a simple application, which consists of a container, which inherits from the main container of tkinter librarry, Frame and has two buttons in the container, which the first button will show a message when it’s clicked and the second button will quit the application when it’s clicked. As shown in the image below:

from tkinter import Button, Frame, Tk, messagebox as mbox
from datetime import datetime
class MyGuiApp(Frame) :
	
	def __init__(self, window=None): 
		super().__init__(window)
		self.window = window
		self.pack(expand=True, fill='both')
		self.createButtons()

	def createButtons(self): 
		...
	def showMessageBox(self):
		... 

Details of the createButtons() and showMessageBox() methods are as follows:

The createButtons() method is which we layout the two buttons.

def createButtons(self): 
   self.actionButton = Button(self, text = 'Click Me To Show Message')
   self.actionButton["command"] = self.showMessageBox
   self.actionButton.pack()
   self.actionButton.place(x=50, y=50, width=200, height = 30)
		
   self.quitButton = Button(self, text='Quit', fg='red')
   self.quitButton["command"] = self.window.destroy
   self.quitButton.pack()
   self.quitButton.place(x=75, y=120, width=150, height = 30)

The first button, create an instance variable actionButton, set its text label as ‘Click Me To Show Message’ and command, which is what the method showMessageBox() will be called when it’s clicked. And then call the pack() to add it to the parent widget and then we use the place() method to place it in a specific position.

The same goes to the second button, quitButton, the little difference is its command set to self.window.destroy, which will quit the application when it’s clicked.

The showMessageBox() method basically displays the message and shows the current date and time

def showMessageBox(self):
   now = datetime.now()
   date = now.strftime("%d/%b/%Y")
   time = now.strftime("%H:%M:%S")
   mbox.showinfo("New Message!", "Hello World! Today's Date is "+ date + ", current time is "+ time)

Finally, we have a self-contained application Frame, we need to create a window and add the frame to it to show it.

window = Tk()
window.geometry('300x250') 
window.configure(bg='grey')
window.title('My First Python GUI App Example')
app = MyGuiApp(window=window)
window.mainloop()

The above lines of code, create a window object, set its size (300×250), background color (grey) and title and runs the event loop to listen for events such as button clicks.

Python has an awesome library known as Matplotlib, which you can use to create all sort of graphs and charts by simply a few lines of code.

The matplotlib does not come by default with your Python installation. If you don’t have it, you’ll need to install it by the following command

python -m pip install matplotlib

Here are a few quick example of charts that it can create for you.

1. Line chart

import matplotlib.pyplot as plt
plt.plot([1, 3, 6], [5, 8, 6.5])
plt.show()

2. Bar chart

import matplotlib.pyplot as plt
plt.bar(['Apple','Orange','Grape'], [3, 8, 10], color = '#FF9922')
plt.show()

3. Pie Chart

import matplotlib.pyplot as plt
plt.pie([3, 8, 10],  labels =  ['Apple','Orange','Grape'], colors = ["#334455", "#FFAA34", "#508932"])
plt.show()

4. Histogram – the kind of graph that shows frequency distributions

import matplotlib.pyplot as plt
plt.hist([76, 77, 75, 76, 75, 74, 76, 76, 78, 74, 76, 76, 74, 76, 74,
  77, 76, 76, 74, 75, 77, 75, 75, 76, 75, 77, 76, 74, 75, 75,
  74, 76, 77, 75, 73, 75, 77, 75, 75, 76, 74, 76, 76, 75, 76,
  75, 75, 75, 76, 74, 88,89, 76, 76, 76, 78,77,76,76, 76])
plt.show()

Line 2 – to create a histogram, simply call the method hist() and supply it with a parameter of an array of numbers. E.g, the above shows the distribution of scores of math test in a class of 60 students.

More info about Matplotlib can be found on here