Image Caching Server
Another project I worked on at E-Commerce Solutions was a product
search engine with data supplied from many partner websites. One
problematic aspect of the search engine was trying to display
product thumbnails on the results page.
Along with the product's information, each partner site also
sent us the URL to a full-size picture of the product. Image size
varied greatly from site to site, but we wanted to display small
thumbnails all formatted to similar dimensions. The first solution
was to set the
of the image tag, but unfortunately we did not know the original
dimensions ahead of time, so attempting to hardcode dimensions
resulted in horribly distorted pictures. The second solution used
errors if there was a problem downloading the image. Either solution
required the client to download the entire original picture before
it could scale it down, and some of these photos were full-screen.
My solution was to create an intermediate proxy server that would
download the images from the partner sites as they were requested
and then scale the pictures down on the fly before sending them
to the client browser. For efficiency, once an image was scaled,
it was saved in a cache on the server so future requests did not
require repeat trips to the partner servers.
I wrote the proof of concept in Python using the Python standard
library and the Python Imaging Library (PIL). The URL of a requested
image was sent to the proxy server which used Python's URL handling
libraries to pull the image from the partner site. The PIL library
loaded the image and extracted its dimensions. With this information,
the server was able to properly calculate the image's height and
width to fit the search results page. The PIL library scaled down
the image and saved a copy to disk, keeping a list of scaled image
in memory. The scaled image was then sent back to the client's
The next time the same image was requested, the server simply
redirected the client to the local copy sitting on the server's
hard drive. If the requested image could not be found on the partner's
site, then the client was redirected to a generic "Picture
Not Available" graphic.
The solution was dubbed the Image Caching Server, and a full
implentation was written by one of my coworkers in C as a custom
Apache module running on a Linux server.