View source for Python usage notes - PIL

{{Pythonstuff}}
{{stub}}


==Importing==

While I habitually always used 
<code lang="python">
import Image, ImageChops
</code>


It's better to use:

<code lang="python">
from PIL import Image, ImageChops
</code>

because that'll work with Pillow (this is apparently by design){{verify}}


===PIL and Pillow=== 

'''tl;dr:''' If PIL continues to not be very actively developed, then we all want to move to Pillow now (= install pillow instead of PIL).


Pillow is an interface-compatible drop-in fork of PIL.

The motivation:
* PIL wasn't very actively developed
: (and therefore wan't very py3k)
* PIL was apparently sort of nasty to package

Pillow does its own development, and tried to stay in sync with PIL, although since PIL sees little development, it ''has'' started to drift. {{verify}}



See also:
* http://blog.aclark.net/2013/03/15/the-story-of-pillow/

* https://pillow.readthedocs.io/en/3.3.x/

==Pixels and speed==
{{stub}}

An image behaves like a two-dimensional mapping - or a one-dimensional one.
You want fast array/pixel access, and you want loops to be fast.

<!--
Note that having data in C and doing python-side calculations means a lot of copying back and forth. Ideally, all operations are done on the C side, which is probably only practical with libraries like numpy and scipy.
-->


Your options include ('''roughly from fast to slow'''):
* if you like the idea of [[Halide]], there's a python interface
: takes a little getting used to, though.

* PIL supports the array interface, meaning you can use numpy, scipy, and often get their C-like speed.
: I would suggest this for ''all'' nontrivial calculation
: details vary, see notes below

* get/put arrays - which works for a few simpler operations,
: such as per-pixel lookup tabling, e.g. <tt>out.putdata([lut[x] for x in im.getdata()])</tt>
: treats the image as a flattened, one-dimensional iterable (of color tuples according to whatever mode/bands it has)
: putdata also lets you scale (multiply) and offset (add) arguments
: it seems that psyco may help

* assignment/fetches via <tt>im[x,y]</tt>

* <tt>getpixel((x,y))</tt> and <tt>putpixel((x,y),v)</tt>



Note that [http://imageio.github.io ImageIO] reads a lot of formats, with means that if you focus on Halide or numpy/scipy, you can skip PIL. 

This can make sense if your data is varied data, rather than necessarily 3-channel uint8 photos. PIL ''does'' supports varied formats but sometimes makes you work for it.

==Loading / converting ==

Note: This is biased to converting RGB images.   (Grayscale may be simpler, )

Other-colorspace images and alpha channels can make things more involved, and sometimes be very hard.


====Read from various typical image files====
 Image.open(filename)

====Read compressed file data from memory====
e.g. if you have fetched it from the network, memcache, etc.
 Image.open( StringIO.StringIO( pngdata ) )


====Read from uncompressed raw pixel data====
When knowing its size and type, e.g.
 Image.frombuffer("RGB",(640,480), rawpixels)

For more control: the next parameter is the decoder (e.g. "raw"), and any beyond are parameters to that decoder

====Numpy, scipy====

Assuming you have a PIL after 1.1.6, you can use the fact that it supports the array interface (before that it was more involved).


You can deal with arbitrary numyp arrays. 
There is scipy's 
* [http://docs.scipy.org/doc/scipy-0.14.0/reference/ndimage.html ndimage] is, as a type, a little more tuned to images{{verify}} (where array is completely generic)
* scipy's skimage, a.k.a. scikit-image[http://scikit-image.org/docs/dev/api/api.html] has some very useful things




'''numpy to PIL'''

you'ld often control your dimensionality and type/bit depth on the numpy side {{comment|(though can also do the latter by converting it to a known type first)}}, e.g. 
 Image.fromarray( ary )
 Image.fromarray( ary.astype(numpy.uint8) )


'''PIL to numpy'''
 ary = numpy.array( im ) # You can also convert while loading by specifying a dtype.

And you can get a no-copy (read-only, same-type) view like:
 ary = numpy.asarray( im )







: '''On array versus asarray'''
<code lang="python">
imar = numpy.array(im)   # writable copy
# or, if the first step isn't in-place, you can avoid a copy by doing:
#imar = numpy.asarray(im) # readonly view 

# [do fancy stuff here]

# as things are likely to become float along the way, you often want to 
#  choose an output type to work towards (and scale/clamp values if necessary)
im  = Image.fromarray(  imar.astype(numpy.uint8)  )
</code>


Notes:
* on {{inlinecode|asarray()}} versus {{inlinecode|array()}}:
** array() ''always'' constructs something new{{verify}} based on the input
** asarray is part of the array protocol{{verify}}, meant as a 'guarantee we now see it as a numpy array'
*** If the object handed to it is a subclass of ndarray (e.g. which since 1.1.6 include PIL image objects{{verify}}), then a read-only, non-copied view is returned
*** If the object only presents itself as python tuples, lists, or nestings of such, a new array is returned
*** TODO: check about a.flags.writeable = True  -- possibly copy-on-write?


Some tests I did to verify I understood the array/assaray difference:
<code lang="python">
>>> im = Image.fromstring('L',(2,2), '\x50\x00\xff\x12','raw')
>>> aa = numpy.asarray(im)
>>> aa
array([[ 80,   0],
      [255,  18]], dtype=uint8)
>>> aa[0][0]=130
RuntimeError: array is not writeable

>>> a2 = numpy.array(im)
>>> a2[0][0]=130
>>> a2
array([[130,   0],
      [255,  18]], dtype=uint8)
>>>
>>> im2=Image.fromarray(a2)
>>> im2
<Image.Image image mode=L size=2x2 at 0x83DCFCC>
>>> im2.getpixel((0,0))
130
</code>
<!--
>>>
>>>
>>> l=[1,2]
>>> la=numpy.asarray(l)
>>> la
array([1, 2])
>>> l[0]=9
>>> l
[9, 2]
>>> la
array([1, 2])
>>> la.dtype
dtype('int32')
-->

See also:
* http://effbot.org/zone/pil-pixel-access.htm


<!--
: '''on size and channels'''
Keep in mind numpy axes are effectively row,column while you probably think about images as width*height 
{{comment|(in other words, y,x  and x,y respectively)}}, which can get confusing if you mix PIL code with numpy code.

The color channels are the third axis.

So example, a <tt>679x496</tt> RGB image will have a <tt>.shape</tt> of <tt>(496, 679, 3)</tt>



Keep in mind that the third axis is not spatially related (the first two are), which matters to various filters.
If you want to e.g. do a <tt>guassian_blur</tt>, you'll probably want to say sigma=(2,2,0) to not blur along RGB (which would give you a nearly-grayscale image).



Numpy lets you transpose <tt>transpose()</tt>, which essentially lets you change the order in which you look at the axes.

For example, if you want to go from numpy's <tt>(y,x,channel)</tt> to <tt>(channel,x,y)</tt>:
<code lang="python">
imar = imar.transpose(2, 0, 1)   # or 1,0,2 to just swap rows and columns

imar[0] = np.maximum( nimg[0]-nimg[0].min(), 0) 
imar[1] = np.maximum( nimg[1]-nimg[1].min(), 0)
imar[2] = np.maximum( nimg[2]-nimg[2].min(), 0)

# (and eventually put it back with another transpose)
imar = imar.transpose(1, 2, 0)
</code>

Although in this particular case, just doing the indexing would be shorter and not confusing:
<code lang="python">
imar[::0] = np.maximum( nimg[::0]-nimg[::0].min(), 0) 
imar[::1] = np.maximum( nimg[::1]-nimg[::1].min(), 0)
imar[::2] = np.maximum( nimg[::2]-nimg[::2].min(), 0)
</code>


If for some reason you have flattened the image into a 1D array along the way,
you can ask numpy to view it as 2D again (fast as it doesn't change data).
For example, consider something like:
<code lang="python">
sat_ary = numpy.zeros( im.size[0] * im.size[1], dtype=numpy.uint8 ) # monochrome-style image
i=0
for r,g,b in im.getdata(): # this PIL function flattens the image into an 1D array
    h,l,s = colorsys.rgb_to_hls(r/255., g/255., b/255.) # h,l,s are in 0..1
    sat_ary[i]=255.*s 
    i+=1
sat_ary.resize( (height,width) ) # make numpy view it as 2D again (we remembered its size)
</code>


-->

====OpenCV====
{{stub}}

OpenCV has matrix type and image types:
* Image formats are based on:
** IPL_DEPTH_8U - 8-bit unsigned int, grayscale
** IPL_DEPTH_8S - 8-bit signed int, grayscale
** IPL_DEPTH_16U - 8-bit unsigned int, grayscale
** IPL_DEPTH_16S - 8-bit signed int, grayscale
** IPL_DEPTH_32S - 8-bit signed int, grayscale
** IPL_DEPTH_32F - 32-bit float
** IPL_DEPTH_64F - 64-bit float

* Multi-channel storage are implied by calls like:
 cv.CreateImage((320,200), cv.IPL_DEPTH_8U, 3) // three 8-bit unsigned int channels
 # Note that  cv calls will less flexible ideas

* Matrix (Mat) data types are named like:
 CV_<bit_depth>(S|U|F)C<number_of_channels>
 e.g. CV_8UC3


OpenCV to PIL:
 # assuming a 3-channel IPL_DEPTH_8U image that you store RGB into:
 pi = Image.fromstring("RGB", cv.GetSize(cv_im), cv_im.tostring())
 # OpenCV seems to like BGR, so you may wish to first have done:
 cvtColor(image, cv2.COLOR_RGB2BGR)


* RGB
* BGR
* also has HLS, HSV, CIE Lab, CIE Luv, Bayer
* CvtColor does color space conversions
* mixChannels allows some reordering
* To/from compressed file: 
 pyopencv.imread("file.png") and pyopencv.imwrite("file.png") (or in cv: cv.LoadImage() and cv.SaveImage())

* OpenCV to PIL:
 Image.fromstring("RGB", cv.GetSize(cimg), cimg.tostring())

* PIL to OpenCV:
 cimg2 = cv.CreateImageHeader(pimg.size, cv.IPL_DEPTH_8U, 3)      # cimg2 is a OpenCV image 
 cv.SetData(cimg2, pimg.tostring())

http://www.comp.nus.edu.sg/~cs4243/conversion.html


====Pygame====
{{stub}}

Pygame has various pixel formats, pick one that suits your needs, probably RGB or RGBA.

Pygame:
* P - 8bpp, palettized
* RGB - 24bpp image
* RGBX - 32bpp, unused channel
* RGBA - 32bpp
* ARGB - 32bpp
* RGBA_PREMULT - 
* ARGB_PREMULT - 
* pygame.image.fromstring(data, size, format, flipped=False), pygame.image.tostring(surface, size, format)

http://pygame-users.25799.x6.nabble.com/BGRA-to-RGB-RGBA-fastest-way-to-do-so-td163.html



====Cairo to PIL====
{{stub}}

If you don't need alpha, then it seems Cairo's RGB24 and PIL's RGBA are compatible {{comment|(<tt>RGBX</tt>, but compressors may not like that without first doing a <tt>convert('RGB')</tt>)}}

If you need alpha, you need some in-memory reordering. 


* Cairo to PIL (no alpha)
 surface = cairo.ImageSurface(cairo.FORMAT_RGB24, w,h)
 im = Image.frombuffer("RGBX", size, surface.get_data(), "raw", "RGBX", 0,1).convert("RGB")

* PIL to Cairo (no alpha)
 TODO


Cairo pixel formats[http://www.cairographics.org/manual/cairo-Image-Surfaces.html#cairo-format-t]:
* CAIRO_FORMAT_ARGB32 - pre-applied alpha
* CAIRO_FORMAT_RGB24  - 32 bits per pixel, 8 per channel, upper 8 bits unused)
* CAIRO_FORMAT_A8 - 8-bit alpha
* CAIRO_FORMAT_A1 - 1-bit alpha, packed.
* CAIRO_FORMAT_RGB16_565
* CAIRO_FORMAT_RGB30 - RGB, 10 bits per channel

PIL pixel formats:
* RGB  24bits per pixel, 8-bit-per-channel RGB)
* RGBA (8-bit-per-channel RGBA)
* RGBa (8-bit-per-channel RGBA, remultiplied alpha)
* RGBX (8-bit-per-channel, padded), considered an internal format, compressors may not like it
* 1 - 1bpp, often for masks
* L - 8bpp, grayscale
* P - 8bpp, paletted
* I - 32-bit integers, grayscale
* F - 32-bit floats, grayscale
* CMYK - 8 bits per channel, 4 channels
* YCbCr - 8 bits per channel, 3 channels
-->


====Agg to PIL====
{{stub}}


You can ask Agg's figure/canvas for RGB pixel data, which you can send to [[PIL]]. 

You could then save into a (c)StringIO (probably using a compressed format) if you want to bypass disk entirely (e.g. in web serving).


See an example at [[Python usage notes - Matplotlib, pylab#Agg]]

==Semi-sorted==

===On paletted images ("P" mode)===

Paletted image data are uint8, with pixel values actually being indexes into the palette.

''Internally'', they are almost identical to "L"-mode images {{comment|(code easily makes the mistake of using what are palette indices as grayscale values, which rarely makes any sense)}}


The palette (see <tt>im.getpalette()</tt>) is basically a lookup table for colors, 
for RGB meaning a 768-byte array (R<sub>0</sub>,G<sub>0</sub>,B<sub>0</sub>, R<sub>1</sub>,G<sub>1</sub>,B<sub>1</sub>, ...)

You can change the palette with {{inlinecode|im.putpalette(palette)}}
* palette should be a list of 768 entries, but is zero-padded if a shorter list is handed to putpalette
* for "P" mode images: changes the palette.
* for "L" mode images: converts to "P" mode image, using this palette (could be useful for false coloring)


* Paletted images are also a memory-compact way of using ImageDraw


Conversion:
* .convert()ing it to RGB simply converts it through the palette

* .convert()ing it to L seems to just chuck the palette {{verify}}
: which usually makes no sense
: except e.g. when you know a single color to make a mask from, e.g. the transparent color in a GIF:
<code lang="python">
def nontransparency_mask(pci):
    if pci==transparency_index:
        return 255
     else:
        return 0
mask = transparent_gif_image.convert('L').point(nontrans_mask) 
# ...which is an "L" mode mask storing a mask that could be stored in a "1" mode image.
</code>



===ImageFont IOError: cannot open resource===

Means file not found for the specific font you requested.


When you don't specify a directory, it'll look in a few places (system fonts, current directory) -- I should figure out which and how exactly.

Arguably the best-defined way is to package a font with your code and give it the absolute path to that.

==See also==
* [http://www.pythonware.com/library/pil/handbook/index.htm Python Imaging Library handbook]



==Some experiments==

===PNG transparency===
<code lang="python">
''' Create a transparent PNG from a non-transparent image.

    In the code below, white becomes is transparent and black opaque.
    Transparency is currently linear with the luminance, which is rarely what you want.

    You would probably want to create a LUT (e.g. via function to im.point) with much more biased responses.
'''
import sys,Image,ImageChops

try:
    im = Image.open(sys.argv[1])
    (r,g,b) = im.split()
    a = im.convert("L")
    a = ImageChops.invert(a) #comment this out if you want black transparent, white opaque
    im2 = Image.merge("RGBA",(r,g,b,a))
    im2.save(sys.argv[1]+"-trans.png",None)
except:
    print "Failed. Possibly you supplied no filename, or it's not a valid image."
</code>

===Animated GIFs===

For some notes on GIF structure, see [[Image_notes#GIF]]

====Detecting====

Doesn't have an existing function, but it's easy enough:

<code lang="python">
def is_animated(im, returntostart=True):
    ''' Returns whether this image has more than one frame.
        Since GIF is often incrementally drawn, this reader can only seek to the next frame and rewind to 0.
        Since the test is seeing if we can seek, we return to the start afterwards.
    '''
    try:
        im.seek(0) # redundant in almost all cases. Why would you have gone to other frames if you didn't know it was animated?
        im.seek(1)
        if returntostart:
            im.seek(0)
        return True
    except EOFError:
        return False
</code>

====Reading====
{{stub}}

<code lang="python">
# you can't tell the number of frames before reading,
# but you can break when it's done (basically what ImageSequence does too)
while 1:
    frame = 0  # start at the start
    try:
       im.seek(frame) # The image will now be at this position
                      # and look like a grayscale or pallette image (L or P mode)
       print "\nFrame %d"%frame

       # There is also metadata, both global and per-frame (somewhat mixed)
       print im.info # new keys:       version      (global, "GIF87a" or "GIF89a")
                     #  if applicable: background   (global, palette colour index, only present when there is a global color table)
                     #                 duration     (per-frame. PIL uses milliseconds, not centiseconds as the GIF stores)
                     #                 transparency (per-frame, palette colour index)
       primt im.tile # used by decoding, can be useful to inspect only the region that was updated.

       frame += 1 # for next loop
    except EOFError:
        break
</code>


PIL's GIF reader consists of python code reading the structure of a GIF file, while LZW decoding and (uncompressed) writing is done in C code (the <tt>im.tile</tt> data is used to tell it where to look). 
The code in seek() loads the next image frame, has it decoded and copied into the overall image (done by shared code in ImageFile).

It looks like PIL does not yet correctly handle all variants of incremental rendering {{verify}}


<tt>im.tile</tt> looks like <tt>('gif', (30, 55, 280, 383), 95045L, (8, False))</tt>
The contents are <tt>(decodertag , (left,top,width,height), file_offset, extra)</tt>,
where extra is extra information for the decoder, in this case <tt>(bits,interlaced)</tt>

====Writing====
{{stub}}

PIL itself won't write animated GIFs, or compressed GIFs,

Animation:
* There is <tt>gifwriter.py</tt> for PIL and Pillow - and which was apparently merged into pillow at some point{{verify}}
* There is also the more independent <tt>images2gif.py</tt>, and lets you control a few more things more easily, e.g. durations per frame.
* uncompressed GIFs are quickly megabytes large, though...

Compression:
* There's a line you can change in <tt>GifImagePlugin.py</tt> that makes it call <tt>ppmtogif</tt> to write LZW-compressed GIFs into a file, but it's not very helpful for ''animated'' GIFs.


Animation + compression:
* While there has been some movement, the best solution (in terms of file size and such) still seems to use some external program 
: either to hand it the uncompressed gif, or e.g. hand it pngs and how to compose it, depending on which program you actually use.


(You can also try bugging me. I've got a mostly-finished pure-python LZW-compressed halfway-color-clever animated-GIF writer somewhere)



[[Category:Programming]]
[[Category:Audio, video, images]]