you're reading...
tiling, wonthurt

JPG compresses better and uncompresses faster than ECW

The following comparison method has been applied to three lossless raster maps (urban orthophoto, rural orthophoto and a 8-bpp map):

  1. Compress the original image to JPG.
  2. Compress the original image to ECW using different compression levels until the resulting ECW file is slightly larger than the JPG file obtained in the previous step (this gives the ECW file a slight advantage).
  3. Uncompress both files to a format that imagemagick can read.
  4. Compare the uncompressed versions with the original image to quantify their lossiness.

A screenshot illustrating this process can be seen here. I have used GDAL and imagemagick for this. Lossiness is quantified by computing the PSNR value (the higher the better: very lossy images have a low PSNR). The result of this is that JPG files are smaller but less lossy than ECW files:

Image size (pixels) JPG size (Mbytes) ECW size (Mbytes) JPG PSNR ECW PSNR JPG wins?
5355 x 5349 5.36 5.67 37.7 35.5 YES
5351 x 5154 6.53 6.93 35.1 34.2 YES
4096 x 4096 5.08 5.45 27.6 27.4 YES

Faster

The fact that JPG uncompresses faster than ECW can clearly be seen with just one test. The uncompress time can be estimated with the time Linux command applied to gdal_translate:

From JPG to BMP: 18.4 seconds (16.1 seconds if we discount copy time)
From ECW to BMP: 27.1 seconds (24.8 seconds if we discount copy time)
Copy BMP to BMP: 2.3 seconds (estimated time needed to copy result to disk)

So what is good about ECW?

The ECW format allows fast access to application-defined zones of the raster with very low memory consumption. The typical scenario is that the application’s viewport needs to paint about one million pixels on the screen, while the original raster is a huge image with hundreds of millions or even billions of pixels. The JPG format needs to be fully expanded, so usually most of the resulting pixels will not be used.

How can we avoid uncompressing unneeded pixels?

We must use JPG images in a way that does not force us to uncompress pixels that are not going to be painted. One way to achieve this is by cutting up the original image in pieces that are small enough so that -after quickly selecting the needed ones- we can cover the application’s viewport without uncompressing a number of pixels that is significantly larger than the viewport size. In other words: use JPG tiles.

Advertisements

Discussion

12 thoughts on “JPG compresses better and uncompresses faster than ECW

  1. You should do the test using jpeg tiles. It will probably change the result because I think it’s more expensive to compress the borders of the images than the inside area.

    Posted by jacarma | 29/03/2010, 8:46 AM
    • Hello, thanks for the feedback. I don’t think 256 x 256 is small enough to cause any problems, and it fits very well the JPG algorithm blocks, doesn’t it? I’m convinced Google chose that tile size after checking it is a JPG-friendly size. Regards.

      Posted by gvsigmobileonopenmoko | 29/03/2010, 5:14 PM
  2. If you are going to use JPEG tiles, you should also have tiles for lower resolutions, and this usually gives you 40% extra data, so ECW still compresses better.

    Posted by Morten | 29/03/2010, 4:52 PM
    • Hello. In this post I’m not talking about pyramids or resampling down. The third image, for example, which comes from a 8-bpp map, should not be resampled down, because it contains ‘features’, such as place-names, roads and individual buildings with their specific symbology. These features would disappear with a resampling, which would make the map useless.

      I’m just comparing the compression quality with an implicit formula like:

      Compression quality = 1 / (Lossiness x Compression ratio)

      which I think is the most objective formula (excluding nonrealistic cases, such as Lossiness = 0 or Compression ratio ~= 0). Regards.

      Posted by gvsigmobileonopenmoko | 29/03/2010, 10:30 PM
    • Reduced resolution data levels (rrd) typically proceed from 2x down to a small tile (around 128 x 128 or 64 x 64) and will be 33.33% of the 0 level (full resolution) data when nothing is compressed. If all rrds are compressed to the same quality as the 0 level data, the rrds should add 33.33% to the file size. If the file footprints are small (say 1024 x 1024 and smaller), calculations on the fly are quick enough these days and rrds are not needed.

      Posted by Paul Beaty | 13/04/2010, 2:05 PM
  3. Were you testing the old ECW/JP2 CODEC SDK v3.3 or a higher version? Did you take into account the file size overhead the ECW file has because of the internal pyramid levels (zoom levels) of the ECW file? In effect, you have compressed the zero level data in the ECW file further than the zero level data in the JPEG file. Thus we should expect a little more loss in the ECW file. Would this size comparison be a better one between ECW and a Progressive JPEG?

    Posted by Paul Beaty | 30/03/2010, 2:05 PM
    • Hello. Please correct me if I’m wrong:

      On Google Maps, for example, each zoom level is independent. Level N replicates and amplifies the graphic information contained in lenel N-1, so there is a lot of redundancy. In other words, If you have level N, you can derive levels 0 to N-1 from it.

      As opposed to that, in the wavelet-based formats, each level amplifies (without redundancy) the graphic information contained in the previous level, so the overhead, as you call it, is not an optional thing: you can only uncompress the image totally after reading all the levels. This type of approach is very suitable when one needs to browse the image at different zoom levels.

      If you think the second paragraph is wrong, is it then possible to create a ECW file with only the deepest layer? GDAL does not allow this, as far as I know. I have used the version of ECW SDK that comes with GDAL 1.7.

      So this is how I see it: using ECW implies sacrificing a bit of compression quality (compared to JPG) for the sake of fast sub-image browsing and fast resampling.

      Regards.

      Posted by gvsigmobileonopenmoko | 30/03/2010, 5:11 PM
      • The point of the overhead discussion was to point out that simple file sized comparisons cannot be used to determine which has a better 0 level (full resolution) quality. No, the reduced resolution data (rrd) layers are not optional and you cannot create an ECW file with only 0 level data. Indeed, they are desired. ECW contains the 0 level data and multiple reduced resolutions as well.

        The developer is not forced to proceed through all the resolution layers to get to the 0 level data. They can target a specific resolution level, and go straight to that. Many users like to progress through the rrds. This technique called progressive rendering. Progressive rendering gives the user quick feedback with a quick grab of rrd image data for the geographic extent they are viewing. The user can immediately zoom or pan to refine the viewing area. Progressive rendering is actually slightly slower, but human test indicate many ‘feel’ progressive rendering is faster. Again, with ECW progressive rendering is an option.

        Yet I must say, you have some good points concerning the value of traditional JPEG as a delivery format. Many in the mapping community are not aware that ERDAS offers a JPEG tile delivery approach in addition to ECW released in 2009. ERDAS calls it Optimized Tile Delivery Format (OTDF) OTDF was codenames Wombat and was under development when ERDAS purchased ER Mapper.

        OTDF is very, very fast. Yet, OTDF is not as fast as ECW v3.6, and not nearly as fast as the soon to be released ECW v4.1. ECW v3.6 was never released to the market, but v4.1 should be available before the Northern Hemispherical Summer Solstice.

        Posted by Paul Beaty | 13/04/2010, 2:12 PM
  4. Hello. Thanks for all that info.

    I was unsure about ECW being redundant or not.

    Many people believe that the ECW format has supernatural powers and that JPG does not compare to it in any way. Hopefully this post has shed some light on the whole thing.

    Regards.

    Posted by gvsigmobileonopenmoko | 13/04/2010, 4:33 PM

Trackbacks/Pingbacks

  1. Pingback: Xbox 360 E71 Error Repair Tips | XBox 360 Fix Red - 28/03/2010

  2. Pingback: GIS-Lab Blog» Архив блога » Новости вокруг 52 - 22/04/2010

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: