This project has moved and is read-only. For the latest updates, please go here.

Resizing 5+mb gif files throws MemoryAllocationException

Jan 21, 2015 at 7:50 AM
Edited Jan 21, 2015 at 7:56 AM
Hi,

I've used Magick.NET to create Micro-service which makes resizing of different types of images.

And it works like a charms even with 20mb jpeg images!

But in case of resizing big .gif images it throws exception.
When I'm trying to resize about 4-5 gif animations concurrently I'm usually get MemoryAllocationException inside collection.ToByteArray(). Even when I'm trying to avoid duplication of collection as byte array, and trying to write resized image directly to filesystem I'm getting same exception in collection.Write() method.


Example image: https://www.dropbox.com/s/5muvrpocb8tf7q3/exampleGifImage.gif?dl=0

Here is my code:

Simplest implementation:
using (var collection = new MagickImageCollection(path))
{
    collection.Coalesce();
    foreach (MagickImage image in collection)
    {
        image.Resize(width, height);
    }
    return collection.ToByteArray();
}
Optimized version:
using (var collection = new MagickImageCollection(path))
{
    collection.Coalesce();
    foreach (MagickImage image in collection)
    {
        image.Resize(width, height);
    }

    collection.Optimize();
    collection.OptimizeTransparency();
    collection.Write(path);
}
Regards,
Dmitry Zaets.
Jan 21, 2015 at 12:14 PM
The size of the image is not really important it is the size in bits that an image takes. Your example file is not a good example because it is not that big but you can use the following calculation for the amount of memory that is necessary to store an image in memory:
width * height * (pixel channels) * (bits per pixel)
252 * 118 * 4 * 16 = 1903104 bits = 0.237888 mb

frames * (bits per frame)
255 * 1903104 = 485291520 bits = 60 mb
You probably have a gif that has a bigger width/height and has a lot of frames that is causing these problems.
Jan 21, 2015 at 1:19 PM
Edited Jan 21, 2015 at 1:22 PM
Thanks for quick reply!

I've tried exactly that gif from my example. (Actually it's around 7mb.)

I've just executed resizing about 10 time for same gif concurrently.
(Use case: Image upload form, opened in 10 windows, to simulate 10 users.)

As result - first 4-6 gif files was resized successfully, and all next got exception while writing to disk.


Regards,
Dmitry Zaets.
Jan 21, 2015 at 1:33 PM
The point I am trying to make is that there is a difference between the size on disk and the size in memory. When Magick.NET loads a GIF file the pixels will be translated to an internal memory format. For the example you gave me this will use 60mb.
Jan 21, 2015 at 2:28 PM
Yep, sure.
I'm understand that point.

And I see that there is no problems with free memory on my desktop or server, and it's far from limit of free RAM.

So my question is: may be I can make some special configurations to avoid occurrence of memory limit exception in Magick.NET? (Use more RAM or swap on disk, if all memory is used)

Also when ImageMagick (I mean native library) throws exception - it make no memory cleanup and it produce a memory leak.

Regards,
Dmitry Zaets.
Jan 21, 2015 at 3:18 PM
Edited Jan 21, 2015 at 3:29 PM
ImageMagick should use the disk when memory cannot be allocated but there might be a bug in the library. Can you create a small program that I can use to recreate your issue?

When an exception is raised the memory will be cleaned when the MagickImage is disposed. Are you sure the MemoryAllocationException is raised by ImageMagick because it is out of memory?
Jan 21, 2015 at 3:54 PM
Edited Jan 21, 2015 at 3:57 PM
Here is a link, sorry for a big size of archive (around 31mb)

https://www.dropbox.com/s/0bv3v4c84co3ovo/MagickImageGifResizerExample.zip?dl=0

M.b. you can find what exactly I'm doing wrong.
Jan 21, 2015 at 10:05 PM
Your code looks correct and I can reproduce the issue. Not sure why this happening, I will let you know when I know more.
Jan 22, 2015 at 11:01 PM
I think I know what is going on. This is some kind of corner case. Each image will take around 96mb (my previous calculation was incorrect 118 should be 188). When the image is written it takes twice that amount of memory to reduce the number of colors to 256 to fit in a gif color palette. Your example is for 10 images and this is really close to 2Gb which is the memory limit for a 32 bit process. The part that reduces the colors will not swap to disk when the memory limit is reached. Switching to x64 will probably solve your issue. Or you should limit the number of threads.
Jan 23, 2015 at 2:36 PM
Thanks for your answer!

M.b. you can suggest why does AnyCPU nuget package is using x86 library instead of x64 on both my desktop and server?
Should I use x64 package instead?

Can I somehow limit resources for MagicImage.dll so it will swap to disk or wait till another thread finishing writing to file? (I've already found how to limit memory for MagicImage application but haven't found if is there a possibility to do that through the wrapper)

Thanks for your time.

Regards,
Dmitry Zaets.
Jan 23, 2015 at 2:44 PM
You are probably running your application in a 32bit application pool. I did not crash on my machine when I used the x64 package.

You can use ResourceLimits to force your application to use the disk much faster. But this will probably decrease your performance. A better solution would be to limit the number of concurrent threads or 'upgrade' to x64.