1000+ Single Page JPEG encoded TIFF takes lot of memory and throws an error.

Jun 5, 2014 at 5:18 AM

We are having an issue with 1000+ Jpeg Encoded Single Page TIFF images. Basically we are creating an MagickImage Collection object for each Jpeg Encoded Single Page TIFF in for loop. After importing couple of 200+ pages it used to throws an error as mentioned below. and also it takes around 1-2 GB of memory as well.

Error :
Magick: IO Error Writing Tag data 'TIFFWriteDirectoryTagData' @error/tiff.c/TIFFErrors/561
Here is the piece of code :
'Generate MagicKImage Object to remove jpeg header if exists.
Dim InBitMap As Bitmap = Nothing
Using images As New ImageMagick.MagickImageCollection(FilePath)
    InBitMap = images.ToBitmap()
End Using
Note : You can take any single page jpeg encoded tiff and make it 1000 copies and import it.

Kindly Suggest !!
Jun 5, 2014 at 10:39 AM
What do you mean by 'After importing couple of 200+ pages'? Your code suggest you only perform a single action for each image. And the code from your example cannot raise the error you are writing. This error will only happen when you write a tiff file.
Jun 5, 2014 at 11:12 AM
Edited Jun 6, 2014 at 10:46 AM
I am trying to import 1000+ Single page JPEG Encoded Tiff and save it to another location. So i will take each jpeg tiff, create MagickImageCollection Object and get the bitmap and based on the bit-depth applying the compression method and re-save it to memory stream and get the binary data for save it to another location. Have a look on below code snippet for more detail.
Dim InBitMap As Bitmap = Nothing
Using images As New ImageMagick.MagickImageCollection(FilePath)
    InBitMap = images.ToBitmap()
End Using                    

Dim m_Stream As New MemoryStream()

'Get the Encoder and its codec info
Dim CodecInfo As ImageCodecInfo = GetEncoderInfo("image/tiff")
Dim ImgEncoder As New Encoder(CodecInfo.Clsid)

'Encoder Parameters for Compresssion and to denote MultiFrame
Dim EncParms As New EncoderParameters(2)
If InBitMap.PixelFormat = PixelFormat.Format1bppIndexed Then
    EncParms.Param(0) = New EncoderParameter(Encoder.Compression, EncoderValue.CompressionCCITT4)
    EncParms.Param(0) = New EncoderParameter(Encoder.Compression, EncoderValue.CompressionLZW)
End If
EncParms.Param(1) = New EncoderParameter(Encoder.SaveFlag, EncoderValue.MultiFrame)

'Encoder Parameters for Adding different Frames.
Dim EncAddFrame As New EncoderParameters(1)
EncAddFrame.Param(0) = New EncoderParameter(Encoder.SaveFlag, EncoderValue.FrameDimensionPage)

'Encoder Parameters for Closing Frame.
Dim EncCloseFrame As New EncoderParameters(1)
EncCloseFrame.Param(0) = New EncoderParameter(Encoder.SaveFlag, EncoderValue.Flush)

Dim Dimension As FrameDimension = New FrameDimension(InBitMap.FrameDimensionsList(0))
If InBitMap.GetFrameCount(Dimension) > 1 Then
    SyncLock oLock
        For ctr As Integer = 0 To InBitMap.GetFrameCount(Dimension) - 1
            InBitMap.SelectActiveFrame(Dimension, ctr)
            If ctr = 0 Then
                InBitMap.Save(m_Stream, CodecInfo, EncParms)
            End If
    End SyncLock
    InBitMap.Save(m_Stream, CodecInfo, EncParms)
End If
m_retBinaryData = m_Stream.ToArray

After approx 200+ page get saved then it will throw an error while generating bitmap from ToBitmap(). Also, the same error has been observed if we write the images (I.e. MagickImageCollection) to MemoryStream and generating bitmap object from memory Stream.
Jun 5, 2014 at 1:57 PM
Edited Jun 5, 2014 at 7:28 PM
Can you add VB.NET after the '```' (Syntax Highlighting) and remove the tabs at the beginning? This will make it more readable for me.

And are you using the latest version of Magick.NET. The 'ToBitmap' method cannot create the error message you are reporting.
Jun 6, 2014 at 10:52 AM
Yes, i am using Latest version of MagickImage (i.e. Magick.NET- ToBitmap() returns the error like i described in my initial thread. Also, i have tried writing the images collection to Memory stream and then generate the bitmap, in that case also the same error occurs.

Here is the sample code which i have tried out as well along with rest of the code.
Dim MS As New MemoryStream()
Using images As New ImageMagick.MagickImageCollection(FilePath)
End Using
Dim InBitMap As New Bitmap(MS)
Kindly suggest !!
Jun 6, 2014 at 3:08 PM
I can run the following code without any problems. Are you sure your issue is not related to a specific image?
using (MagickImage image = new MagickImage("logo:"))
  image.CompressionMethod = CompressionMethod.JPEG;

for (int i = 0; i < 10000; i++)
  using (MagickImageCollection images = new MagickImageCollection(@"C:\logo.tiff"))
    Bitmap test = images.ToBitmap();
Jun 9, 2014 at 11:07 AM
No, This issue is not specific to any image. normal Multipage tiffs are working fine. Jpeg Encoded Single Page tiff images are only creating problems.

We know that writing to file system is working fine. when you write to memory stream that is when the problem arises for large set of jpeg encoded single page tiff. For my case, it also happens when ToBitmap() method is called.

I have a error screenshot with me which i want to share with you. could you help me how can i upload directly in this forum not as a image link to other website.
Jun 9, 2014 at 11:14 AM
Does my code raise an exception on your machine?

You could use dropbox or contact me trough Codeplex so you can e-mail it to me directly.
Jun 9, 2014 at 1:28 PM
Let me try out with your code in a sample application.

here is the error screenshot image link :
Jun 9, 2014 at 3:14 PM
Your screenshot is not public.
Jun 10, 2014 at 5:15 AM
Jun 10, 2014 at 10:59 AM
Edited Jun 10, 2014 at 10:59 AM
Have you tried my code?
Jun 10, 2014 at 1:13 PM
Yes, i have tried out and your code works fine. The Problem lies with the byte array of each file which took memory. After implementing couple of changes in calling method i am now able to upload 1000 Single Jpeg Encoded Tiff images.

Thanks for your help during this. I really appreciate.