c++ - zlib inflate stream and avail_in -

-

part of application i'm working on involves receiving compressed data stream in zlib (deflate) format, piece piece on socket. routine receive compressed data in chunks, , pass inflate more data becomes available. when inflate returns z_stream_end know full object has arrived.

a simplified version of basic c++ inflater function follows:

void inflater::inflate_next_chunk(void* chunk, std::size_t size) {    m_strm.avail_in = size;    m_strm.next_in = chunk;    m_strm.next_out = m_buffer;     int ret = inflate(&m_strm, z_no_flush);    /* ... check errors, etc. ... */ }

except strangely, every like... 40 or times, inflate fail z_data_error.

according zlib manual, z_data_error indicates "corrupt or incomplete" stream. obviously, there number of ways data getting corrupted in application way beyond scope of question - after tinkering around, realized call inflate return z_data_error if m_strm.avail_in not 0 before set size. in other words, seems inflate failing because there already data in stream before set avail_in.

but understanding every call inflate should empty input stream, meaning when call inflate again, shouldn't have worry if didn't finish last call. understanding correct here? or need check strm.avail_in see if there pending input?

also, why there ever pending input? why doesn't inflate consume available input with each call?

inflate() can return because has filled output buffer not consumed of input data. if happens need provide new output buffer , call inflate() again until m_strm.avail.in == 0.

the zlib manual has say...

the detailed semantics follows. inflate performs 1 or both of following actions:

decompress more input starting @ next_in , update next_in , avail_in accordingly. if not input can processed (because there not enough room in output buffer), next_in updated , processing resume @ point next call of inflate().

you appear assuming compressed input fit in output buffer space, that's not case...

my wrapper code looks this...

bool cdatainflator::inflate(    const byte * const pdatain,    dword &datainsize,    byte *pdataout,    dword &dataoutsize) {    if (pdatain)    {       if (m_stream.avail_in == 0)       {          m_stream.avail_in = datainsize;          m_stream.next_in = const_cast<byte * const>(pdatain);       }       else       {          throw cexception(             _t("cdatainflator::inflate()"),             _t("no space input data"));       }    }     m_stream.avail_out = dataoutsize;    m_stream.next_out = pdataout;     bool done = false;        {       int result = inflate(&m_stream, z_block);        if (result < 0)       {          throwonfailure(_t("cdatainflator::inflate()"), result);       }        done = (m_stream.avail_in == 0 ||               (dataoutsize != m_stream.avail_out &&               m_stream.avail_out != 0));    }    while (!done && m_stream.avail_out == dataoutsize);     datainsize = m_stream.avail_in;     dataoutsize = dataoutsize - m_stream.avail_out;     return done; }

note loop , fact caller relies on datainsize know when of current input data has been consumed. if output space filled caller calls again using inflate(0, 0, pnewbuffer, newbuffersize); provide more buffer space...


Comments

Post a Comment

Popular posts from this blog

node.js - Mongoose: Cast to ObjectId failed for value on newly created object after setting the value -

-
this question has answer here: what's mongoose error cast objectid failed value xxx @ path “_id”? 7 answers even though has been asked many times because entity has no valid objectid field. i came across situation objectid set value can casted objectid still error. the schema: var serviceschema = parammongoose.schema({ servicename:{ type: parammongoose.schema.types.objectid , ref: 'servicenames',required:true } }); pseudo code: servicefromdb=new service({servicename:'some name'}); servicefromdb.servicename='000000000000000000000001'; servicefromdb.save(function(paramerror,paramdata){ if(paramerror){ console.log('but but...',servicefromdb,paramerror) } }); the output of code is: but but... { servicename: 000000000000000000000001, _id: 55079a90286f49280364f78b } { [casterror: cast objectid faile...
Read more

gradle error "Cannot convert the provided notation to a File or URI" -

-
i have defined 2 system properties in gradle.properties: systemprop.builddir=build systemprop.cfgdir=build\\cfg and have following task defined in build.gradle: task clean(group:'clean',description:'clean project') << { ant.sequential { delete(dir: system.properties.builddir) mkdir(dir: system.properties.builddir) delete(dir: system.properties.cfgdir) mkdir(dir: system.properties.cfgdir) } } this generates following error: execution failed task ':clean'. > cannot convert provided notation file or uri: {dir=build}. following types/formats supported: - string or charsequence path, e.g 'src/main/java' or '/usr/include' - string or charsequence uri, e.g 'file:/usr/include' - file instance. - uri or url instance. but equivalent block of code not generate error , works expected: task clean(group:'clean',description:'clean project') << { ...
Read more

python - NameError: name 'subprocess' is not defined -

-
i'm trying make rng website: http://jamesdotcom.com/?p=417 . when run following code: captureimage = subprocess.popen( [ "fswebcam", "-r", "356x292", "-d", "/dev/video0", "static.jpg", "--skip", "10" ], stdout=devnull, stderr=devnull) captureimage.communicate() i following error message: nameerror: name 'subprocess' not defined what should do? thanks have imported first? import subprocess captureimage = subprocess.popen(["fswebcam", ...) subprocess included in python standard library since python 2.4. should not necessary install it.
Read more