[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [xmlblaster] ssl and compression with SOCKET
this is a useful extension to our basic SOCKET implementation.
If you are willing to donate it under LGPL license we would be
happy to add it to the xmlBlaster distribution.
If so, you can send the patch directly to my mail address.
You would need to add a test case as well and some documentation in
the SOCKET requirement (xmlBlaster/doc/requirements/protocol.socket.xml).
Is there a compatible C compression library with a free license around
which could be added to the C/C++ client library?
Do you have a property for a minimum message size to switch on
Yes, there is a compatible C compression library. I mentioned that jzlib
was used as the java library. As it is based on (actually, quite
copy-paste work) the GNU zlib C library, the libs and their
inputs/outputs have to be fully compatible. It's also stated on their
Let me explain the implementation details:
- Currently, compression is turned on if and only if SSL sockets are
used. However, these features can be separated easily.
- The _whole_ tcp stream is compressed, because it was the easiest to
implement it this way. This means that the "compression window" is not
reinitialized on every message; The dictionary isn't flushed in the
deflater, so repeating sequences are compressed even if they were in a
previous message. I thought that would enable the deflater to compress
the beginning of the message, too, and helps to achive better results
when the same type of small messages are sent frequently (in my case,
that is very true).
What I don't know is how all this behaves when random messages are sent
on a connection. I mean, what if virtually every message would have some
ugly binary content, for example? I still believe this would not make
the messages bigger on average.
A deeper understanding (and more modification) of the SOCKET
implementation would be required if I wanted to compress individual
messages. This is not impossible, either. Moreover, if I put the
compression filter in the right place, we would have compression for
every protocol, wouldn't we?
Thinking about this, I already have another implementation which, in a
way, supports compressing individual messages: it only compresses a
block of bytes when the stream is flushed or when its buffer is full.
When xmlBlaster sends a message, it writes the whole message to the
OutputStream, then flushes it. When flush is invoked, the buffer state
would enable it to decide whether to compress the current buffer (which
should be a message, or a big part of it) or not. This way, we would
lose the state of the deflater object and start with a new one on every
flush, compress the buffer, and see if it's smaller than the original
buffer. If it is, we'd send it compressed, otherwise not. Some simple
protocol wrapper is also needed, but that is no problem.
I think these are our options.
I will assemble a patch for you as soon as I sepatated the SSL and
compression layer and cleaned up some code. If you like the idea of
deciding compression on a per-flush basis, then that will be included, too.