Re: [xmlblaster-devel] Clarification on socket protocol exception..

[Marcel Ruff <mr at marcelruff.info>]

Brad Clements wrote:
> I am working on a new pure Python implementation of the xmlBlaster socket 
> protocol
>
> I have a question about the encoding of exceptions
>
> On
>
>
> http://xmlblaster.org/xmlBlaster/doc/requirements/protocol.socket.html
>
> This example is given
>
> Example of an XmlBlasterException as a response on a publish() (a content is not shown):
>
>    "        84**E**17711*publish*oxf6hZs**QueueOverflow*The destination queue is full*0*"
>   

 qos      key    len   content
+-----*---------*-----*----------+

I think the "0*" trail is the 'len' == 0 which ends with '\0',
the content is nothing here.

> I am wondering about that last * the one after the 0.
>
> My reading of:
>
> errorCode{string}, message{string}, byteDump of exception
>
> makes me think that byteDump = "0*"
>
> that is, the trailing null is part  of the byteDump.
>
> Otherwise it serves no purpose, since
>
> a) the userData is entirely one single exception "block"
>
> b) userData length is determined from message length.
>
> So, is that last * in the example a mistake, or will there really be an extra \x00 at 
> the end of the byteDump?
>
>
> Also in flag, it says:
>
> 90 (typically 'Z') The userData is compressed with gzip (see jzlib and zlib). 
> Currently there is support to compress the whole socket stream - in such case 
> this flag makes no sense and is not set
>
> How is it that the entire socket stream is compressed? Where do I find the 
> specification for that?
>   
For zlib:stream:

No compression with those flags is implemented, the compression as 
described in
 
http://www.xmlblaster.org/xmlBlaster/doc/requirements/protocol.socket.html#compress
is done above the low level compression you mention.
The drawback with the current approach is that the same connection can 
not mix compressed/uncompressed messages
and does not auto-recognize if the message is compressed or not, the 
nice thing is that it is high efficient
and we could reuse it for other IO stream based protocols - and we have 
C/C++ and Java client implementations.

For C (or C++) see
 xmlBlaster/src/c/socket/xmlBlasterZlib.c
and for Java see
 xmlBlaster/src/java/org/xmlBlaster/util/protocol/Z*Stream.java
(four classes for in and out and for block and stream mode)
They use the (java.net) socket.getInputStream() and socket.getOutputStream()
and intercept the flow of data to be compressed (see 
SocketExecutor.java:76 ff)

The implementations are the spec in this case, anyhow, any more
textual spec is welcome.

> Does this mean I should not compress userData using zlib because it's not 
> supported?
>   
Yes, or you need to add the support server side.
> I am looking at the config docs on this same page and trying to figure out how
> compress/type = "zlib" can possibly work. That is, some messages are 
> compressed and some are not. How can I split messages out of a tcp stream 
> without some demarcation?
>   
I never use it (but it works fine), but it seems it works like this:
The  <i>zlib</i> compresses block wise and violates the
described SOCKET protocol format, it adds a prefix to the message format.
A first byte at the beginning, the <code>compressionFlag</code>, where 
'1' is zlib compressed and '0' is uncompressed.
This is followed by 4 bytes which contain the length of the uncompressed 
message.
If <code>compressionFlag==1</code>, there are another 4 bytes which 
contain the compressed length.
(see ZBlockOutputStream.java:85)
Note that everything is compressed, not only the userData.
> I guess only zlib:stream can really work on tcp, and I would have to know in 
> advance if I was "sending" everything zlib compressed, and if I was receiving 
> compressed data. I guess compression must be the same in both directions.
>   
Yes.
It should probably work with different settings in both directions, 
though i haven't tried it.
> how does compress/type = "zlib" work with compress/minSize = 500.. ? How can I 
> tell if I'm receiving a compressed message and how many bytes it's going to be 
> so I know when to stop blocking on a read?
>   
It works as described above.
> If 'A' adler32 checksum is not supported, why is it in the spec at all?
>   
Yes, this could/shall be removed, it was more some thought what could be 
done with this compression flag.

Marcel