[xmlblaster-devel] Client side persistence using SQLite

[Avinash Atreya <avinashatreya at yahoo.co.in>]

Hi,
 We am looking at the XmlBlaster(XB) C client, more specifically at a 
non-threaded client which offers client side persistence (to ensure that 
messages are not lost).

This is what we use to compile the C client:
gcc -Wall -g -I. -I/usr/local/include/ -L/usr/local/lib -lsqlite \
-UXB_USE_PTHREADS -DXMLBLASTER_PERSISTENT_QUEUE_TEST=1 \
-o $1_HelloWorld HelloWorld_send.c \
util/helper.c util/msgUtil.c util/Properties.c \
socket/xmlBlasterSocket.c socket/XmlBlasterConnectionUnparsed.c \
util/queue/SQLiteQueue.c

(Helloworld_send.c is a simple program that we have written to publish messages 
to the server)

Compiling with the above results in the errors:
socket/XmlBlasterConnectionUnparsed.c:348: structure has no member named `nodeId'
socket/XmlBlasterConnectionUnparsed.c:349: structure has no member named `nodeId'
socket/XmlBlasterConnectionUnparsed.c:365: structure has no member named 
`userObject'

We checked QueueInterface.h and the structure QueueProperties does not have any 
member named nodeId and XmlBlasterStruct no member named userObject. Commenting 
these out results in successful compilation for now as queueProperties->nodeId 
and queueProperties->userObject dont seem to be used at least in the 
XmlBlasterConnectionUnparsed.c. (We am not very familiar with the client side 
code yet and hence do not know about there usage elsewhere)

Also, by the flow there, looks like the ordering of messages is not preserved 
if a send fails. More specifically, if Msg#1 is persisted because the 
connection is lost, and the connection is restored by the time Msg#2 arrives, 
Msg#2 gets sent before Msg#1 though the client sent the latter before. Is it 
the expected behaviour? We are looking Message *Queues* functionality and given 
the same priority, FIFO needs to be guaranteed.

I couldnt find the code that takes messages out of the persistent storage and 
sends it to the server again. (It has to be in the same routine). I understand 
that this is still under test and you plan to add that functionality. Is that so?

The flow I am looking at is something like this:

begin Publish ( msg ...)
QoS = 0
pri = msg.priority
if connection not restored
  persist msg
  update QoS //saying msg queued
else
  if store is not empty
    foreach p in 0..9 do //assume 0 is highest
      msgArr = peek messages from the store with priority p
      if p equals pri
	push back msg into msgArr
      endif
      if senddata(msgArr.....) is successful
        remove all the peeked messages that were successfully sent
        update QoS //indicating all the messages sent
      else
      if p equals pri
        persist msg into store //we only peeked at the others, dint remove them
	update QoS //saying msg queue
      endif
      endif
    done
   else //store is empty
      if senddata (msg... ) is successful
        update QoS //saying msg sent
      else
        persist msg
        update QoS //saying msg queued
      endif
   endif
endif
return QoS
end

PublishArr will look something similar...

We would like to implement this unless we are duplicating any ongoing efforts. 
Please let us know. We will be very happy to receive any suggestions, 
information from your side.

Thanks in advance,
Avinash