Re: [xmlblaster] Newbie: xmlblaster and large messages/recovery of partial transfers

[Marcel Ruff <ruff at swand.lake.de>]

Darrell Berry wrote:

>Hi
>
>Just been pointed at xmblaster. I'm working on a distributed delivery
>mechanism for large files (>100Mb, probably most around 500Mb, some up
>to 2 or 3 Gb)
>
>Xmlblaster seems to offer a lot of what I need -- straightforward
>wrapping of content and metadata, infrastructure for both broadcast and
>p2p communications). However two questions:
>
>- Is xmlblaster robust and efficient for messages of this size?
>
No, you should  drive some tests yourself to find out.
Here is a command line based test, start the three processes in three
different consoles.
(set CLASSPATH first)

java org.xmlBlaster.Main

java org.xmlBlaster.client.reader.SubscribeMessage  -oid  bigtest

java org.xmlBlaster.client.feeder.PublishFile -c  MyBigFile.txt
-xmlKey  "<key oid='bigtest'/>"  -xmlQos "<qos/>"


Provide one of your big files and rename it to MyBigFile.txt before.

>- I see that you have a mechanism for making messages persist in the
>face of the system crashing or presumably one end of a connection going
>going> SO far so good -- however for messages of this size I would want
>to be able to resume transfer from the point at which contact broke off
>--if I've tranferred 90% of a 1Gb message and the receiver gets pulled
>from the network, when it resumes I would want to only have to download
>the remaining 10% -- have you any thoughts on this?
>
>Thx -- looks interesting!
>
>

I believe having a 3 GB file in RAM will be a problem. The messages is
sent in one piece.

You could add a little java client which divides the file into 10 MB
pieces, and the receiver
concatenates them again.
(Hmm, this would be a nice new feature in the xmlBlaster client lib -
eventually in future -)

keep us informed about your tests ...

Marcel