Hi all,

I have a large amount of data written in Sleepycat JE which I now need
to migrate to BDB (non-JE) as the requirement has arisen to access the
data programmatically through different languages.

(it would have been a lot easier if this had been a requirement from
the start, but that's life).

I can use the dump and load scripts to get data from the .jdb files
into BDB files. However I am now having problems reading the
dumped/loaded BDB data using the C++ API. It doesn't help that this is
the first C++ I have used for 8 years, so I am quite rusty.

Here is what I am running into:

1. I know the key/data length of the original JE database. However, how
do these translate into C++? I can get how it works for things like
ints and doubles. However, one of the items in my JE keys are Strings.
What I do not understand is how these will be represented in BDB, so I
don't know how to get them back.

2. Endianness. From the brief investigation that we've done, we can see
that stuff written in Java BDB (not JE) can be read using the Java BDB
API, and stuff written in C++ BDB can be read using C++.

However, stuff written using Java BDB cannot be read using C++ BDB, and
we suspect that we have got bitten by endianness, but don't know how to
tell. It seems odd that Java and C++ on the same platform (Windows XP)
should write differently, but that's all we can think of for why things
are breaking.

The keys and values written in these cases are ints and doubles.

If anyone else has tried something similar, or run into similar
problems, I'd love to hear about it.

Thanks,

Petra

Hi Petra,

I hope this will answer your questions -- it's from
java/src/som/sleepycat/bind/TupleOutput.java, and explains the encoding
of Java types:

* Signed numbers are stored in the buffer in MSB (most significant
byte
* first) order with their sign bit (high-order bit) inverted to cause
negative
* numbers to be sorted first when comparing values as unsigned byte
arrays,
* as done in a database. Unsigned numbers, including characters, are
stored
* in MSB order with no change to their sign bit.
*
* Strings and character arrays are stored either as a fixed length
array of
* unicode characters, where the length must be known by the
application, or as
* a null-terminated UTF byte array.
*
* Null strings are UTF encoded as { 0xFF }, which is not allowed in a
* standard UTF encoding. This allows null strings, as distinct from
empty or
* zero length strings, to be represented in a tuple. Using the
default
* comparator, null strings will be ordered last.
*
* Zero (0x0000) character values are UTF encoded as non-zero values,
and
* therefore embedded zeros in the string are supported. The sequence
{ 0xC0,
* 0x80 } is used to encode a zero character. This UTF encoding is the
same
* one used by native Java UTF libraries. However, this encoding of
zero does
* impact the lexicographical ordering, and zeros will not be sorted
first (the
* natural order) or last. For all character values other than zero,
the
* default UTF byte ordering is the same as the Unicode lexicographical
* character ordering.
*
* Floats and doubles are stored in standard Java integer-bit
representation
* (IEEE 754). Non-negative numbers are correctly ordered by numeric
value.
* However, negative numbers are not correctly ordered; therefore, if
you use
* negative floating point numbers in a key, you'll need to implement
and
* configure a custom comparator to get correct numeric ordering.

Regards,
Michael.

Thanks, Michael. That gives me a lot to go by.

Much appreciated,

Petra