dbTalk Databases Forums  

New Cluster Interconnects

comp.databases.rdb comp.databases.rdb


Discuss New Cluster Interconnects in the comp.databases.rdb forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Richard Maher
 
Posts: n/a

Default New Cluster Interconnects - 05-21-2005 , 08:50 AM






Hi,

I was flicking through the OpenVMS Roadmap presentation yesterday and came
across a couple of very interesting (at least to me) milestones regarding
Cluster Interconnects. Now, I'm pretty useless with hardware, "the laws of
physics capt'n", and not much better as a System Manager so I hope someone
can offer me a lay-man's view of what these developments could mean for VMS
Cluster performance. (In particular the VMS Lock Manager.)

1) I saw 10 Gigabit NIC support scheduled for VMS 8.3 (Depending on which
slide you look at it says "Integrity Servers Only"). Now 10x what a lot of
people are using for a cluster-interconnect at the moment sounds pretty
shit-hot to me! Especially if you're moving big lock-trees around the
cluster. Given that this functionality is less than a year away, surely some
performance figures or at least anecdotal evidence should be available? I
mean, if I was an Rdb engineer that had used piss-poor DLM performance as
the rationale for sticking all of my R&D eggs in the stand-alone single-node
basket, then I'd be interested in what's happening with this. Right? "But
it's not the bandwidth, it's the latency that gets ya." Well that brings me
to the next slide. . .

2) Next Generation Low-Latency Interconnects Post 8.3 (Integrity Servers
Only)

Am I the only person getting their jollies out of this or what?

I forget when Oracle10g was scheduled to arrive but I'd dearly love to hear
from anyone using Cache Fusion and is looking at this!

Will there be a special limit on the distances between nodes for this stuff
to work? (Like memory channel) Can you have a Disaster Tolerant Low Latency
Cluster?

Regards Richard (Just off to have a cold shower) Maher



Reply With Quote
  #2  
Old   
Bill
 
Posts: n/a

Default Re: New Cluster Interconnects - 05-22-2005 , 05:24 AM






Low latency is where the performance appears to be, gig ethernet, and maybe
10g ethernet have too much latency as you scale
"Richard Maher" <maher_rj (AT) hotspamnotmail (DOT) com> wrote

Quote:
Hi,

I was flicking through the OpenVMS Roadmap presentation yesterday and came
across a couple of very interesting (at least to me) milestones regarding
Cluster Interconnects. Now, I'm pretty useless with hardware, "the laws of
physics capt'n", and not much better as a System Manager so I hope someone
can offer me a lay-man's view of what these developments could mean for
VMS
Cluster performance. (In particular the VMS Lock Manager.)

1) I saw 10 Gigabit NIC support scheduled for VMS 8.3 (Depending on which
slide you look at it says "Integrity Servers Only"). Now 10x what a lot of
people are using for a cluster-interconnect at the moment sounds pretty
shit-hot to me! Especially if you're moving big lock-trees around the
cluster. Given that this functionality is less than a year away, surely
some
performance figures or at least anecdotal evidence should be available? I
mean, if I was an Rdb engineer that had used piss-poor DLM performance as
the rationale for sticking all of my R&D eggs in the stand-alone
single-node
basket, then I'd be interested in what's happening with this. Right? "But
it's not the bandwidth, it's the latency that gets ya." Well that brings
me
to the next slide. . .

2) Next Generation Low-Latency Interconnects Post 8.3 (Integrity Servers
Only)

Am I the only person getting their jollies out of this or what?

I forget when Oracle10g was scheduled to arrive but I'd dearly love to
hear
from anyone using Cache Fusion and is looking at this!

Will there be a special limit on the distances between nodes for this
stuff
to work? (Like memory channel) Can you have a Disaster Tolerant Low
Latency
Cluster?

Regards Richard (Just off to have a cold shower) Maher





Reply With Quote
  #3  
Old   
Rich Jordan
 
Posts: n/a

Default Re: New Cluster Interconnects - 05-22-2005 , 06:54 PM



Richard Maher wrote:
Quote:
Hi,

I was flicking through the OpenVMS Roadmap presentation yesterday and came
across a couple of very interesting (at least to me) milestones regarding
Cluster Interconnects. Now, I'm pretty useless with hardware, "the laws of
physics capt'n", and not much better as a System Manager so I hope someone
can offer me a lay-man's view of what these developments could mean for VMS
Cluster performance. (In particular the VMS Lock Manager.)

1) I saw 10 Gigabit NIC support scheduled for VMS 8.3 (Depending on which
slide you look at it says "Integrity Servers Only"). Now 10x what a lot of
people are using for a cluster-interconnect at the moment sounds pretty
shit-hot to me! .
..
..
..
Quote:
2) Next Generation Low-Latency Interconnects Post 8.3 (Integrity Servers
Only)

..
..
..
Quote:
Am I the only person getting their jollies out of this or what?

Regards Richard (Just off to have a cold shower) Maher


I'd be a lot more excited if they weren't artifically limiting it to the
intel based servers. It'll be a long time before those capabilities 'on
integrity' matter to us or any of our customers.

Still, its good news for the long run.


Reply With Quote
  #4  
Old   
prep@prep.synonet.com
 
Posts: n/a

Default Re: New Cluster Interconnects - 05-22-2005 , 07:41 PM



"Bill" <bill (AT) demsky (DOT) name> writes:

Quote:
Low latency is where the performance appears to be, gig ethernet,
and maybe 10g ethernet have too much latency as you scale
Almost everyone I've talked to did not know that GBE over copper
runs 4 120Mbaud lines and has to do a line turnaround, unlike
fibre where it is a real full duplex circuit. With a lot of
small packets, throughput tanks and latency blows out.

--
Paul Repacholi 1 Crescent Rd.,
+61 (08) 9257-1001 Kalamunda.
West Australia 6076
comp.os.vms,- The Older, Grumpier Slashdot
Raw, Cooked or Well-done, it's all half baked.
EPIC, The Architecture of the future, always has been, always will be.


Reply With Quote
  #5  
Old   
Keith Parris
 
Posts: n/a

Default Re: New Cluster Interconnects - 05-27-2005 , 04:56 PM



Richard Maher wrote:
Quote:
1) I saw 10 Gigabit NIC support scheduled for VMS 8.3 (Depending on which
slide you look at it says "Integrity Servers Only"). Now 10x what a lot of
people are using for a cluster-interconnect at the moment sounds pretty
s__t-hot to me! Especially if you're moving big lock-trees around the
cluster. Given that this functionality is less than a year away, surely some
performance figures or at least anecdotal evidence should be available?
One way to estimate the impact is to see how Gigabit Ethernet compared
with Fast Ethernet. Bandwidth went up by close to 10x; latency went from
about 240 microseconds for a round-trip lock conversion request with
Fast Ethernet to about 200 microseconds on Gigabit Ethernet, as measured
on a typical Alpha box of the recent past (Wildfire or ES40). So don't
expect a 10x latency improvement.

Quote:
I mean, if I was an Rdb engineer that had used p___-poor DLM performance as
the rationale for sticking all of my R&D eggs in the stand-alone single-node
basket, then I'd be interested in what's happening with this. Right? "But
it's not the bandwidth, it's the latency that gets ya." Well that brings me
to the next slide. . .
Rdb supports Row Cache in Galaxy Shared Memory between multiple nodes.

Quote:
2) Next Generation Low-Latency Interconnects Post 8.3 (Integrity Servers
Only)

Am I the only person getting their jollies out of this or what?
Nope. I also find this exciting.

Potential candidate technologies one would naturally want to look at
could include Infiniband and RDMA/iWARP.

Infiniband promises low latency, but hasn't really taken off much in the
industry yet, and hardware is expensive. Some initial proponents have
subsequently backed out (like Intel). A lot of people are waiting to see
how this turns out.

RDMA/iWARP looks to have broad potential industry support, and will
quite possibly be built into commodity Ethernet adapters. Latency
wouldn't be quite as low, but price would be low and price/performance
very good.

Should be interesting. In any case, I'm know VMS Engineering has its
finger on the pulse of the technologies available in the marketplace,
and will provide a quality solution with the best interests of the
customers in mind.

Quote:
Will there be a special limit on the distances between nodes for this stuff
to work? (Like memory channel) Can you have a Disaster Tolerant Low Latency
Cluster?
Infiniband has some fairly low distance limitations, unless you include
an Infiniband router, and that's for IP traffic.

For anything Ethernet-based I don't expect distance limitations. Other
than how far you can drive light over fiber, there's no inherent
distance limit in Gigabit Ethernet today, for example. But of course the
longer your inter-site distance, the more delay there is due to the
speed of light over the distance, which mitigates against low latency.
---
Looks likely I'll be doing a hands-on workshop at HP Technology Forum on
Long-Distance OpenVMS Clusters. We'll explore some of the impacts of
long distances on performance in that workshop, for folks who are
interested.


Reply With Quote
  #6  
Old   
Dr. Dweeb
 
Posts: n/a

Default Re: New Cluster Interconnects - 05-27-2005 , 07:37 PM



Keith Parris wrote:
Quote:
Richard Maher wrote:
1) I saw 10 Gigabit NIC support scheduled for VMS 8.3 (Depending on
which slide you look at it says "Integrity Servers Only"). Now 10x
what a lot of people are using for a cluster-interconnect at the
moment sounds pretty s__t-hot to me! Especially if you're moving big
lock-trees around the cluster. Given that this functionality is less
than a year away, surely some performance figures or at least
anecdotal evidence should be available?

One way to estimate the impact is to see how Gigabit Ethernet compared
with Fast Ethernet. Bandwidth went up by close to 10x; latency went
from about 240 microseconds for a round-trip lock conversion request
with Fast Ethernet to about 200 microseconds on Gigabit Ethernet, as
measured on a typical Alpha box of the recent past (Wildfire or
ES40). So don't expect a 10x latency improvement.

I mean, if I was an Rdb engineer that had used p___-poor DLM
performance as the rationale for sticking all of my R&D eggs in the
stand-alone single-node basket, then I'd be interested in what's
happening with this. Right? "But it's not the bandwidth, it's the
latency that gets ya." Well that brings me to the next slide. . .

Rdb supports Row Cache in Galaxy Shared Memory between multiple nodes.

2) Next Generation Low-Latency Interconnects Post 8.3 (Integrity
Servers Only)

Am I the only person getting their jollies out of this or what?

Nope. I also find this exciting.

Potential candidate technologies one would naturally want to look at
could include Infiniband and RDMA/iWARP.

Infiniband promises low latency, but hasn't really taken off much in
the industry yet, and hardware is expensive. Some initial proponents
have subsequently backed out (like Intel). A lot of people are
waiting to see how this turns out.

RDMA/iWARP looks to have broad potential industry support, and will
quite possibly be built into commodity Ethernet adapters. Latency
wouldn't be quite as low, but price would be low and price/performance
very good.

Should be interesting. In any case, I'm know VMS Engineering has its
finger on the pulse of the technologies available in the marketplace,
and will provide a quality solution with the best interests of the
customers in mind.

Will there be a special limit on the distances between nodes for
this stuff to work? (Like memory channel) Can you have a Disaster
Tolerant Low Latency Cluster?

Infiniband has some fairly low distance limitations, unless you
include an Infiniband router, and that's for IP traffic.

For anything Ethernet-based I don't expect distance limitations. Other
than how far you can drive light over fiber, there's no inherent
distance limit in Gigabit Ethernet today, for example. But of course
the longer your inter-site distance, the more delay there is due to
the speed of light over the distance, which mitigates against low
latency. ---
Indeed - it is the physical absolute barrier for comminucation of data or
energy over distance, something to do with instantaneous causal events not
being possible and all that (if I recall my physics of relativity correctly)
:-)

Dweeb.

Quote:
Looks likely I'll be doing a hands-on workshop at HP Technology Forum
on Long-Distance OpenVMS Clusters. We'll explore some of the impacts
of long distances on performance in that workshop, for folks who are
interested.



Reply With Quote
  #7  
Old   
Dr. Dweeb
 
Posts: n/a

Default Re: New Cluster Interconnects - 05-27-2005 , 07:47 PM



Dr. Dweeb wrote:
Quote:
Keith Parris wrote:
Richard Maher wrote:
1) I saw 10 Gigabit NIC support scheduled for VMS 8.3 (Depending on
which slide you look at it says "Integrity Servers Only"). Now 10x
what a lot of people are using for a cluster-interconnect at the
moment sounds pretty s__t-hot to me! Especially if you're moving big
lock-trees around the cluster. Given that this functionality is less
than a year away, surely some performance figures or at least
anecdotal evidence should be available?

One way to estimate the impact is to see how Gigabit Ethernet
compared with Fast Ethernet. Bandwidth went up by close to 10x;
latency went from about 240 microseconds for a round-trip lock
conversion request with Fast Ethernet to about 200 microseconds on
Gigabit Ethernet, as measured on a typical Alpha box of the recent
past (Wildfire or ES40). So don't expect a 10x latency improvement.

I mean, if I was an Rdb engineer that had used p___-poor DLM
performance as the rationale for sticking all of my R&D eggs in the
stand-alone single-node basket, then I'd be interested in what's
happening with this. Right? "But it's not the bandwidth, it's the
latency that gets ya." Well that brings me to the next slide. . .

Rdb supports Row Cache in Galaxy Shared Memory between multiple
nodes.
2) Next Generation Low-Latency Interconnects Post 8.3 (Integrity
Servers Only)

Am I the only person getting their jollies out of this or what?

Nope. I also find this exciting.

Potential candidate technologies one would naturally want to look at
could include Infiniband and RDMA/iWARP.

Infiniband promises low latency, but hasn't really taken off much in
the industry yet, and hardware is expensive. Some initial proponents
have subsequently backed out (like Intel). A lot of people are
waiting to see how this turns out.

RDMA/iWARP looks to have broad potential industry support, and will
quite possibly be built into commodity Ethernet adapters. Latency
wouldn't be quite as low, but price would be low and
price/performance very good.

Should be interesting. In any case, I'm know VMS Engineering has its
finger on the pulse of the technologies available in the marketplace,
and will provide a quality solution with the best interests of the
customers in mind.

Will there be a special limit on the distances between nodes for
this stuff to work? (Like memory channel) Can you have a Disaster
Tolerant Low Latency Cluster?

Infiniband has some fairly low distance limitations, unless you
include an Infiniband router, and that's for IP traffic.

For anything Ethernet-based I don't expect distance limitations.
Other than how far you can drive light over fiber, there's no
inherent distance limit in Gigabit Ethernet today, for example. But
of course the longer your inter-site distance, the more delay there
is due to the speed of light over the distance, which mitigates
against low latency. ---

Indeed - it is the physical absolute barrier for comminucation of
data or energy over distance, something to do with instantaneous
causal events not being possible and all that (if I recall my physics
of relativity correctly) :-)

Dweeb.

But perhaps not http://homepage.sunrise.ch/homepage/...pace-time.html

Dweeb.

Quote:
Looks likely I'll be doing a hands-on workshop at HP Technology Forum
on Long-Distance OpenVMS Clusters. We'll explore some of the impacts
of long distances on performance in that workshop, for folks who are
interested.



Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.