 | |
#41
 
| |||
| |||
|
Re: sequential disk read speed -
08-23-2008
, 01:04 AM
|
Why can a disk only read 64KB at a time? Is this a valid assumption? Is this a disk limitation or a file system limitation? The author has to provide some numbers to show how to calculate an estimation for disk access. Frankly, 64KB seems a little small for a modern computer other than a desktop machine. |
+rotational delay and lower transfer rate giving about the same
product as for an enterprise HD.
Quote:
|
What you might consider is the rise of solid state storage, which will start replacing moving disk hardware in the next few years. This with multi-core processors will change database design radically. We work in a trade where everything you know is wrong in five years  |
imagined. CPU caches lead to significant variation in memory access
times.
I few years ago I wrote a transient B+Tree and compared the
performance to the STL map (a red black tree) that ships with MS
Visual C++. I ran tests involving inserting a million randomly
generated keys on a map keyed by 32 bit integers. The B+Tree was
twice as fast at insertions and deletions, 35% faster at look up, and
10 times faster at iteration through the elements.
#42
| |||
| |||
|
|
Why can a disk only read 64KB at a time? Is this a valid assumption? Is this a disk limitation or a file system limitation? The author has to provide some numbers to show how to calculate an estimation for disk access. Frankly, 64KB seems a little small for a modern computer other than a desktop machine. |
+rotational delay and lower transfer rate giving about the same
product as for an enterprise HD.
Quote:
|
What you might consider is the rise of solid state storage, which will start replacing moving disk hardware in the next few years. This with multi-core processors will change database design radically. We work in a trade where everything you know is wrong in five years |
imagined. CPU caches lead to significant variation in memory access
times.
I few years ago I wrote a transient B+Tree and compared the
performance to the STL map (a red black tree) that ships with MS
Visual C++. I ran tests involving inserting a million randomly
generated keys on a map keyed by 32 bit integers. The B+Tree was
twice as fast at insertions and deletions, 35% faster at look up, and
10 times faster at iteration through the elements.
#43
| |||
| |||
|
|
Why can a disk only read 64KB at a time? Is this a valid assumption? Is this a disk limitation or a file system limitation? The author has to provide some numbers to show how to calculate an estimation for disk access. Frankly, 64KB seems a little small for a modern computer other than a desktop machine. |
+rotational delay and lower transfer rate giving about the same
product as for an enterprise HD.
Quote:
|
What you might consider is the rise of solid state storage, which will start replacing moving disk hardware in the next few years. This with multi-core processors will change database design radically. We work in a trade where everything you know is wrong in five years |
imagined. CPU caches lead to significant variation in memory access
times.
I few years ago I wrote a transient B+Tree and compared the
performance to the STL map (a red black tree) that ships with MS
Visual C++. I ran tests involving inserting a million randomly
generated keys on a map keyed by 32 bit integers. The B+Tree was
twice as fast at insertions and deletions, 35% faster at look up, and
10 times faster at iteration through the elements.
#44
| |||
| |||
|
|
Why can a disk only read 64KB at a time? Is this a valid assumption? Is this a disk limitation or a file system limitation? The author has to provide some numbers to show how to calculate an estimation for disk access. Frankly, 64KB seems a little small for a modern computer other than a desktop machine. |
+rotational delay and lower transfer rate giving about the same
product as for an enterprise HD.
Quote:
|
What you might consider is the rise of solid state storage, which will start replacing moving disk hardware in the next few years. This with multi-core processors will change database design radically. We work in a trade where everything you know is wrong in five years |
imagined. CPU caches lead to significant variation in memory access
times.
I few years ago I wrote a transient B+Tree and compared the
performance to the STL map (a red black tree) that ships with MS
Visual C++. I ran tests involving inserting a million randomly
generated keys on a map keyed by 32 bit integers. The B+Tree was
twice as fast at insertions and deletions, 35% faster at look up, and
10 times faster at iteration through the elements.
#45
| |||
| |||
|
|
Why can a disk only read 64KB at a time? Is this a valid assumption? Is this a disk limitation or a file system limitation? The author has to provide some numbers to show how to calculate an estimation for disk access. Frankly, 64KB seems a little small for a modern computer other than a desktop machine. |
+rotational delay and lower transfer rate giving about the same
product as for an enterprise HD.
Quote:
|
What you might consider is the rise of solid state storage, which will start replacing moving disk hardware in the next few years. This with multi-core processors will change database design radically. We work in a trade where everything you know is wrong in five years |
imagined. CPU caches lead to significant variation in memory access
times.
I few years ago I wrote a transient B+Tree and compared the
performance to the STL map (a red black tree) that ships with MS
Visual C++. I ran tests involving inserting a million randomly
generated keys on a map keyed by 32 bit integers. The B+Tree was
twice as fast at insertions and deletions, 35% faster at look up, and
10 times faster at iteration through the elements.
#46
| |||
| |||
|
|
Why can a disk only read 64KB at a time? Is this a valid assumption? Is this a disk limitation or a file system limitation? The author has to provide some numbers to show how to calculate an estimation for disk access. Frankly, 64KB seems a little small for a modern computer other than a desktop machine. |
+rotational delay and lower transfer rate giving about the same
product as for an enterprise HD.
Quote:
|
What you might consider is the rise of solid state storage, which will start replacing moving disk hardware in the next few years. This with multi-core processors will change database design radically. We work in a trade where everything you know is wrong in five years |
imagined. CPU caches lead to significant variation in memory access
times.
I few years ago I wrote a transient B+Tree and compared the
performance to the STL map (a red black tree) that ships with MS
Visual C++. I ran tests involving inserting a million randomly
generated keys on a map keyed by 32 bit integers. The B+Tree was
twice as fast at insertions and deletions, 35% faster at look up, and
10 times faster at iteration through the elements.
#47
| |||
| |||
|
|
On Aug 21, 8:36 am, Darren <anon5... (AT) yahoo (DOT) com> wrote: I am learning about database systems, and I am reading a book called "Physical Database Design". It gets to a bit about a large sequential access (e.g. for a full table scan), and does the following: It says "Since most disk systems use prefetch buffers to speed up table scans, we assume a 64 KB prefetch block" So to calculate the time for a full table scan, it multiples the number of 64KB blocks by the time it takes to seek and read (2.02ms). In other words, it is seeking each 64KB block. Why can a disk only read 64KB at a time? Is this a valid assumption? Is this a disk limitation or a file system limitation? A high end modern HD with 4ms average seek will on average take about 7ms to access and an additional 0.5ms to read a randomly located 64k buffer. * This mismatch shows that 64k blocks are too small for optimal read performance. * 512k or 1Mb blocks would be more suitable.-Hide quoted text - - Show quoted text - |
disk, the file system, or the database code?
#48
| |||
| |||
|
|
On Aug 21, 8:36 am, Darren <anon5... (AT) yahoo (DOT) com> wrote: I am learning about database systems, and I am reading a book called "Physical Database Design". It gets to a bit about a large sequential access (e.g. for a full table scan), and does the following: It says "Since most disk systems use prefetch buffers to speed up table scans, we assume a 64 KB prefetch block" So to calculate the time for a full table scan, it multiples the number of 64KB blocks by the time it takes to seek and read (2.02ms). In other words, it is seeking each 64KB block. Why can a disk only read 64KB at a time? Is this a valid assumption? Is this a disk limitation or a file system limitation? A high end modern HD with 4ms average seek will on average take about 7ms to access and an additional 0.5ms to read a randomly located 64k buffer. * This mismatch shows that 64k blocks are too small for optimal read performance. * 512k or 1Mb blocks would be more suitable.-Hide quoted text - - Show quoted text - |
disk, the file system, or the database code?
#49
| |||
| |||
|
|
On Aug 21, 8:36 am, Darren <anon5... (AT) yahoo (DOT) com> wrote: I am learning about database systems, and I am reading a book called "Physical Database Design". It gets to a bit about a large sequential access (e.g. for a full table scan), and does the following: It says "Since most disk systems use prefetch buffers to speed up table scans, we assume a 64 KB prefetch block" So to calculate the time for a full table scan, it multiples the number of 64KB blocks by the time it takes to seek and read (2.02ms). In other words, it is seeking each 64KB block. Why can a disk only read 64KB at a time? Is this a valid assumption? Is this a disk limitation or a file system limitation? A high end modern HD with 4ms average seek will on average take about 7ms to access and an additional 0.5ms to read a randomly located 64k buffer. * This mismatch shows that 64k blocks are too small for optimal read performance. * 512k or 1Mb blocks would be more suitable.-Hide quoted text - - Show quoted text - |
disk, the file system, or the database code?
#50
| |||
| |||
|
|
On Aug 21, 8:36 am, Darren <anon5... (AT) yahoo (DOT) com> wrote: I am learning about database systems, and I am reading a book called "Physical Database Design". It gets to a bit about a large sequential access (e.g. for a full table scan), and does the following: It says "Since most disk systems use prefetch buffers to speed up table scans, we assume a 64 KB prefetch block" So to calculate the time for a full table scan, it multiples the number of 64KB blocks by the time it takes to seek and read (2.02ms). In other words, it is seeking each 64KB block. Why can a disk only read 64KB at a time? Is this a valid assumption? Is this a disk limitation or a file system limitation? A high end modern HD with 4ms average seek will on average take about 7ms to access and an additional 0.5ms to read a randomly located 64k buffer. * This mismatch shows that 64k blocks are too small for optimal read performance. * 512k or 1Mb blocks would be more suitable.-Hide quoted text - - Show quoted text - |
disk, the file system, or the database code?
 |
« Previous Thread
|
Next Thread »
| Thread Tools | |
| Display Modes | |
Linear Mode
| |
Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.