 | |
#71
 
| |||
| |||
|
Re: sequential disk read speed -
08-24-2008
, 09:29 PM
|
On Aug 21, 12:08 am, David BL <davi... (AT) iinet (DOT) net.au> wrote: On Aug 21, 8:36 am, Darren <anon5... (AT) yahoo (DOT) com> wrote: I am learning about database systems, and I am reading a book called "Physical Database Design". It gets to a bit about a large sequential access (e.g. for a full table scan), and does the following: It says "Since most disk systems use prefetch buffers to speed up table scans, we assume a 64 KB prefetch block" So to calculate the time for a full table scan, it multiples the number of 64KB blocks by the time it takes to seek and read (2.02ms). In other words, it is seeking each 64KB block. Why can a disk only read 64KB at a time? Is this a valid assumption? Is this a disk limitation or a file system limitation? A high end modern HD with 4ms average seek will on average take about 7ms to access and an additional 0.5ms to read a randomly located 64k buffer. This mismatch shows that 64k blocks are too small for optimal read performance. 512k or 1Mb blocks would be more suitable But what dictates the block size? Is this defined by the physical disk, the file system, or the database code? |
used to refer to the intersection of a sector and a track, but now
days “sector” tends to be used instead. It is the smallest unit of
reading/writing and is often 512 bytes. Some disks use 1024 byte
sectors.
A file system provides buffering, and that allows an application to
seek within a file and read/write a single byte. However behind the
scenes an entire sector must be read/written to disk.
Typically a DBMS will read/write the disk without file buffering
provided by the OS. For example on Win32, the function CreateFile can
take the parameter FILE_FLAG_NO_BUFFERING. This forces the DBMS to
work at the granularity of sectors – and it’s fairly low level. Eg
there is a requirement that memory buffers be aligned on 512 byte
boundaries to comply with the DMA constraints.
To avoid excessive seeking the DBMS will tend to organise the store
into much courser units that are typically called “blocks”. The block
size is up to the DBMS, but in practise will always be a multiple of a
sector. In some cases it may relate back to track or cylinder
boundaries, but that constraint is not imposed by the disk controller
(which will happily allow for random access to any sector).
#72
| |||
| |||
|
|
On Aug 21, 12:08 am, David BL <davi... (AT) iinet (DOT) net.au> wrote: On Aug 21, 8:36 am, Darren <anon5... (AT) yahoo (DOT) com> wrote: I am learning about database systems, and I am reading a book called "Physical Database Design". It gets to a bit about a large sequential access (e.g. for a full table scan), and does the following: It says "Since most disk systems use prefetch buffers to speed up table scans, we assume a 64 KB prefetch block" So to calculate the time for a full table scan, it multiples the number of 64KB blocks by the time it takes to seek and read (2.02ms). In other words, it is seeking each 64KB block. Why can a disk only read 64KB at a time? Is this a valid assumption? Is this a disk limitation or a file system limitation? A high end modern HD with 4ms average seek will on average take about 7ms to access and an additional 0.5ms to read a randomly located 64k buffer. This mismatch shows that 64k blocks are too small for optimal read performance. 512k or 1Mb blocks would be more suitable But what dictates the block size? Is this defined by the physical disk, the file system, or the database code? |
used to refer to the intersection of a sector and a track, but now
days “sector” tends to be used instead. It is the smallest unit of
reading/writing and is often 512 bytes. Some disks use 1024 byte
sectors.
A file system provides buffering, and that allows an application to
seek within a file and read/write a single byte. However behind the
scenes an entire sector must be read/written to disk.
Typically a DBMS will read/write the disk without file buffering
provided by the OS. For example on Win32, the function CreateFile can
take the parameter FILE_FLAG_NO_BUFFERING. This forces the DBMS to
work at the granularity of sectors – and it’s fairly low level. Eg
there is a requirement that memory buffers be aligned on 512 byte
boundaries to comply with the DMA constraints.
To avoid excessive seeking the DBMS will tend to organise the store
into much courser units that are typically called “blocks”. The block
size is up to the DBMS, but in practise will always be a multiple of a
sector. In some cases it may relate back to track or cylinder
boundaries, but that constraint is not imposed by the disk controller
(which will happily allow for random access to any sector).
#73
| |||
| |||
|
|
On Aug 21, 12:08 am, David BL <davi... (AT) iinet (DOT) net.au> wrote: On Aug 21, 8:36 am, Darren <anon5... (AT) yahoo (DOT) com> wrote: I am learning about database systems, and I am reading a book called "Physical Database Design". It gets to a bit about a large sequential access (e.g. for a full table scan), and does the following: It says "Since most disk systems use prefetch buffers to speed up table scans, we assume a 64 KB prefetch block" So to calculate the time for a full table scan, it multiples the number of 64KB blocks by the time it takes to seek and read (2.02ms). In other words, it is seeking each 64KB block. Why can a disk only read 64KB at a time? Is this a valid assumption? Is this a disk limitation or a file system limitation? A high end modern HD with 4ms average seek will on average take about 7ms to access and an additional 0.5ms to read a randomly located 64k buffer. This mismatch shows that 64k blocks are too small for optimal read performance. 512k or 1Mb blocks would be more suitable But what dictates the block size? Is this defined by the physical disk, the file system, or the database code? |
used to refer to the intersection of a sector and a track, but now
days “sector” tends to be used instead. It is the smallest unit of
reading/writing and is often 512 bytes. Some disks use 1024 byte
sectors.
A file system provides buffering, and that allows an application to
seek within a file and read/write a single byte. However behind the
scenes an entire sector must be read/written to disk.
Typically a DBMS will read/write the disk without file buffering
provided by the OS. For example on Win32, the function CreateFile can
take the parameter FILE_FLAG_NO_BUFFERING. This forces the DBMS to
work at the granularity of sectors – and it’s fairly low level. Eg
there is a requirement that memory buffers be aligned on 512 byte
boundaries to comply with the DMA constraints.
To avoid excessive seeking the DBMS will tend to organise the store
into much courser units that are typically called “blocks”. The block
size is up to the DBMS, but in practise will always be a multiple of a
sector. In some cases it may relate back to track or cylinder
boundaries, but that constraint is not imposed by the disk controller
(which will happily allow for random access to any sector).
#74
| |||
| |||
|
|
If you have a 100GB database and you put it on single 100GB disk drive, your best average seek time is the average seek time of the disk drive, but if you put the database on four 100GB disk drives, the the best average seek time will only be a fraction of the seek time of the single disk. Suppose that the full-stroke seek time on the 100GB disk is 7ms and the track-to-track seek time is 1ms. Well, with four disks, instead of an average 4ms seek time, the individual seek time of each disk is reduced to roughly 2.5ms |
platter the head doesn't have such a large range of tracks to move
over?
Quote:
|
, and since there are four disks, the average seek time for the disk subsystem is reduced to a quarter of that or roughly .625ms. |
seeking must be independent. To achieve that I think the striping
would need to be very coarse (eg 512kb or 1Mb).
#75
| |||
| |||
|
|
If you have a 100GB database and you put it on single 100GB disk drive, your best average seek time is the average seek time of the disk drive, but if you put the database on four 100GB disk drives, the the best average seek time will only be a fraction of the seek time of the single disk. Suppose that the full-stroke seek time on the 100GB disk is 7ms and the track-to-track seek time is 1ms. Well, with four disks, instead of an average 4ms seek time, the individual seek time of each disk is reduced to roughly 2.5ms |
platter the head doesn't have such a large range of tracks to move
over?
Quote:
|
, and since there are four disks, the average seek time for the disk subsystem is reduced to a quarter of that or roughly .625ms. |
seeking must be independent. To achieve that I think the striping
would need to be very coarse (eg 512kb or 1Mb).
#76
| |||
| |||
|
|
If you have a 100GB database and you put it on single 100GB disk drive, your best average seek time is the average seek time of the disk drive, but if you put the database on four 100GB disk drives, the the best average seek time will only be a fraction of the seek time of the single disk. Suppose that the full-stroke seek time on the 100GB disk is 7ms and the track-to-track seek time is 1ms. Well, with four disks, instead of an average 4ms seek time, the individual seek time of each disk is reduced to roughly 2.5ms |
platter the head doesn't have such a large range of tracks to move
over?
Quote:
|
, and since there are four disks, the average seek time for the disk subsystem is reduced to a quarter of that or roughly .625ms. |
seeking must be independent. To achieve that I think the striping
would need to be very coarse (eg 512kb or 1Mb).
#77
| |||
| |||
|
|
If you have a 100GB database and you put it on single 100GB disk drive, your best average seek time is the average seek time of the disk drive, but if you put the database on four 100GB disk drives, the the best average seek time will only be a fraction of the seek time of the single disk. Suppose that the full-stroke seek time on the 100GB disk is 7ms and the track-to-track seek time is 1ms. Well, with four disks, instead of an average 4ms seek time, the individual seek time of each disk is reduced to roughly 2.5ms |
platter the head doesn't have such a large range of tracks to move
over?
Quote:
|
, and since there are four disks, the average seek time for the disk subsystem is reduced to a quarter of that or roughly .625ms. |
seeking must be independent. To achieve that I think the striping
would need to be very coarse (eg 512kb or 1Mb).
#78
| |||
| |||
|
|
If you have a 100GB database and you put it on single 100GB disk drive, your best average seek time is the average seek time of the disk drive, but if you put the database on four 100GB disk drives, the the best average seek time will only be a fraction of the seek time of the single disk. Suppose that the full-stroke seek time on the 100GB disk is 7ms and the track-to-track seek time is 1ms. Well, with four disks, instead of an average 4ms seek time, the individual seek time of each disk is reduced to roughly 2.5ms |
platter the head doesn't have such a large range of tracks to move
over?
Quote:
|
, and since there are four disks, the average seek time for the disk subsystem is reduced to a quarter of that or roughly .625ms. |
seeking must be independent. To achieve that I think the striping
would need to be very coarse (eg 512kb or 1Mb).
#79
| |||
| |||
|
|
If you have a 100GB database and you put it on single 100GB disk drive, your best average seek time is the average seek time of the disk drive, but if you put the database on four 100GB disk drives, the the best average seek time will only be a fraction of the seek time of the single disk. Suppose that the full-stroke seek time on the 100GB disk is 7ms and the track-to-track seek time is 1ms. Well, with four disks, instead of an average 4ms seek time, the individual seek time of each disk is reduced to roughly 2.5ms |
platter the head doesn't have such a large range of tracks to move
over?
Quote:
|
, and since there are four disks, the average seek time for the disk subsystem is reduced to a quarter of that or roughly .625ms. |
seeking must be independent. To achieve that I think the striping
would need to be very coarse (eg 512kb or 1Mb).
#80
| |||
| |||
|
|
If you have a 100GB database and you put it on single 100GB disk drive, your best average seek time is the average seek time of the disk drive, but if you put the database on four 100GB disk drives, the the best average seek time will only be a fraction of the seek time of the single disk. Suppose that the full-stroke seek time on the 100GB disk is 7ms and the track-to-track seek time is 1ms. Well, with four disks, instead of an average 4ms seek time, the individual seek time of each disk is reduced to roughly 2.5ms |
platter the head doesn't have such a large range of tracks to move
over?
Quote:
|
, and since there are four disks, the average seek time for the disk subsystem is reduced to a quarter of that or roughly .625ms. |
seeking must be independent. To achieve that I think the striping
would need to be very coarse (eg 512kb or 1Mb).
 |
« Previous Thread
|
Next Thread »
| Thread Tools | |
| Display Modes | |
Linear Mode
| |
Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2013, Jelsoft Enterprises Ltd.
Copyright ©2000 - 2013, Jelsoft Enterprises Ltd.