dbTalk Databases Forums  

Nightly Data Structure Refresh

microsoft.public.sqlserver.dts microsoft.public.sqlserver.dts


Discuss Nightly Data Structure Refresh in the microsoft.public.sqlserver.dts forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Steve R
 
Posts: n/a

Default Nightly Data Structure Refresh - 08-04-2005 , 11:21 AM






We are receiving a reporting database from a vendor nightly. We want to
upload this data and structures to SQL Server nightly. We want it to be as
flexible as possible - if the vendor adds/deletes a table/column, the import
job should handle that.

Apparently, the Import/Export wizard in SQL Server and the Upsizing Wizard
in Access both require you to specify what tables to import. This would
prohibit the job from creating new tables/columns on the fly. Any
suggestions for a solution to this problem?

Thanks for the help...


Reply With Quote
  #2  
Old   
Darren Green
 
Posts: n/a

Default Re: Nightly Data Structure Refresh - 08-04-2005 , 11:43 AM






Backup and restore?

Transfer Objects Task ?

--
Darren Green
http://www.sqldts.com
http://www.sqlis.com

"Steve R" <SteveR (AT) discussions (DOT) microsoft.com> wrote

Quote:
We are receiving a reporting database from a vendor nightly. We want to
upload this data and structures to SQL Server nightly. We want it to be
as
flexible as possible - if the vendor adds/deletes a table/column, the
import
job should handle that.

Apparently, the Import/Export wizard in SQL Server and the Upsizing Wizard
in Access both require you to specify what tables to import. This would
prohibit the job from creating new tables/columns on the fly. Any
suggestions for a solution to this problem?

Thanks for the help...




Reply With Quote
  #3  
Old   
Stu
 
Posts: n/a

Default Re: Nightly Data Structure Refresh - 08-04-2005 , 11:50 AM



Couple of questions:

1. Are you trying to append this information to an existing reporting
database, or are you simply discarding a database daily and replacing
it with this new reporting db?

2. What format is the reporting db in (Access, SQL Server)?

3. How much control do you have over the vendor deliverable? Do you
have administrative rights on the box from whence the data comes, or is
it some form of EDI?

Stu


Reply With Quote
  #4  
Old   
Steve R
 
Posts: n/a

Default Re: Nightly Data Structure Refresh - 08-04-2005 , 12:15 PM



1. We want to discard the database daily and replace it with the new db.

2. The vendor delivers an Access database, which we want to upload/import
into SQL Server.

3. We do not have control over the vendor's deliverable. As far as we are
concerned, it is only a file to be FTP'ed to us nightly. We will process
from there.


"Stu" wrote:

Quote:
Couple of questions:

1. Are you trying to append this information to an existing reporting
database, or are you simply discarding a database daily and replacing
it with this new reporting db?

2. What format is the reporting db in (Access, SQL Server)?

3. How much control do you have over the vendor deliverable? Do you
have administrative rights on the box from whence the data comes, or is
it some form of EDI?

Stu



Reply With Quote
  #5  
Old   
Steve R
 
Posts: n/a

Default Re: Nightly Data Structure Refresh - 08-04-2005 , 12:16 PM



We are starting with the Access database and I am not sure how a backup and
restore would work with this. The Transfer Objects Task only works from one
SQL Server to another, correct?


"Darren Green" wrote:

Quote:
Backup and restore?

Transfer Objects Task ?

--
Darren Green
http://www.sqldts.com
http://www.sqlis.com

"Steve R" <SteveR (AT) discussions (DOT) microsoft.com> wrote in message
news:116327AF-3861-4406-845A-020093FAF674 (AT) microsoft (DOT) com...
We are receiving a reporting database from a vendor nightly. We want to
upload this data and structures to SQL Server nightly. We want it to be
as
flexible as possible - if the vendor adds/deletes a table/column, the
import
job should handle that.

Apparently, the Import/Export wizard in SQL Server and the Upsizing Wizard
in Access both require you to specify what tables to import. This would
prohibit the job from creating new tables/columns on the fly. Any
suggestions for a solution to this problem?

Thanks for the help...





Reply With Quote
  #6  
Old   
Rick Gittins
 
Posts: n/a

Default Re: Nightly Data Structure Refresh - 08-04-2005 , 12:27 PM



I needed to handle something similar a while back.

Here are the steps that I did:
1) Created a DTS and this import looks today.
2) Exported the DTS to VB.
3) Cleaned up the DTS for to add a new task would be calling a procedure.
4) For each table in Access get the field name.
5) Call the procedure to add each field to the DTS (in VB code)

I know this is a breif explaination of what I did, but if you want more info
let me know.

Rick

"Steve R" <SteveR (AT) discussions (DOT) microsoft.com> wrote

Quote:
1. We want to discard the database daily and replace it with the new db.

2. The vendor delivers an Access database, which we want to upload/import
into SQL Server.

3. We do not have control over the vendor's deliverable. As far as we are
concerned, it is only a file to be FTP'ed to us nightly. We will process
from there.


"Stu" wrote:

Couple of questions:

1. Are you trying to append this information to an existing reporting
database, or are you simply discarding a database daily and replacing
it with this new reporting db?

2. What format is the reporting db in (Access, SQL Server)?

3. How much control do you have over the vendor deliverable? Do you
have administrative rights on the box from whence the data comes, or is
it some form of EDI?

Stu





Reply With Quote
  #7  
Old   
David Portas
 
Posts: n/a

Default Re: Nightly Data Structure Refresh - 08-04-2005 , 01:01 PM



Is the requirement to merge this with an existing DB or just to load
the vendor's DB onto your server. If it's the latter then just
BACKUP/RESTORE or detach/attach.

If you actually want to merge client schema changes into your own DB
while keeping your existing data then you face quite a challenge.
Suppose the data you already have in your tables violates the
constraints, datatypes or other metadata changes that the client has
made? Do you then want to delete invalid data from your database?

It's certainly possible to automate or semi-automate simple changes
like adding or removing columns. For that you'll probably want a
package with more sophisticated metadata management features than DTS
has. Consider purchasing some third-party ETL tools for the job.

--
David Portas
SQL Server MVP
--


Reply With Quote
  #8  
Old   
Steve R
 
Posts: n/a

Default Re: Nightly Data Structure Refresh - 08-04-2005 , 01:24 PM



I want to load the vendor's DB onto my server from Access to SQL Server. We
will completely replace the data and schema nightly.

Can you do a backup/restore or detach/attach from Access to SQL? if so, can
it be scheduled?


"David Portas" wrote:

Quote:
Is the requirement to merge this with an existing DB or just to load
the vendor's DB onto your server. If it's the latter then just
BACKUP/RESTORE or detach/attach.

If you actually want to merge client schema changes into your own DB
while keeping your existing data then you face quite a challenge.
Suppose the data you already have in your tables violates the
constraints, datatypes or other metadata changes that the client has
made? Do you then want to delete invalid data from your database?

It's certainly possible to automate or semi-automate simple changes
like adding or removing columns. For that you'll probably want a
package with more sophisticated metadata management features than DTS
has. Consider purchasing some third-party ETL tools for the job.

--
David Portas
SQL Server MVP
--



Reply With Quote
  #9  
Old   
Steve R
 
Posts: n/a

Default Re: Nightly Data Structure Refresh - 08-04-2005 , 01:28 PM



I want to load the vendor's DB onto my server from Access to SQL Server. We
will completely replace the data and schema nightly.

Can you do a backup/restore or detach/attach from Access to SQL? if so, can
it be scheduled?


"David Portas" wrote:

Quote:
Is the requirement to merge this with an existing DB or just to load
the vendor's DB onto your server. If it's the latter then just
BACKUP/RESTORE or detach/attach.

If you actually want to merge client schema changes into your own DB
while keeping your existing data then you face quite a challenge.
Suppose the data you already have in your tables violates the
constraints, datatypes or other metadata changes that the client has
made? Do you then want to delete invalid data from your database?

It's certainly possible to automate or semi-automate simple changes
like adding or removing columns. For that you'll probably want a
package with more sophisticated metadata management features than DTS
has. Consider purchasing some third-party ETL tools for the job.

--
David Portas
SQL Server MVP
--



Reply With Quote
  #10  
Old   
Stu
 
Posts: n/a

Default Re: Nightly Data Structure Refresh - 08-04-2005 , 02:12 PM



I'm not as familiar with Access as I used to be, so take the following
ideas as conceptual only; I think that you can use VBScript to open up
an Access database, and cycle thruhg all of the table and view objects
in that database, generating the appropriate SQL creation statements
for those objects. You'll have to avoid crosstab queries, but I think
those are the only Access views that won't translate into SQL Server (i
could be wrong).

You'll then have to drop the database on SQL Server (if it exists), and
then run the newly created SQL script that you generated to create
those objects. You'll then need to copy data from your tables to your
target database. It can be done, but it'll be a helluva DTS package,
and could take a while to run each time. You could probably accomlish
all of it in a single ActionScript item.

It actually sounds like fun, but then I'm a geek

Stu


Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.