Radicore Forum: How To » Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern

Home » RADICORE » How To » Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern (Do you have any advice about how to structure a program that will be moving data between pgsql and sqlsrv databases?)

Show: Today's Messages :: Polls :: Message Navigator
E-mail to friend

Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern [message #4749]

Fri, 22 May 2015 18:14

rafs
Messages: 69
Registered: May 2015

Member

I apologize if this is too off-topic, but I am hoping you might have a suggestion for how I might proceed to use your radicore infrastructure for an ETL solution. There is an older sqlsrv database with four sets of tables, each set of tables is similar to the other sets with the addition of some peculiarities/customizations. I have redesigned the schema to consolidate each of the four sets into a single set of tables (that encompasses all of the union of all customizations). I am connecting to the databases and have all of my classes created via the radicore IMPORT DATABASES and TABLES and COLUMNS, etc, then EXPORT PHP. This is all fine.

There are three directions I am considering, as to where to put my code to make the transformations: (1) just a jumble of procedures (functions) in my main script code to transform an array of one type to an array of another type; (2) create a new class called Transformer that takes ($fieldarray, $classname) and returns $fieldarray of the equivalent type (from its opposite database); or (3) to put the transformation code into the std.table.class.inc and extend it in the subclasses.

If (3), then maybe defining:
$fieldarray_transformed = getData_transformed($where);

Do you have any suggestions?

Thanks.

Report message to a moderator

[Message index]

		Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: rafs on Fri, 22 May 2015 18:14
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: AJM on Sat, 23 May 2015 09:46
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: rafs on Tue, 02 June 2015 22:21
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: rafs on Tue, 02 June 2015 23:28
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: AJM on Wed, 03 June 2015 05:09
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: rafs on Wed, 03 June 2015 05:31
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: rafs on Wed, 03 June 2015 05:32
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: rafs on Wed, 03 June 2015 06:32
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: rafs on Wed, 03 June 2015 19:16
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: AJM on Thu, 04 June 2015 04:37
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: rafs on Thu, 04 June 2015 18:39
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: rafs on Fri, 05 June 2015 03:08
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: AJM on Fri, 05 June 2015 04:47
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: rafs on Thu, 18 June 2015 12:52
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: AJM on Fri, 19 June 2015 05:20
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: rafs on Fri, 19 June 2015 23:48
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: AJM on Sat, 20 June 2015 04:15
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: rafs on Sun, 21 June 2015 16:14
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: AJM on Mon, 22 June 2015 04:48
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: rafs on Sun, 21 June 2015 12:33
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: rafs on Sun, 21 June 2015 13:16
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: AJM on Mon, 22 June 2015 04:20
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: rafs on Wed, 08 July 2015 03:28
		Re: Batch Script connecting two databases in an Extract Tranform Load (ETL) Design Pattern By: AJM on Wed, 08 July 2015 05:17

Previous Topic:	How to use actions(submit,submitstay,copy,quit)
Next Topic:	tutorial4 Viewing the Tree structure

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Mon Jul 07 00:30:41 EDT 2025

Total time taken to generate the page: 0.09946 seconds