Monday, March 26, 2012

long processing time

Hi all,
I have a datawarehouse which contain a historical table holding close to 100
million records... this table is growing exponentially. Everytime I process
my cube, data has to be read from this table and it is taking very long
before data is returned (between 2-3 hours)
Is there anyway for me to speed this up?
Troubled,
NestorIf you have the Enterprise edition of SQL Server you should look into
partitioning. With partitions you can process things in parallel and you
can also possibly only re-process recent data.
Depending on your situation you may also be able to look into
incremental processing, but this only works if your dimensions do not
change (you can add new members, but moving existing members under new
parents will break incremental processing)
You should also make sure your schema is optimized so that you are
minimizing the joins that have to be done against the relational data
source.
Regards
Darren Gosbell [MCSD]
Blog: http://www.geekswithblogs.net/darrengosbell
In article <edXVSA44FHA.1416@.TK2MSFTNGP09.phx.gbl>, n3570r@.yahoo.com
says...
> Hi all,
> I have a datawarehouse which contain a historical table holding close to 1
00
> million records... this table is growing exponentially. Everytime I proces
s
> my cube, data has to be read from this table and it is taking very long
> before data is returned (between 2-3 hours)
> Is there anyway for me to speed this up?
> Troubled,
> Nestor
>
>

No comments:

Post a Comment