Comments on: The Core Performance Fundamentals Of Oracle Data Warehousing – Partitioning http://structureddata.org/2010/01/25/the-core-performance-fundamentals-of-oracle-data-warehousing-partitioning/?utm_source=rss&utm_medium=rss&utm_campaign=the-core-performance-fundamentals-of-oracle-data-warehousing-partitioning Data, Databases, Performance & Scalability Mon, 30 Jan 2012 17:05:12 -0500 hourly 1 http://wordpress.org/?v=3.3.1 By: Greg Rahn http://structureddata.org/2010/01/25/the-core-performance-fundamentals-of-oracle-data-warehousing-partitioning/#comment-23114 Greg Rahn Fri, 23 Sep 2011 15:10:11 +0000 http://structureddata.org/?p=816#comment-23114 Understand that indexes are structures to access data sets that are very selective (few rows). If a query needs 1/10 of the data in the table (1M of 10M rows, in this case) that really is not selective enough for an index. Besides, in the worse case (1 row you want per block), that would translate in 1M single block reads. With a partition scan, the IO size could be up to 1MB. So you see the difference here? If you have to access a significant amount of blocks would you rather do that one 8KB block at a time, or 1MB (128 8K blocks) at a time. It simply comes down to the physics of data access and the times associated with it. Understand that indexes are structures to access data sets that are very selective (few rows). If a query needs 1/10 of the data in the table (1M of 10M rows, in this case) that really is not selective enough for an index. Besides, in the worse case (1 row you want per block), that would translate in 1M single block reads. With a partition scan, the IO size could be up to 1MB. So you see the difference here? If you have to access a significant amount of blocks would you rather do that one 8KB block at a time, or 1MB (128 8K blocks) at a time. It simply comes down to the physics of data access and the times associated with it.

]]> By: ivin kris http://structureddata.org/2010/01/25/the-core-performance-fundamentals-of-oracle-data-warehousing-partitioning/#comment-23111 ivin kris Fri, 23 Sep 2011 11:30:20 +0000 http://structureddata.org/?p=816#comment-23111 I read this, "Partition pruning is the simplest and also the most substantial means to improve performance using partitioning. Partition pruning can often improve query performance by several orders of magnitude. For example, suppose an application contains an Orders table containing a historical record of orders, and that this table has been partitioned by week. A query requesting orders for a single week would only access a single partition of the Orders table. If the Orders table had 2 years of historical data, then this query would access one partition instead of 104 partitions. This query could potentially execute 100 times faster simply because of partition pruning." THat is where I am not sure, how. If the business date column is indexed, in a table of 10 million rows, 1 million being in the current business date, how does partitions limit the number of rows to be scanned. It does tell what internally it means to say that the queries can execute faster in smaller partitions than the ones scanning multiple partitions. I am an application programmer and not a DBA. Is it possible to give little more indepth explanation of an overhead associated to multi-partitions scanning and the performance issues related to it. And I am not even talking about multi-partitions here. What if the table isn't partitioned at all. Will the query still perform slower? I read this,
“Partition pruning is the simplest and also the most substantial means to improve performance using partitioning. Partition pruning can often improve query performance by several orders of magnitude. For example, suppose an application contains an Orders table containing a historical record of orders, and that this table has been partitioned by week. A query requesting orders for a single week would only access a single partition of the Orders table. If the Orders table had 2 years of historical data, then this query would access one partition instead of 104 partitions. This query could potentially execute 100 times faster simply because of partition pruning.”

THat is where I am not sure, how. If the business date column is indexed, in a table of 10 million rows, 1 million being in the current business date, how does partitions limit the number of rows to be scanned. It does tell what internally it means to say that the queries can execute faster in smaller partitions than the ones scanning multiple partitions. I am an application programmer and not a DBA. Is it possible to give little more indepth explanation of an overhead associated to multi-partitions scanning and the performance issues related to it. And I am not even talking about multi-partitions here. What if the table isn’t partitioned at all. Will the query still perform slower?

]]> By: Greg Rahn http://structureddata.org/2010/01/25/the-core-performance-fundamentals-of-oracle-data-warehousing-partitioning/#comment-23108 Greg Rahn Fri, 23 Sep 2011 04:40:14 +0000 http://structureddata.org/?p=816#comment-23108 Again, I'd suggest to read the <a href="http://download.oracle.com/docs/cd/E11882_01/server.112/e16541/toc.htm" rel="nofollow">Database VLDB and Partitioning Guide</a>. Many of these topics are discussed there. Again, I’d suggest to read the Database VLDB and Partitioning Guide. Many of these topics are discussed there.

]]> By: Greg Rahn http://structureddata.org/2010/01/25/the-core-performance-fundamentals-of-oracle-data-warehousing-partitioning/#comment-23107 Greg Rahn Fri, 23 Sep 2011 04:36:49 +0000 http://structureddata.org/?p=816#comment-23107 <strong>1) I meant that if the BTree is balanced and the depth is always 3, the number of io’s needed to select data in a table without partitions vs with partitions should not have difference.isn’t it? Why is it said that partitions eliminate the need to scan largely unwanted data. Could you please explain. </strong> Again, index balance and depth has nothing to do with the number of IOs to the table. It's also irrelevant to the point being made here. Partitioning allows the identification of data that is not needed. I'd suggest you read <a href="http://download.oracle.com/docs/cd/E11882_01/server.112/e16541/partition.htm#I1050101" rel="nofollow">Partition Pruning</a> section in the <a href="http://download.oracle.com/docs/cd/E11882_01/server.112/e16541/partition.htm" rel="nofollow">Partitioning Concepts</a> chapter of the <a href="http://download.oracle.com/docs/cd/E11882_01/server.112/e16541/toc.htm" rel="nofollow">VLDB and Partitioning Guide</a> -- it explains it all. <strong>2) if oracle uses multi-version read concurrency why does large data load have contentions with other transactions reading data from the table. It shouldn’t isn’t it?</strong> It' doesn't. Not sure where you are reading that from my post. <strong>3) Depending on the reuirement? My requirement is that I ma having to update large amount of data in the table when the users are running queries against it. Do I need a partition swap in this scenario was my question.</strong> Bulk updates in general should be avoided. Bulk updates 1) are logging and 2) are slowed down by compression. The fastest way to do a bulk update is to do a nologging insert, changed the data in flight, and partition swap or table rename, given an appropriately sized system. 1) I meant that if the BTree is balanced and the depth is always 3, the number of io’s needed to select data in a table without partitions vs with partitions should not have difference.isn’t it? Why is it said that partitions eliminate the need to scan largely unwanted data. Could you please explain.

Again, index balance and depth has nothing to do with the number of IOs to the table. It’s also irrelevant to the point being made here. Partitioning allows the identification of data that is not needed. I’d suggest you read Partition Pruning section in the Partitioning Concepts chapter of the VLDB and Partitioning Guide — it explains it all.

2) if oracle uses multi-version read concurrency why does large data load have contentions with other transactions reading data from the table. It shouldn’t isn’t it?

It’ doesn’t. Not sure where you are reading that from my post.

3) Depending on the reuirement? My requirement is that I ma having to update large amount of data in the table when the users are running queries against it. Do I need a partition swap in this scenario was my question.

Bulk updates in general should be avoided. Bulk updates 1) are logging and 2) are slowed down by compression. The fastest way to do a bulk update is to do a nologging insert, changed the data in flight, and partition swap or table rename, given an appropriately sized system.

]]> By: ivin kris http://structureddata.org/2010/01/25/the-core-performance-fundamentals-of-oracle-data-warehousing-partitioning/#comment-23105 ivin kris Fri, 23 Sep 2011 03:11:43 +0000 http://structureddata.org/?p=816#comment-23105 Assuming I have range partition on the date column, all the users are running queries for the same date(current business date), when I have large updates that needs to be processed for the data in this partition, 1) How does partition swap help for large load manageability reducing contentions with user read activities. Isn’t the multi-version read consistency already addressing the contention? 2) If there was no partition, why would the number of rows to be scanned becomes more if all the queries need the data for the same date (Current business date) (the business date column is indexed) Assuming I have range partition on the date column, all the users are running queries for the same date(current business date), when I have large updates that needs to be processed for the data in this partition,

1) How does partition swap help for large load manageability reducing contentions with user read activities. Isn’t the multi-version read consistency already addressing the contention?
2) If there was no partition, why would the number of rows to be scanned becomes more if all the queries need the data for the same date (Current business date) (the business date column is indexed)

]]> By: ivin kris http://structureddata.org/2010/01/25/the-core-performance-fundamentals-of-oracle-data-warehousing-partitioning/#comment-23100 ivin kris Fri, 23 Sep 2011 02:48:54 +0000 http://structureddata.org/?p=816#comment-23100 1) I meant that if the BTree is balanced and the depth is always 3, the number of io's needed to select data in a table without partitions vs with partitions should not have difference.isn't it? Why is it said that partitions eliminate the need to scan largely unwanted data. Could you please explain. 2) if oracle uses multi-version read concurrency why does large data load have contentions with other transactions reading data from the table. It shouldn't isn't it? 3) Depending on the reuirement? My requirement is that I ma having to update large amount of data in the table when the users are running queries against it. Do I need a partition swap in this scenario was my question. 1) I meant that if the BTree is balanced and the depth is always 3, the number of io’s needed to select data in a table without partitions vs with partitions should not have difference.isn’t it? Why is it said that partitions eliminate the need to scan largely unwanted data. Could you please explain.
2) if oracle uses multi-version read concurrency why does large data load have contentions with other transactions reading data from the table. It shouldn’t isn’t it?
3) Depending on the reuirement? My requirement is that I ma having to update large amount of data in the table when the users are running queries against it. Do I need a partition swap in this scenario was my question.

]]> By: Greg Rahn http://structureddata.org/2010/01/25/the-core-performance-fundamentals-of-oracle-data-warehousing-partitioning/#comment-23099 Greg Rahn Fri, 23 Sep 2011 02:32:20 +0000 http://structureddata.org/?p=816#comment-23099 <strong>If Oracle B Tree index is balanced, why would a table with no paritions need to scan more rows if the time column is index and the query is based on that?</strong> First, balance has nothing to do with what rows are retrieved from a table. Partitioning allows multi-block scans, where index access translates to single block access to the table. For a large number of rows partitioning will prevail when it comes to performance. <strong>And if oracle uses multi-version read concurrency why does large data load bother concurrency? </strong> Bother concurrency of what? <strong>Why can I not load large data directly to the table instead of loading to temp table and swapping the partition?</strong> You can certainly load directly into a table. Both are options depending on the requirements. If Oracle B Tree index is balanced, why would a table with no paritions need to scan more rows if the time column is index and the query is based on that?
First, balance has nothing to do with what rows are retrieved from a table. Partitioning allows multi-block scans, where index access translates to single block access to the table. For a large number of rows partitioning will prevail when it comes to performance.

And if oracle uses multi-version read concurrency why does large data load bother concurrency?
Bother concurrency of what?

Why can I not load large data directly to the table instead of loading to temp table and swapping the partition?
You can certainly load directly into a table. Both are options depending on the requirements.

]]> By: ivin kris http://structureddata.org/2010/01/25/the-core-performance-fundamentals-of-oracle-data-warehousing-partitioning/#comment-23098 ivin kris Fri, 23 Sep 2011 02:14:40 +0000 http://structureddata.org/?p=816#comment-23098 If Oracle B Tree index is balanced, why would a table with no paritions need to scan more rows if the time column is index and the query is based on that? And if oracle uses multi-version read concurrency why does large data load bother concurrency? Why can I not load large data directly to the table instead of loading to temp table and swapping the partition? If Oracle B Tree index is balanced, why would a table with no paritions need to scan more rows if the time column is index and the query is based on that? And if oracle uses multi-version read concurrency why does large data load bother concurrency? Why can I not load large data directly to the table instead of loading to temp table and swapping the partition?

]]> By: Greg Rahn http://structureddata.org/2010/01/25/the-core-performance-fundamentals-of-oracle-data-warehousing-partitioning/#comment-17112 Greg Rahn Mon, 28 Mar 2011 03:22:17 +0000 http://structureddata.org/?p=816#comment-17112 @Eric "The Product_Dim (SCD-1) has a composite PK (COMPANY_ID, ITEM_ID)" I think your problem starts there. I'm quite sure that dimensions are supposed to have a single column surrogate primary key. It seems to me like your product_dim table has both products and companies in it (500M unique products does seem rather large to me.) Partitioning has a purpose, be it for queries or for loads; what is the purpose of your partitioning? @Eric

“The Product_Dim (SCD-1) has a composite PK (COMPANY_ID, ITEM_ID)”

I think your problem starts there. I’m quite sure that dimensions are supposed to have a single column surrogate primary key. It seems to me like your product_dim table has both products and companies in it (500M unique products does seem rather large to me.)

Partitioning has a purpose, be it for queries or for loads; what is the purpose of your partitioning?

]]> By: Eric http://structureddata.org/2010/01/25/the-core-performance-fundamentals-of-oracle-data-warehousing-partitioning/#comment-17037 Eric Wed, 23 Mar 2011 18:36:35 +0000 http://structureddata.org/?p=816#comment-17037 "There are more than 500M unique PK in this dimension" means "There are more than 500M unique PK values in this dimension" “There are more than 500M unique PK in this dimension” means “There are more than 500M unique PK values in this dimension”

]]>