Dimension design: A different perspective


Pre-requisites: An infocube is already created and active, and filled will data, which will be used for analysis of dimension tables.

Dimension to Fact Ratio Computation: This ratio is a percentage figure of the number of records that exists in the dimension table to the number of records in fact table or what percentage of fact table size is a dimension table. Mathematically putting it down, the equation would be as below:

          Ratio = No of rows in Dimension table X 100 / No of rows in Fact Table

Dimension Table Design Concept: We have been reading and hearing over and over again that the characteristics should be added into a dimension if there exists a 1:1 or 1:M relation and they should be in separate dimension if there exists a M:M relation. What is this 1:1 or 1: M? This is the relation which the characteristics share among each other.
For instance if one Plant can have only one Storage Location and one storage location can belong to only one plant at any given point of time, then the relation shared among them is 1:1.
If 1 Functional Location can have many equipment but one equipment can belong to only one functional location then the relation shared between the functional location and Equipment is 1:M.
If 1 sales order can have many materials and one material can exist in different sales orders then there absolutely is no dependence among these two and the relation between these two is many to many or M: M.

Challenges in understanding the relationship: Often we SAP BI consultants depend on the Functional consultants to help us out with the relationship shared between these characteristics / fields. Due to time constraint we generally cannot dedicate time to educate the functional consultants on the purpose of this exercise, and it takes a lot of time to understand this relationship thoroughly.


Scenario: An infocube ZPFANALYSIS had few dimensions which were way larger than the preferred 20% ratio. This had to be redesigned such that the performance was under 20% ratio.
This ratio could be either manually derived by checking the number of entries in the desired dimension table (/BIC/D<infocube name><dimension number>) to the fact table (/BIC/F<Infocube Name> or /BIC/E<Infocube name>) or a program SAP_INFOCUBE_DESIGNS can be executed in SE38 which reports this ratio for all the dimensions, for all the infocubes in the system.

SAP_INFOCUBE_DESIGNS:
1.jpg
We can find from the report that the total number of rows in the fact table is 643850. Dimension 2 (/BIC/DZPFANLSYS2) has around 640430 rows, which is 99% (99.49%)of the fact table rows and Dimension 4(/BIC/DZPFANLSYS4) has around 196250 rows, which is 30%  (30.48%)of the fact table rows.

Infocube ZPFANLSYS:
ZPFANLSYS.jpg

Approach:

Step 1: Analysis of the dimension table /BIC/DZPFANALSYS2 to plan on reducing the number of records.
/BIC/DZPFANLSYS2
3.jpg

Fact table:
4.jpg
Dimension table holds 1 record more than the fact table.
View the data in the table /BIC/DZPFANLSYS2 (Table related to Dimension 2) in SE12 and sort all the fields. This sorting will help us spot the rows which have repeated values for many columns, which will eventually lead to understanding the relationship between the characteristics (columns in dimension table).
5.jpg

Identifying the relationships:
Once the sorting is done we need to look out for the number of values that repeat across the columns. All the records which repeat could have been displayed in a single row with one dimension id assigned if all the columns had same data. The repetition is a result of one or more columns which contribute a unique value to each row. Such columns if removed from the table then the number of rows in the table will come down.

In the below screenshot I’ve highlighted the rows in green that were repeating themselves with new dimension IDs, as only 2 columns SID_ZABNUM and SID_0NPLDA have new values for every row. These two columns having new values for every row have resulted in rest of the columns repeating themselves and in turn increasing the data size in the dimension table. Hence it can be easily said that these two columns do not belong in this dimension tables, so the related characteristics (ZABNUM and 0NPLDA) need to be removed out of this dimension.
Few rows could be found which repeat themselves for most of the columns, but have a new value once in a while for some columns, as highlighted in yellow in the below screenshot. This indicates that these columns share a 1:M relation with the rest of the columns with repeated rows and these could be left in the same dimension.
6.jpg
Conclusion: The columns marked in green belong to this dimension tables and the columns marked in red needs to be in other dimension tables.
7.jpg
Step 2: Create a copy infocube C_ZPFAN and create new dimensions to accommodate ZABNUM and 0NPLDA.
8.jpg
ZABNUM was added to dimension C_ZPFAN8 and 0NPLDA was added to C_ZPFAN7. These were marked as line item dimensions as they have only one characteristic under them.
Analysed the issue with dimension 4 in the similar way and changed other dimensions to help the situation.

Post changes, loaded the data into the copy infocube C_ZPFAN and found the number of records in the dimension table /BIC/DC_ZPFAN2 to be 40286.
9.jpg

Ratio: 40286 / 657400 * 100 = 6.12 %


SAP_INFOCUBE_DESIGNS:
10.jpg

Dimension2 of the copy infocube: /BIC/DC_ZPFAN2
11.jpg
Even now there a few repeated rows and columns, but the ratio is within 20%. We can create up to 13 dimensions, but it is always better to keep a dimension or two free for future enhancements.

Hope this was helpful.


Transaction Codes For Filling Setup Tables LO Extractors
Credits: Todor Peev

An overview of Datasources and the programs filling  the relevant setup table (named MC*SETUP). With this handy table you can  find the status of your current job or previous initialization jobs  through SM37.

T-Code         Purpose
OLI1BW       INVCO Stat. Setup: Material Movemts
OLI2BW       INVCO Stat. Setup: Stor. Loc. Stocks
OLI3BW       Reorg.PURCHIS BW Extract Structures
OLI4BW       Reorg. PPIS Extract Structures
OLI4KBW     Initialize Kanban Data
OLI6BW        Recompilation Appl. 06 (Inv. Ver.)
OLI7BW        Reorg. of VIS Extr. Struct.: Order
OLI8BW        Reorg. VIS Extr. Str.: Delivery
OLI9BW        Reorg. VIS Extr. Str.: Invoices
OLIABW       Setup: BW agency business
OLIB             PURCHIS: StatUpdate Header Doc Level
OLID             SIS: Stat. Setup - Sales Activities
OLIE              Statistical Setup - TIS: Shipments
OLIFBW        Reorg. Rep. Manuf. Extr. Structs
OLIGBW       Reconstruct GT: External TC
OLIH             MRP Data Procurement for BW
OLIIBW         Reorg. of PM Info System for BW
OLIKBW        Setup GTM: Position Management
OLILBW        Setup GTM: Position Mngmt w. Network
OLIM             Periodic stock qty - Plant
OLIQBW       QM Infosystem Reorganization for BW
OLISBW       Reorg. of CS Info System for BW
OLIX             Stat. Setup: Copy/Delete Versions
OLIZBW       INVCO Setup: Invoice Verification

Datasource                          Tcode           Program
2LIS_02*                              OLI3BW       RMCENEUA
2LIS_03_BX                         MCNB          RMCBINIT_BW
2LIS_03_BF                         OLI1BW       RMCBNEUA
2LIS_03_UM                        OLIZBW       RMCBNERP
2LIS_04* orders                   OLI4BW       RMCFNEUA
2LIS_04* manufacturing      OLIFBW       RMCFNEUD
2LIS_05*                              OLIQBW      RMCQNEBW
2LIS_08*                              VTBW          VTRBWVTBWNEW
2LIS_08* (COSTS)              VIFBW        VTRBWVIFBW
2LIS_11_V_ITM                   OLI7BW     RMCVNEUA
2LIS_11_VAITM                   OLI7BW     RMCVNEUA
2LIS_11_VAHDR                  OLI7BW     RMCVNEUA
2LIS_12_VCHDR                 OLI8BW     RMCVNEUL
2LIS_12_VCITM                   OLI8BW     RMCVNEUL
2LIS_12_VCSCL                  OLI8BW     RMCVNEUL
2LIS_13_VDHDR                 OLI9BW     RMCVNEUF
2LIS_13_VDITM                   OLI9BW     RMCVNEUF
2LIS_17*                              OLIIBW      RMCINEBW
2LIS_18*                              OLISBW     RMCSNEBW
2LIS_45*                              OLIABW     RMCENEUB

Document update is where the transaction (documents) are updated in the application tables. This update is normally a synchronous update, i.e. if the update does not go through for what ever reason, the complete transaction is rolled back.
Statistical update is the update of statistics for the transaction – like LIS or extractors for BW.
V1 – synchronous update. If the update is set to V1, then all tables are update and if any one fails, all are rolled back. Done for all transaction plus critical statistics like credit management, etc.
V2 – asynchronous update – transactions are updated and statistical updates are done when the processor has free resources. If the statistical update fails, the transaction would have still gone through and these failures have to be addressed separately.
V3 – batch update – statistics are updated using a batch (periodic) job like every hour or end of the day. Failure behavior is same as V2 updates.

Statistical update is also used as to describe the initial setup of the statistical tables for LO/LIS. When old transactions are updated in LO/LIS as a one time exercise, then it is called a statistical update also. Once these tables are upto date will all transactions, then every transaction is updated in them using V1, V2 or V3.

By Martin Grob


Introduction


In reality dimensions in an InfoCube are often designed by business terms (like material, customer etc.) This often leads to the impression that InfoCube dimensions should be designed based on business constraints. This although should not be the leading criteria and shouldn't drive the decision. 
Aside from the datavolume which depends on the granularity of the data in the InfoCube, performance is very much depending on how the InfoObject are arranged in the dimensions. Although this has no impact on the size of the fact table it certainly has one on the size of the dimensions.


How is a dimension then designed?

The main goal distributing the InfoObjects in their dimensions must be to keep the dimensions as small as possible. The decision on how many dimension and what InfoObjects go where is purely technical driven. In some cases this matches the organisational view but this would only be a conicidence and not the goal.

There is a few guidelines that should be considered assigning InfoObjects to dimensions:
  • Use as many dimensions as necessary but it's more important to minimize dimension size rather than the number of dimensions.
  • Within the dimension only characteristics that have a 1:n relation should be added (e.g. material and product hierarchy)
  • Within a dimension there shouldn't be n:m relations. (e.g. product hierarchy and customer)
  • Document level InfoObjects or big characteristics should be designed as Line-Item dimensions. Line item dimensions are not a true dimensions they have a direct link between the fact table and the SID table. 
  • The most selective characteristics should be at the top of the dimension table
  • Don't mix characteristics with values that change frequently causing large dimension tables. (e.g. material and promotions)
  • Consider also to combine unrelated characteristics it can improve performance by reducing the number of table joins. (you only have 13 dimensions so combine the small ones)

As a help the report (SE38) SAP_INFOCUBE_DESIGNS can be used.
image001.png
This yellow marked dimension should be converted into a line item dimension if it contains a document level characteristic or it is simply bad design.

The maximum number of entries  a dimension potentially can have is calculated through the cartesian product of all SID's. (e.g. 10'000 customer and 1'000 product hierarchies lead to 10'000'000 possible combinations in the dimension table. It's unlikely that this is going to happen and while designing the dimension this should also be considered - analyzing the possibilities of all customers buying all products in this case.
In cases where there is an m:m relationship it usually means there is a missing entity between those two and therefore they should be stored in different dimensions.
Once data is loaded into the InfoCube a check on the actual number of records loaded into the dimension table vs. the number of record in the fact table should be done. As a rule of thumb the ratio should be between 1:10 and 1:20.


Degenerated Dimensions

If a large dimension table reaches almost the size of the fact table when measured the number of rows in the tables it's a degenerated dimension. The OLAP processer has to join two big tables which is bad for the query perfromance. Such dimensions can be marked as Line Item Dimensions causing the database not to create an actual dimension table. Checking the table /BIC/F<INFOCUBE> will then show that instead of the DMID dimension key the SID of the degenerated dimension table is placed in the fact table. (Field name RSSID). With this a join of the two tables is eliminated. Those dimensions can only hold one InfoObject as a 1:1 relationship must exist between the SID value and the DIMID.
Dimensions with a lot of unique values can be set to High Cardinality which changes the method of indexing dimensions. (ORA DB only) This results in a switch from a bitmap index to a B-Tree index.
image001-1.png
Defining a dimension as Line Item Dimension / High Cardinality


Conclusion

Finding the optimal model and balancing the size and the number of dimensions is a delicate excercise.
Dimensions in MultiProvider do not have to follow the underlying InfoCubes definitions. Those can be focused on the end users need and be structrured by the organizations meaning. This does not affect the performance as the MultiProvider does not have a physically existing datamodel on the database.    
Designing the dimension in an InfoCube correctly can have a significant improvement on performance!