The idea is that modeling constructs could be pre-defined and developed to promote rapid time to value, design flexibility, data consistency, and user-friendliness. These building blocks could be collected into libraries, customized, and reused. The building blocks could consist of star schema, analytical data set, data lab, and other types of building blocks along with guidelines for use and mappings to pre-defined data models, such as the Teradata industry logical data models. Different variations and recommended BI tools could also be defined for various building blocks.
What are your thoughts on using a building block approach to build semantic data models?
Thanks a lot for your very interesting question.
To start the discussion, I have added a page on my Database Answers Web Site that summarises my current thinking :-
Thanks for the insight. Even I am very keen to understand the need and use of Semantic layer modellling in Teradata world and what I understand about that is Teradata generally prefers to model the base layers for Data Warehouses using a 3NF model thereby isolating the reporting needs from the DW loading needs which is a reasonable approach. Once the 3NF model is in place then we can start thinking about more user centric layers on top of the 3NF base layer.
These layers are either built as Materialized views (using JI) or physically denormalized layers which are given to the users for their reporting and these are the so called Sematic layers which are named as close to the business names which are easily understood to the users. Bottom line is that the Semantic Layers are more de-normalized in nature.
Please correct me if I am wrong.
And just forgot to mention that I am one of the followers and learners of the wealth of Data models you have posted on your website :)
Your thoughts on building the semantic layer in a Teradata enviornment are very similar to mine.
Given that a warehouse has been implemented in a normalized fashion, how might one create a dimensional model or analytical model or virtually any other kind of model to presents the data in an alternate fashion which might be optimized for data mining or other kinds of analysis?
One way would be to create alternate views of the data. For example, several normalized tables may be joined in a "dimension" view that the client can then access. If the view needs to perform better, Join Indexes on the base tables may be all that is required.
I was once was involved, with a previous employer, on developing pre-defined semantic models optionally provided upon purchase of our product. These models were composed of components similar to such building blocks intended to be mainly re-usable, though at that time they weren’t thought to be easily customizable given that they were implemented as physical tables. The main value was notably for faster time to value.
I am just wondering given that users are now provided with flexibility on design and configuration, then is this approach getting into the area of Agile BI, if not Agile Datawarehousing?
I think that building blocks are really a good idea for Semantic Modelling. In my opinion, it is posible to have a good 3NF as Core and have one or more Semantic Models laying on top of it, because there could be many different ways of seeing the same information. And Semantic Models har more to do with "navigating" the information.
I also think that most business see the same kind of basic information, for example what I mean is that most of retailers want to see "daily sales", "monthly revenue", "stocks turns", etc. And using Teradata's iLDM as the 3NF Core Model could improve the implementation of these Metrics with the utilization of Building Blocks. Maybe a little of customization would be needed but the basics can be delivered in a more standarized and effective way.
I absolutely agree with sbarreda. Semantic layers are a great way of isolating the load process than the usage of the data which is generally designed together :( A 3NF data model gives better performnce while loading the data using Teradata load utilities and the Semantic layers gives better performance in data usage using Teradata best practices like Soft RIs, and other techniques.
People generally counter argue that when we have join indexes then why to use Physical tables in the semantic tables. I still prefer the physical ones given the reasons below.
1. JI take space and so do the physical tables.
2. Physical tables gives me more control on where to use them rather than relying upon the stats everytime to chose JIs.
3. Unlike the Join Indexes I can always query a physical table to get the data if required.
Working with consultants in developing the core DW model and haven't started the semantic layer yet, however, they indicated that they are more familiar (skilled) in creating physical tables from dimensional modeling (star schemas, extended star schemas, snowflakes, etc). Are there any good examples, courses, reading material, etc. where this practice can be transitioned to start creating views in lieu of physical tables in the data access layer of the warehouse? I understand in a few cases you may want to create aggregrate tables to satisfy requirements, but for the most part, this is a performance consideration. Any guidelines on taking this route as well? :-)
I always feel that creating physical tables will be creating duplicate data again. Using views makes more sense I feel. Say for example in a big organization where there are different departments.
Sales department, accounting department, HR department etc may want to see the same data in a different way. A closed business for sales maybe a booked business for accounting. Some departments want data at a bi-weekly ect and others at half-yearly or yearly . Similarly, Things deparments look at products, geography, time etc differ. To answer unanswered questions, it is easy to ahve views.
I advocate using ROLAP rather than MOLAP. Also, existing MOLAP be replaced by ROLAP. Teradata is strong enough to handle it. Of course, it depends how well-versed and business-case fitting a designer/ developer work it out.
Please let me know if you agree with me or not.
Thanks and regards,