Blog

Temporal Data Modeling

ENTARCO Techniques for Modeling Temporal Data

(© 2011, ENTARCO USA Inc.)

This document describes the techniques for modeling data so that the requirements for keeping track of facts of interest over time (past, present, and future) can be correctly represented and are useful for designing implementations of these requirements.

This technique for modeling temporal data described in this paper has evolved over about 20 years of innovation and practical experience. This technique has been deployed in production use in large database environments for about 15 years. In these uses, the techniques are applied to every database table as a standard convention. This technique has proven to be very beneficial in being able to develop logic using a small set of sample logic which can be used for every database access. This approach has a significant impact on improving enterprise data quality, value, and reusability; and reducing application development time and cost.

The techniques described herein are identified by a Temporal Data Types (TDT) and are sequentially numbered for identification purposes.

There are four conditions that exist that determine which technique is to be used. These conditions are:

1. Can the business data content of the entity be changed?
2. Can the entity be ended without a successor?
3. Can there be future entities?
4. Can there be new “past” entities?

The following chart specifies the conditions for each TDT described below.

TDT 1 – This technique is used to model data when an occurrence of the data can be created but it cannot be changed or ended.

There are a few objects that when their existence is recognized, their existence cannot be “deleted” or ended. Enterprise and Person are two examples of these kinds of objects. Therefore, the data that describes these objects must be modeled as dependent entity types to these objects. However, these objects get modeled as independent entity types as follows.

An ENTERPRISE has a unique identifying NUMBER that is not allowed to change and once the ENTERPRISE exists, it cannot be ended.

Note: Due to variations in the definition of “Timestamp” in various environments, the use of “Timestamp” in this article refers a “date/time” attribute that can be set by the program logic.

When a new ENTERPRISE is created, the Date and Time are recorded along with the User ID or program name that created the entity.

Any change to the information that describes an ENTERPRISE will be recorded in dependent entity types that record data over time.

Note: For illustration purposes, only the first eight characters of the “Timestamp” attribute are shown in the examples.

TDT 2 – This technique is used to model data where the data is allowed to be changed in a dependent entity type and to maintain the history of previous attribute values or relationship memberships.

In this case, the CREATE_DATE_TIME is recognized as the date and time that the fact was “effective”, i.e. valid or true. For cases where the EFFECTIVE DATE TIME is, or may be, different than the CREATE_DATE_TIME, see TDT’s 3 thru 6.

 

 

(Business Data Rule: ENTERPRISE always ASSIGNED One (at a time) ENTERPRISE_REGISTRATION_STATUS_TYPE)

When a new entity is created the CREATE DATE TIME is used as part of the unique identifier, the DEACTIVATE DATE TIME attribute is set to null (or empty), and the DEACTIVATE USER CODE is empty. 

 

When the ENTERPRISE REGISTRATION STATUS for the ENTERPRISE changes, a new entity with the latest ENTERPRISE REGISTRATION STATUS TYPE is created. The previous entity is updated with the DEACTIVATE DATE TIME set to the new entity’s CREATE DATE TIME and the DEACTIVATE USER CODE is set to the new entity’s CREATE USER CODE.

 

Specifying the ENTERPRISE NUMBER and a comparison date time will retrieve the ENTERPRISE REGISTRATION STATUS for an ENTERPRISE. The comparison date time will be between the CREATE DATE TIME and the DEACTIVATE DATE TIME where the comparison date is greater than or equal to the CREATE DATE TIME and (the DEACTIVATE DATE TIME is null or (the DEACTIVATE DATE TIME is not null and the comparison date is less than the DEACTIVATE DATE TIME. A comparison date time of 6/15/2003 will retrieve a status of “Complete” and a comparison date time of 4/30/2003 will retrieve a status of “Incomplete”.

TDT 3 – This technique is used to model data where the requirement is to record changes in dependent entity types, to be able to represent effective date ranges, maintain the history of previous attribute values or relationship memberships, but future effective dates are not allowed, and the data cannot be ended without a successor.

This example records changes in ENTERPRISE ECONOMIC ACTIVITY STATUS which is a dependent entity type of ENTERPRISE that records changes in relationship membership with an ENTERPRISE ECONOMIC ACTIVITY STATUS TYPE. The EFFECTIVE DATE TIME records when the ENTERPRISE ECONOMIC ACTIVITY STATUS became effective. An ENTERPRISE must have an ENTERPRISE ECONOMIC ACTIVITY STATUS TYPE assigned, so there is no END DATE TIME.

 

Because the status is not allowed to end, this example will require system logic to ensure continuity across time from when the status is first effective onward. The EFFECTIVE DATE TIME and the SUPERCEDED DATE TIME are used to record the effective date range of the information. When a change is recorded, the previous outdated information will have its SUPERCEDED DATE TIME set to the new EFFECTIVE DATE TIME. In the most recent occurrence, the SUPERCEDED DATE TIME will have a NULL (empty) value. The CREATE DATE TIME and the DEACTIVATE DATE TIME function as they did in the form explained above and are used to record the audit trail of when data changed in the system.

 

Specifying the ENTERPRISE NUMBER, an effective comparison date time, and an audit comparison date time will retrieve the ENTERPRISE ECONOMIC ACTIVITY STATUS TYPE for an ENTERPRISE. The audit comparison date time will be between the CREATE DATE TIME and the DEACTIVATE DATE TIME and the effective comparison date time will be between the EFFECTIVE DATE TIME and the SUPERCEDED DATE TIME where the comparison date is greater than or equal to the CREATE DATE TIME and (the DEACTIVATE DATE TIME is null or (the DEACTIVATE DATE TIME is not null and the comparison date is less than the DEACTIVATE DATE TIME.

TDT 4 – This technique is used to model data where the requirement is to record changes in dependent entity types, to be able to represent effective date ranges, maintain the history of previous attribute values or relationship memberships, and future effective dates are allowed, but the data cannot be ended without a successor.

This example records changes in ENTERPRISE NAME which is a dependent entity type of ENTERPRISE that records changes in attribute value. The EFFECTIVE DATE TIME records when the ENTERPRISE NAME became effective. It is possible to receive a name change that will become effective in the future. An ENTERPRISE must have an ENTERPRISE name, so there is no END DATE TIME.

 

 

 

Other than allowing a future effective date, the logic for storage and retrieval is the same for this temporal data type as for TDT3 above including the requirement for additional logic to ensure continuity across time from when the name is first effective onward

TDT 5 – This technique is used to model data where the requirement is to record changes in dependent entity types, to be able to represent effective date ranges, maintain the history of previous attribute values or relationship memberships, and future effective dates are allowed and the data can be ended.

The example below for ENTERPRISE TRADE NAME is a situation where the dependent entity type for ENTERPRISE records changes in an attribute value with an effective date range that can end. It is possible for an ENTERPRISE to stop using a trade name. Instead of SUPERCEDED DATE TIME, the effective date range will be between EFFECTIVE DATE TIME and END DATE TIME.

 

 

 

Similar logic is used for both storage and retrieval as in the examples above. END DATE TIME has the same function logically as SUPERCEDED DATE TIME in “TDT3” above. Because a trade name can be ended, there is no need for logic to enforce continuity across time. A separate document contains the detailed information of how the system should maintain the date values to preserve the history for both the effective date range and the audit trail.

TDT 6 – This technique is used to model data where the requirement is to create the data and to be able to end its use.

This technique is most often used to model data that is used to classify and categorize other data. These entity types are normally maintained by a data administrator and populated using a utility. The only change to this kind of entity type would be to end its use.

 

 

 

Since there may be situations where the entity is entered by mistake, the CANCEL DATE TIME and CANCEL USER CODE attributes are used to indicate that this entity was cancelled and should be ignored. The CANCEL DATE TIME will always contain a null (empty) value unless it has been cancelled. When retrieving a list of valid status types, the comparison date time will be between the EFFECTIVE DATE TIME and both the END DATE TIME and CANCEL DATE TIME.

Canceling an entity is only allowed when it has not yet been used in an association with a dependent entity type in a relationship membership. If the entity has been used in a relationship membership by a dependent entity, then the entity should be ended instead of cancelled by setting the END DATE TIME.

In some cases, the dependent entity type will itself be used to classify and categorize other data. If this is the case, then it would be acceptable to cancel the entity as long as the dependent entity has not itself been used in a relationship membership with a downstream dependent entity type. In this situation, the dependent entity should be cancelled if it has a CANCEL DATE TIME or deactivated if it has a DEACTIVATE DATE TIME since its parent entity has been cancelled.

Similarly, if the END DATE TIME is set in this type of entity that is used to classify and categorize other data; then the related downstream dependent entity types that are used in the same manner should also have their entities ended if they are related to the parent entity that has been ended. This cascading effect of ending or canceling an entity should stop when the downstream entity type records the transactions that happen in the real world as opposed to being used to classify and categorize other data.

NOTE: In the above examples where SUPERCEDE DATE TIME, END DATE TIME, and DEACTIVATE DATE TIME they are specified as being “Optional” attributes. However, if the selected DBMS does not handle these fields correctly and you need to make them “Mandatory”, use a maximum date value such “01/01/3000” and then reset it to the actual data/time when updating a row. This approach is actually easier to code for than using the “Optional” attribute approach.

This approach to modeling temporal data is relatively new. Therefore it is not well understood so very few know how to do it, and thus it will likely be hotly debated. This often results in a rationalization, however false, that “if I don’t understand it or I don’t know how to do it, it must not be important or necessary; and therefore you don’t need it”. The underlying motivation here is that if you want something and I do not know how to do it or provide it, you do not need it.

In 2003, Chris Date, who is considered (and has been for many years; decades, as a matter of fact) one of the leading thinkers and the leading author on the subject of data modeling and database design, published a book on temporal data which was probably the first book that dealt with the issue of keeping track of facts over time. We, at ENTARCO, have been developing and maturing the techniques for modeling data to keep track of every fact over time (temporal data) for about twenty years. We have much empirical evidence of the enterprise business need, use, and value of universal and consistent temporal data to the point where the enterprise business users that now have it extensively implemented would not give it up. It would be their highest priority criteria for any data model and database.

There is also a case for saying that the data model cannot be properly normalized without providing for keeping track of every fact over time. To properly identify an attribute value, we need to know when it came into existence and/or when was it a fact, valid for use, or true.

The use of this technique has resulted in the following benefits. By applying the technique universally and consistently with a specified set of patterns:

A. It provides an enhanced capability to identify and resolve issues because you can universally
and consistently know “who did what, when” or more simply, “what happened when” which is
fundamental to identifying and resolving many operational issues.

B. It provides a very useful means to make assessments about the quality and currency of the data.
Because data can deteriorate with age, this technique provides a means to always know when the
data was created and how old it is.

C. It dramatically reduces the amount of time and effort to deal with the issues of keeping history.
The result is we always have the history. Often when history is not captured when it is available
it cannot be re-constructed.

D. It provides the ability to take accurate snapshots of the status of anything at any point in
time that provides the ability to see what was true in the past and when, what is true now, and
to the extent that future data is entered, you can see what we expect to be true in the future.

E. It dramatically facilitates the ability to time phase the data for analytical and reporting
purposes. This inherently supports classifying data about anything of interest to the enterprise
people by any time dimension such as:

1. Daily, weekly, monthly, quarterly, annually, semi-annually, bi-annually;
2. Week-to-date, month-to-date, year-to-date, quarter-to-date, inception-to-date,
3. Being able to compare day-to-day, week-to-week, month-to-month, year-to-year data;
4. Or any combination of the above.

From a more technical and operational perspective there are some very useful operational benefits of using temporal data. If all business data is kept as temporal data, using inserts where updates are only used to show that older data has been superseded by newer data, then some very costly issues are immediately resolved. Delete authority should not be granted on database tables since only insert, update and select are required. Updates are allowed to supersede previously-known knowledge. This eliminates the need for part of the RI, the second part for delete restrict. Because of this feature of the approach, then the indexes required to support delete restrict for foreign key constraints are not required. In addition, batch processes can now become background processes running in the same environment as online processes in the foreground as long as frequent commits are taken. Since business data is never updated and lost, then backups are not required before beginning batch processes and data entered by online processes will not be lost since recovery does not require restoration to a prior backup. This eliminates the need for a batch window or for separate copies of databases where online processes are only allowed to access data as of the end of the last batch processing.

 

Appendix A, MEA Standard Attribute Definitions

This appendix contains the Methodology for Enterprise Architecture standard definitions for the standard attributes used in keeping track of data over time. These attributes are used in combinations as specified by the Temporal Data Types (TDT) specified in the Techniques for Modeling Data Over Time document.

CREATE DATE TIME

CREATE DATE TIME is the date and time this occurrence was entered on the database by the APPLICATION USER (or BATCH JOB).

ADDITIONAL INFORMATION: This date is a system supplied CURRENT TIMESTAMP and is the date and time that the data was inserted on the database. This date is also considered to the date and time that the enterprise knew about the data unless there are other dates such as RECEIVED DATE, POST MARKED DATE required and used to acknowledge when the data was known to the enterprise.

CREATE USER CODE

CREATE USER CODE is the code assigned to the APPLICATION USER (or BATCH JOB) who created this entity.

EFFECTIVE DATE TIME

EFFECTIVE DATE TIME is the first date and time that this occurrence of *entity type name* is valid for use.

ADDITIONAL INFORMATION: Typically, the date will be displayed in the user interface without the time unless the situation requires it and when the date is entered without the time, the time will default to 00:00:00.

END DATE TIME

END DATE TIME is the last date and time this occurrence of *entity type name* is valid for use.

ADDITIONAL INFORMATION: Typically, the date will be displayed in the user interface without the time unless the situation requires it and blank if the value is null (or equal to a maximum date/time) value. It the date entered by the user is greater than the CURRENT DATE, the time will default to null (or a maximum date/time) value for the date. If the date entered is equal to the CURRENT DATE, then the user is asked “Do you wish to end this entity immediately?” If the response is “Yes”, then END DATE TIME is set to the CURRENT TIMESTAMP. If the response is “No”, the time will default to null (or a maximum date/time) value.

DEACTIVATE DATE TIME

DEACTIVATE DATE TIME is the system date and time this occurrence became not valid for use.

ADDITIONAL INFORMATION: In a new entity, DEACTIVATE DATE TIME will be set to a null (or maximum date/time) value. When an entity is superceded by a new entity the DEACTIVATE DATE TIME in the old entity will be updated to the value of the CREATE DATE TIME in the new entity.

DEACTIVATE USER CODE

DEACTIVATE USER CODE is the code assigned to the APPLICATION USER (or BATCH JOB) who deactivated this entity.

SUPERCEDE DATE TIME

SUPERCEDE DATE TIME is the last date and time this occurrence of *entity type name* is in effect.

ADDITIONAL INFORMATION: This attribute is not a logical business attribute. SUPERCEDE DATE TIME is used in lieu of an END DATE TIME in an entity type that has a mandatory relationship with a parent entity type that cannot end and therefore cannot exist without an occurrence which is in effect. When the entity is created the attribute value should be set to a null (or maximum date/time) value. When a new entity is created with later information, the value of SUPERCEDE DATE TIME should be set to the value of the new entity’s EFFECTIVE DATE TIME.

CANCEL DATE TIME

CANCEL DATE TIME is the system date and time this occurrence was cancelled by the APPLICATIONS USER (or BATCH JOB).

ADDITIONAL INFORMATION: In a new entity, CANCEL DATE TIME will be set to a null (or maximum date/time) value. Cancelled means that the entity was created in error, should not have existed and should not ever by displayed as valid value. This attribute can only be used if the entity has not been referenced in a relationship to another entity. If the entity has been referenced by another entity, then the entity can only be ended by using the END DATE TIME attribute.

USER CODE

USER CODE is the code assigned to the APPLICATION USER (or BATCH JOB) who cancelled this entity.

Douglas T. Erickson

President, ENTARCO USA Inc.

© 2011, ENTARCO USA Inc.

Enterprise Management Services

“How much does Enterprise Management Services cost and long does it take to implement?”

My response to this question is, “How much has it cost and how long has it taken to implement Information Technology (IT), or Information Systems (IS), or before that, Data Processing (DP)?”  The answer to this question is, “Well, IT is not a project and you don’t really implement IT.  IT is a function that has many deliverables, and each deliverable has its own cost and schedule”.  So it is with Enterprise Management Services.

 To put this question in perspective, let me ask you a question.  “How much does your enterprise spend on IT each year…an average?”  Now, multiple that number by 10 to get a sense of how much traditional IT has cost your enterprise over the last 10 years.  Adjust the average per year to compensate for changes over the last twenty years and then see what the approximate 20 year cost is.  Consider this, it everything stays the same what will your enterprise spend over the next 10-20 years.  Now consider what you have, how much it cost, and how long it took to get that which you now have.  Is this what you and your enterprise wants or needs to look forward to over the next 10-20 years?

Now, consider this.  Gartner Group forecast in 2010 that Application Overhaul will be the critical IT strategy for the next 10 years.  They also go on to say that,“For many organizations, management of the application portfolio has largely been focused on adding applications according to business requirements. In many cases, little attention was paid to overlapping functionality or cost of integration.”  

There is now an alternative that will cost a lot less and take a lot less time…Enterprise Management Services, the ENTARCO Methodology for Enterprise Architecture (MEA) way!

How much faster, better, and cost effective you ask?

  1. Up to an 89% reduction in applications development cost over conventional approaches.
  2. Up to 80% reduction in development time and effort.
  3. At least half the enterprise data reused at least 15 times.
  4. Extensive reuse of applications logic.
  5. Approaching 100% alignment with business requirements on first implementation.
  6. Dramatically lower maintenance cost, effort, and time.
  7. Unparalleled flexibility and adaptability to accommodate change.

So, you may ask, how is that possible? And, assuming that it is possible, how do you do that?  The following discussion will provide with some insight into the answer to these questions.

Getting Started

First, we develop what we call an Enterprise Architecture Plan.

For those of you familiar with the Zachman Framework for Enterprise Architecture, this planning effort develops the enterprise artifacts (models) specified in Rows 1 and 2 of that framework.

This planning effort is performed using the ENTARCO MEA and it takes from 6 to 10 months, depending on the complexity of the enterprise.  Complexity of an enterprise is a function of the variety and diversity of the products and/or services of the enterprise.

ENTARCO fix prices its services to lead and facilitate this effort.

This plan will contain the twelve models which will comprise the high level architecture of the enterprise and it will be complete and stable in terms of defining the scope and context of the enterprise.

These twelve models will be the rational guidance for the planning, identification, and scope definition for engineering and manufacturing highly aligned, integrated, agile, non-redundant, shareable enterprise databases and applications.  This will replace the rather random, fragmented, and arbitrary approach to identifying enterprise applications and application database projects that has characterized IT planning in the past.

The MEA uses a concept that says that every enterprise has three categories of resources that it manages through a resource life-cycle.  The three categories of resources are:

1.  Primary resource – Revenue Generator = Product and/or Services

2.  Supporting resources – Cost Generators = Labor, Money, Equipment, Material, Supplies

3.  Aggregate resources – Allocations of Primary and Supporting Resources for Management Purposes = Organization, Facility, Project

The resource life-cycle consists of four phases.

1.  Requirements – this phase consists of the processes performed to plan what resources and how much of those resources will be required.

2.  Acquisition – this phase consists of the processes performed to acquire the resources.

3.  Stewardship – this phase consists of the processes performed to inventory and take care of the resources in the possession of the enterprise.

4.  Disposition – this phase consists of the processes performed to end the stewardship of resources in the possession of the enterprise. 

The enterprises products and/or services determine the supporting resources employed by the enterprise.

Resources managed through the life-cycle determine the enterprise logical business processes.

Objects managed by the logical business processes determine the enterprise data requirements.

The use of these concepts guarantees a complete and very stable foundation on which to architect an enterprise; and on which to plan the logical sub-systems (components) of the enterprise that will need to be put in place to have a functioning enterprise.  Therefore, this is a very rational basis for identifying enterprise logical business processes and the corresponding applications to support those business processes and the databases to provide for the enterprises data requirements.  As John A. Zachman would say, “This is the way to rationally identify “slivers” of the enterprise for architecting, engineering, and manufacturing an enterprise.  This becomes a very objective basis for allocating and prioritizing resources for the architecting, engineering, and manufacturing the enterprise.

 Secondly, we proceed with the implementation of the Enterprise Architecture Plan

After the Enterprise Architecture Plan is completed, which is the guidance for managing the Enterprise Management Services function, individual projects for the architecting, engineering, and manufacturing the subsystems that constitute the system know as the enterprise, can proceed.

The schedule for each of these efforts will vary depending on several factors.

1.  The size and complexity of the business process being architected, engineered and manufactured will be the primary determinate of the time and cost of the project.

2.  A significant factor in architecting, engineering, and manufacturing an enterprise sub-system is the availability of reusable data and process logic.  Accordingly, the time and cost of the first few sub-systems will be impacted if they are significant “data users” as opposed to “data creators”.  If the early implementations involve using a lot of data that they don’t create, that will extend the time and cost of those implementations.  However, with the use of the MEA, this turns out to be an investment that has a fairly quick payback. Because any “data work” done for any one implementation is available for reuse on all subsequent implementations all subsequent implementation times and costs are reduced.

3.  As more enterprise sub-systems are implemented and more data and application logic is available for reuse, the time and cost for each implementation will decline relative to earlier implementations.

4.  Because of the shared/reuse of data, we dramatically reduce time and cost of development by significantly reducing, or even eliminating the need for data conversions and interfaces

5.  Schedules are always a function of time and available resources.  To a limited extent, the time it takes to implement a component of the enterprise can be adjusted by increasing or decreasing the resources allocated to the effort.

For these reason, we know from experience that the time and cost to implement enterprise databases and applications are continuous reduced because each implementation provides a highly aligned and integrated sub-systems within an overall enterprise architecture.  As such, each implementation leverages the investment made in previous implementations through its reuse of sharable data and application logic.

Now, for more on how we do EMS the ENTARCO MEA way.

First of all, we borrowed some concepts, principles, and practices from architecting, engineering and manufacturing disciplines that dramatically changed engineering and manufacturing outcomes over the last 100 years or so and applied them to the architecting, engineering, and manufacturing of an enterprise.  A sampling of these concepts, principles, and practices are:

1.  We changed the approach to architecting, engineering, and manufacturing an enterprise from a “job-shop” approach to a much more advanced “custom finished product assembled from standard components built to inventory, for immediate (almost) delivery” approach.  (By the way, many industries go through an intermediate phase called “standard products built to inventory for immediate delivery”, but since we had the benefit of having been in the job-shop world and had already seen the “custom finished product” world, we accelerated the evolution a bit!)

     It turned out that when we applied this approach to the architecting, engineering, manufacturing of an enterprise, we got a really big bonus.  We do not have to build all the parts needed for every product we are to deliver.  In our world, we can build the “parts” (data and process logic) just once and reuse it lots of times!

2.  We have applied Six-Sigma Quality concepts to the entire process of architecting, engineering, and manufacturing an enterprise.  This includes applying principles such as:

a.  Design the architecting, engineering, and manufacturing processes to eliminate defects.

b.  When a defect is discovered, fix it then.  Do not let the defects get delivered to the customer.  Also, fix the process to prevent the defect from occurring again.

c.   Use quality parts to build quality products.

d.  Remember: Quality if free, it is the lack of quality that is expensive!

3.  Strive to reduce cycle-time.  By continuing to reduce the cycle-time of every step in the process, we continuously reduce the opportunities for defects to occur and cost to be incurred.

Secondly, we stop doing a lot of things that take time, cost money, and cause defects.  We stop doing the same data and process analysis, design, and development over, and over, and over, and over again.  How do we do that?

We have learned how to build and maintain complete, correct, and stable yet flexible enterprise models of the data and business processes, and we build them once and reuse them lots of times.  We collect and store one fact (piece of data), one place and reuse it lots of times.  Our experience has proven that over half of the data in an enterprise is used at least an average of 15 times.  We don’t identify and design half of an enterprises data 15 times or more and then store it 15 times or more and make the enterprise maintain it 15 times of more.  That also means that we do not have to design and write the program logic 15 times or more to store, maintain, and retrieve the data 15 times or more…we just call it and execute it again.  We do not copy the code and drop in another program.  It the code needs to change for some reason, we only have to change it one place and we are done!   This is one way we can dramatically reduce the cost and time traditional approaches incur in developing and maintaining applications and databases.  What many people haven’t figured out yet is that the data an enterprise needs is pretty finite, but its use is very dynamic.  People really, really like to use data over, and over, and over again.  As a matter of fact, this concept changes data from being an expense to the enterprise into an asset that continues to provide a very high return on investment!

Caution: You have to know how to identify and design enterprise date so that it is complete, correct, available, and accessible in order to achieve these results.  That is what ENTARCO has proven to be really, really good at!

Thirdly, since we build business process models and we management those models and the applications logic that is designed and developed to perform the business processes, it is relatively easy for us to identify chunks of logic that can be reused….over, and over, and over!  Also, since the logic that is being reused is already tested and in production use, we don’t have to design it and test it, over, and over, and over again!

Fourth, we learned how to get very correct business requirements right the first time.  This results in much shorter development cost and time, less rework (rework always costs more and takes more time; and nobody likes doing it either) and a lot less maintenance cleaning it up after it is put into so-called “production”.  We actually design the enterprise’s business data and processes before we start what traditionally is called systems analysis or design.  The time and cost to design enterprise business data and processes is small compared to the cost and time of designing and building applications and databases.   We also figured out how to identify and design enterprise data and business processes independently and in parallel so that by the time we are done with both, they are both reconciled and complementary to each other.

We design the enterprise databases (which are non-redundant, shareable, with one fact, one place) before developers start writing the code.  Therefore, the code that they write referencing the data (database tables and columns) is already determined to be correct, complete, and stable thereby practically eliminating any rework due to database re-design.  This also is a major factor in achieving massive data reuse throughout the enterprise.  Also, once the data required by the application is determined, we can quickly identify the data that is already in production and tell the developers which data to use and also the logic they can call to store and retrieve the data from the databases.  Oh, and in case you are interested, we have had less than 5 cases of a “performance problem” which were solved with little compromise in over 15 years of production operations.

We design the enterprise data and the supporting enterprise database(s) to keep track of every fact over time without the use of history tables.  Our very advanced temporal data modeling conventions provide for keeping track of what was true when (past, present, and future), and when did we know it.  This quite naturally provides us the capability to use the data in its natural state to do all sorts of time-phase analysis such as year-to-date, inception-to-date, detail and summary comparisons; second, minute, hourly, daily, weekly, monthly, quarterly, semi-annual, annual summaries and comparisons, solve business and customer issues by being able to know and explain what happened when, who did it, where, and when did we know it.

These conventions and techniques dramatically improve the availability of high quality enterprise data, dramatically reduce the cost of using the data including reducing the cost, time, and effort to develop and sustain data warehouses and data marts when they are, in fact, appropriate and necessary.

Because we successfully reduce the volume of data by eliminating redundant data which is collected and stored, we reduce the cost of storing and maintaining the data.  More importantly, our experience shows that our data modeling approach initially identifies more required elements of data than is discovered through traditional or conventional methods of analysis and design!  This helps us promote the reuse of the data, eliminate data redundancy, and eases the path to program logic reuse.

We also reduce time and cost of development by significantly reducing, or even the eliminating the need for data conversions and interfaces.  As an enterprise progresses through the development and implementation of more and more applications designed and constructed using the MEA; and the sharing (reuse of existing production data) of data becomes more extensive, the need for data conversions to populate new applications databases is eliminated.  For the same reasons, the need to develop and maintain data interfaces among applications databases is nearly eliminated.  (We have a case where four major applications have been built using the MEA and we have not built one data interface because the production data is shared by all the applications.  This dramatically reduces the cost, time, and effort to develop and maintain enterprise applications and databases.

Caution: Yes, the initial MEA developed applications will require some data conversions and data interfaces to work within the existing applications portfolio.  However, over time, as more and more applications are overhauled, modernized, or renewed, these interfaces can be eliminated.  This is a situation where we provide a path out of the data conversion/data interface abyss, whereas the conventional or traditional approaches just keep making this abyss bigger and deeper.

Another way in which we can significantly reduce the cost and time to deliver capability to an enterprise is through our standard data and process models and database designs and application logic.  Because of our extensive experience, we can now offer our customers “production ready” components of data and related functionality so that you don’t have to “re-invent the wheel” (data and program logic) again!  This enables our customers to focus on those architectural components that are unique to them.

The big, strategic benefit of using the MEA is that fairly quickly, it will become faster and much more cost effective to replace existing applications and application databases that it will to continue to enhance, maintain, and operate them.  The reason this result is achieved is because of the ever-increasing amount of reusable and shareable data and application logic; and the elimination of the need for data conversions and data interfaces dramatically reduces the cost of the next development and implementation effort.  Yes, folks, it can and does keep getting better!  This is a major reversal of the trend that is being experienced in the IT world today!

 Douglas T. Erickson

President, ENTARCO USA Inc

©2011, ENTARCO USA Inc.

P.S. Implementing EMS is not a project.  EMS is a transformation of IT for the 21st century.  Just as traditional IT had many deliverables, EMS will have many deliverables.  However, how those deliverables are architected, engineered, and manufactured will be dramatically different, compared to the legacy produced by what we know as IT.  The difference is, EMS, the ENTARCO way, will deliver a lot faster, better, and much more cost effectively.

 

Enterprise Management Services: An Introduction

The labels we assign to things have a lot to do with how we perceive them.

In the early days of IT we often called the function and/or department that designed and built automated information systems ‘Data Processing’.  Over the ensuing decades, the label has generally evolved from Data Processing, to ‘Information Systems’, to ‘Management Information Systems’, to the current ‘Information Technology’.

Our attraction and fascination with technology tends to blind us from looking at the basic pervasive problems of the IT industry.  We seemed to have evolved to the point where there is an almost dogmatic belief that our economic and social wellbeing is totally dependent on developing and/or owning “the latest and greatest technology”.  If you have a problem, technology will solve it…. guaranteed”.  The situation has even progressed to where almost any new innovation or invention is called a “technological advancement”.

May I dare say that a new idea or advancement in the way we do something more efficiently or effectively may not involve a “technological advancement”!  The Information systems industry has become so obsessed with technology that it now almost universally likes to be known as the “Information Technology (IT)” industry or profession.  Developing a better way to identify and design business processes, or developing a better way to model data, does not mean that a new or improved technology has taken root.

Accountants count things.  Engineers design things. Construction companies build buildings. Insurance companies insure things. Doctors practice medicine. These people or enterprises may employ automation and IT to do what they do; however, their knowledge and skill precedes the advent or use of technology to perhaps improve their productivity or quality of output. 

I will suggest that the people currently working in what we commonly refer to as the “IT world” deliver—or should deliver—applications and databases that provide the infrastructure and the services to support the effective and efficient operation and management of the enterprise.  I would even go as far to say that we should call this function “Enterprise Management Services”, not Information Systems or Information Technology.

It is not enough, of course, to just change the label; unless we are just trying a new marketing angle because the old label has become passé or the thing that the label refers to is defective or doesn’t work anymore!  Sometimes a new name/label helps people to think about something they know differently; and more importantly, to set forth a change, a re-invention or renaissance if you will, of what we already know—such as what we know today (i.e. 2011) as “IT”.  Perhaps the label, “Enterprise Management Services” would better communicate what enterprises need and what contemporary IT should be striving to deliver.

As we know, there is an IT industry that designs and builds information technology—computers, operating systems, communications hardware and software, telephones, storage devices, etc.  However, just as carpenters are very knowledgeable about carpentry tools and their use, carpenters are not called “Carpenter Technology”; they are called “Carpenters”.  As a matter of fact, if you want a house, you typically would not go to just a Carpenter, you would go to an “Architect”, who would design your house, and then eventually to a “Home Builder” who would construct your house.  This is because a house requires more than a Carpenter; it also requires an Electrician, a Plumber, and maybe even a Landscaper.   A Home Builder (a.k.a. General Contractor) integrates all of these trades for the purpose of building you a house.  A house is like a system with several sub-systems, integrated to function cohesively as a house.

An enterprise is also a system comprised of several sub-systems which should all be functioning in a cohesive, integrated manner.  As we are slowly coming to understand, an enterprise—especially an efficient, effective, high-performance enterprise—needs to be architected, designed, and constructed to be an efficient, effective, and high-performance enterprise.  Therefore, what are needed are concepts, a function, and an organization capable of acting as the “General Contractor for the Enterprise”. 

Some people may assert that “management” is, or should be, considered as the enterprise’s General Contractor.  However, under the current approach—either through IT itself, or through the actions of the enterprise management—the existing concepts, functions, and organizational structures often result in a different “General Contractor” being assigned responsibility to “build” each room in the “house” (i.e. the enterprise).  Even more problematic, each “General Contractor” tends to have different capabilities, skills, competencies, and motivations.  This situation tends to result in rather inconsistent, fragmented enterprises.

It is also important to note that enterprise managers are enterprise operators, not enterprise architects.  Airplane architects and engineers design and build airplanes; under normal circumstances, they do not “operate” them.  Building Architects design and guide the construction of buildings and, again under normal circumstances, they do not occupy nor “operate” the buildings.  “Operating” is not the same as “architecting”, “designing”, or “building” something!

I believe that a major step forward would be to have a well-developed enterprise data and business process architecture.  This architecture should be based on concepts and principles that would ensure that the enterprise architecture and its physical manifestation in the form of enterprise databases and applications that were designed to be empirically sound, comprehensive, stable, and capable of serving the needs of the “enterprise operators” (i.e. management) over a long period of time.

Consequently, perhaps the moment has arrived to move beyond IT and call the function that architects, designs and constructs the enterprise; and that provides the “services” needed by the enterprise management, the “operators of the enterprise”—Enterprise Management Services.

Enterprise Management Services is the function most recently known as the Information Technology function, which used to be known as Information Systems, and before that Data Processing.  However, it is very much evolved beyond the current state of what is currently known as Information Technology.  However, Enterprise Management Services is much evolved beyond its previous incarnations.

Enterprise Management Services is qualified and capable of architecting an enterprise from the ground up.  IT, and its predecessors, basically designed and built pieces of an enterprise without the guidance of an enterprise architecture, one piece at a time based on some organizational sponsorship; one after the other, over a long period of time.

Enterprise Management Services is qualified and capable of designing and developing the applications and databases that are architecturally consist with the enterprise architecture; not compromised by arbitrary actual or assumed direction or funding influences which are at play in the IT world we know today.  

I will post some additional thoughts in the weeks to come regarding ENTARCO’s vision for Enterprise Management Services going forward.  In the meantime, I look forward to your comments, ideas, and feedback!

Douglas T. Erickson

President, ENTARCO USA Inc.

© 2011, ENTARCO USA Inc.

 

To integrate, or not to integrate?  That is the question.

 Enterprise integration has been an enduring issue for business and IT managers for at least thirty years.  How can such a fundamental issue persist for so long?  Perhaps it persists because:

  • We don’t have a sound, shared definition of what we mean by enterprise integration or disintegration, and 
  • We do not fully understand the root causes of enterprise integration or disintegration.

I have some thoughts on what enterprise integration and disintegration is and what causes enterprise integration and disintegration to occur.

According to the dictionary, to “integrate” means to combine one thing with another so they become a whole and bring into equal participation in, or membership in, society or an institution or body.

Therefore, an enterprise is integrated if its parts are combined in a manner which provides for their participation and contribution in a cooperative, mutually supportive manner.  This further means that there is an absence, or a minimization of discord, conflict, fragmentation, redundancy, or competition among the constituent parts of the enterprise.

The basis for enterprise integration should be the enterprise’s business processes, as opposed to its business functions (i.e. Accounting, Marketing, Engineering, Manufacturing, etc.) or its organizational units (i.e. Accounts Receivable, Accounts Payable, Eastern Sales Office, Western Sales Office, etc.)

The first barrier to integrating an enterprise is the obvious attention given to addressing the issue from either a functional or an organizational perspective.  The functions of an enterprise are quite stable; however, a function may perform many processes, and a process may be performed by many functions, thereby creating inherent redundancy of processes.  Organizations, on the other hand, are less stable than functions.  Organizations may also perform many processes, and a process may be performed by many organizations, thereby creating inherent redundancy of processes.

I am of the belief that enterprise integration must be approached based on what I call the “logical business processes”, which, in my opinion, are the most fundamental, stable structure for establishing the foundation for an enterprise. 

The following diagram shows the relationship between Function, Organization, and Process:

The above diagram, in the form of a relational data model, specifies the following: 

  • Organization, Function, and Process are separate, distinct, and independent of each other.  This means that each of these business objects is independent variables.  Therefore, none are subordinate to another and each must be identified and defined separately. 
  • Function Organizationspecifies the relationship between instances of a Function and an Organization:
    • An Organization may perform one or more Function; and
    • A Function may be performed by one or more Organization.
  • Ø Organization Process specifies the relationship between instances of an Organization and a Process:
    • An Organization may perform one or more Process; and
    • A Process may be performed by one or more Organization.
  • Ø Function Process specifies the relationship between instances of a Function and a Process.
    • A Function may perform one or more Process; and
    • A Process may be performed by one or more Function.

The critical point here is that Process is not a decomposition of Organization or Function. 

The Root Cause of Enterprise Disintegration

Most people commonly define Process within the context of an Organization or a Function.  This practice is the primary, root cause of redundancy in the identification of processes.  This flawed practice leads to redundant Process specifications which get implemented in various applications, AND this, in turn, leads to redundant and inconsistent functionality across the entire portfolio of enterprise applications. 

This very defective result happens because most analysis starts with IT Analysts asking people who work in an organization to identify and define their “business requirements” and they do so within the context of their organization.  Another major contributor to this problem is the sponsorship of IT information systems projects by organizational or functional units of the enterprise which often restricts or constrains the project scope.  Additionally, if management decides to change the organization structure—which they do!—the scope and context of a project, and the associated information system(s), will be subject to change more frequently and readily than if the project scope and context are defined by a logical business process.  This situation lays a foundation which is guaranteed to result in enterprise disintegration.

Over time, gaps and redundancies in the implemented “systems” will occur, whether they are manual or automated, and this will cause the disintegration.  Once “systems” become automated, it becomes increasingly expensive and time-consuming to try to overcome disintegration.  The applications, and the business processes they support, will become inconsistent and data redundancy will become rampant.  Then you will have to build data warehouses, cleanse the date, reconcile inconsistencies, and add additional maintenance resources to help deal with the disintegration!  

A Path to Enterprise Integration

The product or service produced by the enterprise—its primary resource—dictates which business processes are required by the enterprise.  It does not dictate how the business processes are to be performed.  The product or service of the enterprise also dictates the supporting resources required to provide the product or service to the market.  Logical business processes are the elementary set of business processes that must be performed to provide a product or service to a market.  Logical business processes are identified and defined by analyzing how resources are managed through their life-cycle (i.e. requirements, acquisition, stewardship, and disposition).  Consequently, the resulting logical business process architecture will be very stable, complete, and non-redundant, thereby eliminating the fundamental, root cause(s) of enterprise disintegration. 

If we base the Enterprise Applications Architecture on the Enterprise Logical Business Process Architecture—and we design and build applications that conform to the scope and context of that architecture—we will not have the redundancies and inconsistencies that currently plague our business applications, and the application code will be much better suited for reuse.  We will also be in a position to design and build databases that are non-redundant, shareable, and reusable across the whole enterprise.

To integrate, or not to integrate?  We have the answer at hand.

Why Build, Why Buy?

 The Proverbial IT Conundrum

 Introduction

Over the past 50 years or so, IT (a.k.a. Data Processing and Information Systems) has been spending hundreds of billions of dollars each year.  After all of this time and money, IT still seems to be stuck in a rut that continues to perpetuate the problems that have plagued the industry from its beginning.  These problems have been so persistent that the IT industry continues to perpetuate them even today.

These problems are frequently described by the following criticisms.

  1.  Takes too long
  2. Costs too much
  3. Doesn’t meet requirements
  4. Difficult to change
  5. Maintenance costs and time are too high
  6. Required data not available or accessible

These problems have been translated into one-word issues that confront the IT industry.  These issues are:

  1. Alignment
  2. Integration
  3. Quality
  4. Responsiveness
  5. Flexibility
  6. Cost
  7. Quality

Alignment and integration have consistently been the top issues facing IT (as identified by IT executives) for the last 30 years or so.

The basic thrust of IT has been to take advantage of advances in technology and use that technology to deliver value to its customers, usually an enterprise, in the form of information systems (i.e. an application and its databases).  This translates into a very strong dependency between the business process, its supporting application, and the associated database(s).

The IT industry has embraced an extensive list of purported solutions to these persistent issues without much success.  This list includes:

  1.  Structured Analysis
  2.  Structured Design
  3.  Structured Programming
  4.  HIPO (Hierarchical Input Output) Technique
  5. Method 1
  6. Prototyping
  7. Enterprise Architecture
  8. RAD (Rapid Application Development)
  9. Object Oriented Analysis and Design
  10. Business Transformation Enabling Program
  11. Use cases
  12. Agile methods
  13. SOA (Service Oriented Architecture)
  14. Data Warehouses
  15. Canonical Data Models
  16. Data Marts
  17. Federated Data Models
  18. Component Based Development
  19. Enterprise Architecture
  20. MDM (Master Data Management)
  21. CRM (Customer Relationship Management)
  22. Process Re-engineering

The above list does not contain the names of hardware and software technologies that have been marketed as providing solutions to persistent IT problems.  That list is also quite lengthy.

Hindsight, of course, has a clearer vision of what is happening or has happened and why.  Looking back over the last 30-40 years, many of the so-called advancements have been solutions to the symptoms of the problem(s), not solutions to the root cause(s) of the problem(s).  There is always a great temptation to “leap to a technical solution”; to go with the apparent quick-and-easy solution even if it just glosses over the causes of the problems.  It is kind of like living an unhealthy lifestyle – we become ill, go the doctor to get immediate relief from the symptoms (a.k.a. please make the pain go away!), but take no action to treat the root causes of our illness.  Treating the symptoms of illness is a lot easier than changing your lifestyle and personal behaviors in order to deal with the real, underlying causes of our problems.

There is a fundamental cause of these problems and it lies in the basic analytical construct that was adopted at the beginning of the development of information systems.  This underlying analytical construct was the “Input-Process-Output” structure for specifying the business process and its target application.  This analytical construct is, in my opinion, what got us off on the wrong track.  Worse yet, the construct was applied backwards.  First we defined the output, then you defined the input, and then we defined the process (transformations) necessary to take the input and produce the output.  At first blush, this approach did have the appearance of a fast path to develop and deliver functionality as fast as you can.  To some extent that was achieved.  However, the cost and the consequences of this construct have been significant.  A natural extension of the Input-Process-Output approach is the design and development of application-specific databases which, in turn, created their own set of data-cost and data-quality problems.

Add to the mix are a variety of strategies that have been put forth as a means to deal with many of the persistent IT problems.  This list includes:

  1. Build versus Buy
  2. Outsourcing (various)
  3. Shared Services
  4. Timesharing Services (various)
  5. SaaS (Software as a service)
  6. Web Applications
  7. Cloud Computing

The “Build Versus Buy” strategy is a very instructive situation.  As with many of the other purported solutions, this strategy does not provide a very strong, convincing argument for either Build or Buy.  Why develop    ?  Why buy    ?  Why does this debate even exist?  The reason is that neither approach has produced stellar results as many enterprises have already found from experience.  Much too often both approaches have resulted in excessive costs far above the initial estimates, schedules that have doubled and tripled, and the resulting systems have been disappointing in terms of alignment with business requirements and promoting enterprise integration.

Very often efforts to develop or to buy and implement packages end up as salvage operations trying to get some value for the cost and time expended.  Sometimes these projects are just stopped and the enterprise just takes its losses.  And sometimes the enterprise then tries the “other” approach, too often with the same results.

There is no profession or industry with a more spotty record that continues to thrive and prosper at others’ expense than the IT industry.  This is not to say that there are no information systems efforts that have achieved some level of reasonable success and delivered value; however, the shining examples of glory are few and far between.  To complicate matters, it is pretty hard to predict which information systems efforts will be successful and which ones won’t.  Why does this situation seem to go on and on?

The problem with both the “Build” and/or “Buy” options is that the fundamental approach to the development of information systems (applications and databases) is defective.  Whether you develop the information system in-house (i.e. the “build” option) or buy a package from a vendor (i.e. the “buy” option), the development approach used in both cases is basically the same.  Therefore, the information systems that are acquired using either approach suffer the same fundamental flaws.

These flaws are not inherent in the information technology itself.  The basic problems lie in the concepts, principles, methods, and techniques that IS professionals use to conceive, specify, design, and build the information systems applications and databases.  (Did you notice that I emphasized applications and databases?  Hopefully, you will understand why shortly.)  This is not a technology problem.  This is a professional knowledge and skill problem, and therefore very much a management problem.

The community of IS professionals has not made any real fundamental changes in their concepts, principles, methods, and techniques in the last 30 years or so.  Most of the efforts to improve the information systems development process have been attempts to improve the building of the application component of the information systems.  These efforts have been focused primarily on increasing programmer productivity, slicing and dicing the work differently, and/or using different tools to do pretty much the same things in the same way as we have done them in the past.  Very little has been done to improve conception and specification of the business processes (notice that I didn’t say applications) and even less in the area of specifying and designing enterprise data (notice that I didn’t say databases).

Some might argue that the advent of the personal computer (PC) and the Internet with Web applications are two very dramatic changes in the way the IS community delivers information systems.  Yes, these did change the infrastructure and the computing platforms; however, IS developers continued conceiving, designing, and developing information systems pretty much as they had done for decades.  In fact, some attempts at changing how they built the information systems actually made the underlying problems worse.  For instance, the data problems have gotten worse, much worse.  And the cost and effort to compensate for these problems is growing with usually, at best, marginal results. 

The IS industry has also been seeking the promised land of “reusability.”  For over 20 years the IS industry has been trying to figure out how to leverage existing functionality by reusing that which they think they know to already exist.  This crusade has been known by such names as component-based development (CBD), model-based development (MBD), and Model Driven Architecture (MDA).  Even the more current popular approaches such as Object Oriented Analysis and Design,  Service Oriented Architecture (SOA) and Agile methods are efforts to discover the “reusability” promised land, however, they are not solving the basic root causes and problems.  In some cases, these efforts are creating more and more redundant code that will need to be maintained for decades to come; and are actually creating more and more redundant data which is further compromising data quality.

There are several reasons why these problems persist.  A few of the reasons for this persistence are the:

  1. IT “Job Shop” business model.
  2. Obsession with Budgets and Schedules.
  3. Search for the “Silver Bullet.”
  4. Process/Application Bigotry, a.k.a. “Data Discrimination.”
  5. IT and User standoff.
  6. Cost Versus Value Disconnect
  7. Focus on Who does What, and the Lack of Focus on How and Why.

The reason I list these in this order is that it generally reflects the premises and priorities that IT management uses in the way they conduct their business.  The first problem is the sequence of this list; if a lot more attention and effort went into the “HOW” and “WHY” in Number 7, most of the other problems would be greatly diminished.

It is highly unlikely that we will ever significantly improve the capability, quality, and maintainability; the alignment, integration, flexibility, and responsiveness; or reduce the time and cost to deploy information systems, as long the “Job Shop” mentality persists. 

The Job Shop approach is the Orville and Wilbur Wright business model.  Orville and Wilbur are recognized as the inventors of the airplane; however, they did very little to advance the process of building airplanes.  As of 2011, we know who has excelled at conceiving, designing, and building airplanes—Boeing, Airbus, Bombardier, and Embraer.  A lot of airplane manufacturers have fallen by the wayside in the meantime.

A key feature of the Job Shop business model is that you wait for a customer to appear and request a product before you do any design work.  The Job Shop approach is very common when an industry is in its infancy.  However, it is not a business model that is effective as an industry matures.  Why?  Let me give you a few reasons.

In the Job Shop approach little, if any, attention is given to designing for reusability, because it is a given that every product is a “custom product”.  Therefore, the attitude is: “How can we design a part for reusability since we don’t know what the next product will need?  Besides, if we attempt to do that, it will take a lot more time and cost a lot more.  And besides that, who knows if there is going to be another product”.  That, of course, will cause budget and schedule paranoia to surface which will kill the whole idea of designing for reusability.  The result is we are trapped in the Job Shop mentality.  Further, there is very little, if any, parts interchangeability or reusability; there are very high lead times, very high product costs, and very high maintenance costs.  Don’t those sound like many of the pervasive and recurring information systems problems?

There are two very fundamental reasons for the obsession with project budgets and schedules.  One is that information systems projects have a really bad record of meeting planned budgets and schedules.  As the costs have escalated, the perception of unacceptable risk has driven many people to move more and more to a “buy” bias, because they perceive that it will provide the lower-cost, least-risk option.  Evidence is pretty strong that this perception is incorrect.

The other fundamental reason is more subtle.  IT management, as well as the enterprise management, tends not to be actively engaged in “how” the information systems development process really works in detail.  So, lacking that knowledge and understanding, they tend to deal with the more general subject of the project budget and schedule since that is what typically interests management the most.  The buyer is always interested in “how much does it cost and when can I get it”.  The buyer would rather not deal with the details of “how” it is built.

As an industry, the information systems development process is not very well defined, and there is not much rigor in the development of metrics that can serve as a basis for developing reasonable information systems development project budgets and schedules.  Budgeting and scheduling in the information systems development arena pretty much involves proposing a budget and schedule that will be acceptable, not necessarily realistic. 

Another aspect of the information systems development budgeting and scheduling process is that it is driven by variations of the following directives or questions:  “Do it for ‘X’ amount of money!” or “Do it by ‘X’ date!” or “What can be done for ‘X’ amount of money?” or “What can you get done by ‘X’ date?”  The situation really gets nasty when these directives and questions get combined!

The first fundamental problem here is that a business process is what it is.  Its size, shape, complexity, etc, does not vary depending on how much money you want to spend on automating it or on how much time it will take.  Don’t get me wrong; I am not advocating that we dispense with using budgets and schedules as planning and control mechanisms.  However, we need to make some fundamental changes in the way they are created and used.

The “Project Approach” has been used since ancient times—in the IS business that means way back in the 1950s!  To complicate matters, the Job Shop and the Project Approach are highly incestuous.  If you have a “Job Shop” business model, you are almost guaranteed to be using the “Project Approach”.

The vast majority of the effort to improve the information systems development process is focused heavily on how to design and build business process functionality, meaning the “application”.  Building business process functionality is important, but it is not the only end-game, and there are one or two other critical aspects that are just as, if not more, important.  How about enterprise alignment, integration, and flexibility?  And then there are the ever-present—and becoming much more relevant—issues of data quality, availability, and accessibility.

Our attraction to and fascination with technology tends to blind us from looking at the basic pervasive problems of the Information Systems industry, which has become so preoccupied with the technology, that it now almost universally likes to be known as the “Information Technology (IT)” industry or profession.

We seem to be emphasizing knowledge and skill in the use of technology over the knowledge and skill in analyzing and designing the enterprise data and business processes.  Without improved enterprise data and business process analysis and design, the employment of the technology will be expensive, over-sold, under-utilized, and too often disappointing.  I have often used a variety of analogies to help people better understand this critical issue.  So, here are a few of those analogies:

  • Just because I have a tractor and know how to operate it does not make me a competent farmer.  There were very good farmers long before there were tractors.
  • Just because I have Microsoft Word and know how to use it does not make me a competent poet or a best-selling author.  There were very good poets and authors long before Microsoft Word existed.
  • Just because I have a fine set of high-tech carpenter tools and know how to use them does not make me a competent carpenter.  There have been some mighty fine carpenters who used very basic hand carpentry tools.
  • Just because I have Microsoft Project and know how to use it does not make me a competent Project Manager.  There were some very good Project Managers long before the existence of Microsoft Project.
  • Just because I have a nifty, hand-held calculator that can calculate a square root does not mean I know how to calculate a square root.  In this case, the calculator knows how to calculate a square root; I know how to operate the calculator.  These are two different skills.  Without the calculator, I become a square-root incompetent, hoping the batteries don’t die at an inopportune moment!

Technology may make a person more productive, but it will not necessarily make them competent at their profession.  We have all heard the cliché “garbage in, garbage out.”  Well, producing more garbage faster just produces a lot of garbage.

The issue isn’t so much whether we should “build or buy”.  The real issue is HOW that which we want to build or buy is built. 

The primary issue in the current build-or-buy debate is WHO is going to build it, and WHAT functional capability does their product provide?  Currently WHO does the building pretty much uses the same old HOW to do the building.  The essential, more relevant question should be “WHO does WHAT and HOW?”  And, you might even need to know a little about WHY!  The current debate mostly focuses on the WHO and the WHAT and very little on the HOW and WHY.  The paramount issues are:

  • “HOW” was the business process designed?
  • “HOW” was the enterprise data designed? and then
  • “HOW” were the databases designed and built? and
  • “HOW” was the application designed and built?

The answers to these “HOW” questions will better explain how the pervasive, persistent causes of information systems problems can be recognized, understood, and avoided.

By the way, what technology is used to build the information systems is not the answer to the HOW question.  Technology is the answer to the “Which tools were used to build the information system?” question!

In most industries the customer spends most of their time and effort knowing about the WHO and WHAT, not a lot about the HOW.  Knowing the answer to the HOW questions is the primary responsibility and under the control of the designer, builder, and manufacturer. 

HOW is the primary determinate of the cost and quality of that which is produced.  The customer’s perception of the value of WHO and WHAT are determined by the cost, quality, and availability of the product and/or service.  The critical difference between Toyota and General Motors is not WHO they are or WHAT they produce, but HOW they conceive, specify, design, and build their product and/or service.

The Information Systems profession needs some major improvements in HOW it conceives, specifies, designs, and builds its products and/or services.  The real question is “Who will be the Toyota of the Information Systems business in the future?”

The answer to this question is: “It will be whoever figures out HOW to fundamentally fix the information systems HOW process.”  Do you suppose that, if the Information Systems profession dramatically improved their HOW, their customers’ perception of WHO they are and WHAT they produce might improve?

Relatively speaking, if I were to make a assessment of  what the information systems industry is today, it is probably a Packard … striving to be a Tucker … when it should be striving to be a Lexus!  (I know; the younger people among us probably won’t understand this analogy!)

The issue is really not so much a conflict between whether to build or buy a package.  It is an issue of HOW the development is done.  Whether you build the information system yourself or buy it, the end result—the information system—was built by someone, somehow.

Only the “build” option has the potential to serve the strategic, tactical, and operational interests of an enterprise.  Currently available packages cannot serve the strategic, long-term interests of an enterprise; they can only serve short-term operational and tactical interests.  It is possible to build packaged applications that can be dramatically better, but it is very difficult for them to excel at alignment, integration, flexibility, and responsiveness, or to deal effectively with the data problem.

Packages, almost by definition, cannot solve their inherent shortcomings of deficient alignment and integration capabilities; and they are a major cause of data redundancy and the attendant data quality problems.  The reason is that a package is always developed out of context of any enterprise.  At least no package vendor has yet produced a comprehensive suite of information systems that would integrate the enterprise vertically and horizontally.  The result is that a major implementation issue with packages is the effort necessary to “interface” the package with the other information systems in the enterprise.  If the enterprise ends up with packages from different vendors, the complexity and difficulty of interfacing these heterogeneous information systems gets magnified.  The resulting redundancy of business process functionality and data will cost the enterprise dearly and unnecessarily, and make the ability to change and adapt roughly the equivalent of trying to break out of an iceberg.

Not only is it important for an enterprise to be flexible and responsive, but, more importantly, the enterprise must be flexible and responsive in the way it wants be, when it needs to be.  Packages, as currently delivered do not, cannot, and will not serve this critical business requirement.

Currently, package vendors thrive best when they have one version of their package and lots of enterprises that buy their package.  The ideal situation for any package vendor is for every one of their customers to use the same version of their package.  Unfortunately, this situation is not going to result in any one of the enterprises in question having a so-called “competitive advantage” and these enterprises are not going to be able to change and adapt on their own terms.  At best, the package user will have the capability to be average in relation to their competitors. Of course, if the enterprise in question is below average, using the package approach would be seen as an improvement.

However, I do think that it is possible to build and deliver “package” solutions that could be considered “starter sets” and then turn the package over to the customer to enhance and maintain as they see fit.  Of course the architecture of the application(s) and the database(s) would have to be dramatically different than what currently exists.

There are several current practices that negatively influence the approach to how enterprises acquire (i.e. build or buy) their information systems.  The most essential issue that must be addressed in order to affect a meaningful improvement in the process of acquiring information systems is the information systems development process.

Information Systems professionals have been building information systems which typically consist of the “application” (i.e. the processing part of the information system) and a “database” (i.e. the data storage and access facility) using a selection of information systems technology.  The basic premise has been that, if you have an application, there must be a database to support that application—which almost universally has led to application-specific databases—which I refer to as “application databases”.  The net result has been application databases which contain redundant, inconsistent, and fragmented data with significant variations in the quality of the structure and content of the data among the application databases.  This is perhaps the most significant root cause of the proverbial “enterprise data problem!”

This “application-database approach” also produces a lot of redundant application logic which may or may not be consistent—usually not.  This situation will most likely persist if we don’t take decisive action to change the current pervasive, conventional systems development methods and project approach to acquiring and implementing information systems.

If we, the community of IS professionals, want to produce information systems that are at least aligned and integrated, then we must stop using over and over and over again the same flawed development process that has produced the same unsatisfactory results for more than 50 years.

I believe that proven advances in Enterprise Data Management and the adoption of an Enterprise Database approach present us with a great opportunity to resolve the current IT conundrum.

By:

Douglas T. Erickson

ENTARCO USA Inc.

 

Enterprise Architecture: What Is It?

 1.0     Introduction

Much discussion and debate goes on about “What is Enterprise Architecture”?  This discussion is very diverse in scope and content.  As evidenced by this discussion, Enterprise Architecture has not achieved a universal, standard, generally recognized definition.

What follows is not an exhaustive answer to the question of Enterprise Architecture: What Is It?  But hopefully, it will add some clarity to the subject.

 To define Enterprise Architecture, we need to define the concepts and disciplines that combine to create what we have come to know as Enterprise Architecture.  We need to understand what architecture is and what an enterprise is in order to understand what Enterprise Architecture is.  It is also useful to understand the distinctions between Enterprise Architecture and other enterprise related concepts, practices, and disciplines.

Note: After 30 years of intimate activity regarding Enterprise Architecture as a thought leader and practitioner, I have arrived at a point where I use the Zachman Framework for Enterprise Architecture (ZFEA)—also known as the Zachman Enterprise Framework and the Zachman Framework for Information Systems—as the framework for defining Enterprise Architecture and also as the framework for defining and organizing ENTARCO’s Methodology for Enterprise Management Services (MEMS).  I have found the ZFEA to be the most neutral, comprehensive, and theoretically and empirically sound basis for architecting, engineering, and manufacturing an enterprise.  Therefore, the ZFEA is used as the anchor point for our definition of Enterprise Architecture.

It turns out that it is also very useful to separate the concept of Enterprise Architecture from the participants and the roles that they play in designing and constructing the enterprise; just as in the building analogy, architecture (meaning building architecture) involves many participants playing a variety of roles.

2.0     Definitions of a few basic terms.

Architecture, according to many dictionaries,is the art or science of designing and constructing buildings; a formation or construction as if the result of conscious act; a unifying or coherent form of method or style of building”.

Enterprise

The word “enterprise” is problematic as there are a variety of definitions or meanings, some formal, others informal, and it is often used interchangeably with words like company, corporation, partnership, agency, department, division, etc.  Also, over time, definitions for the word “enterprise” evolve for purposes of clarifying what is meant by different people for different purposes.

One of the more comprehensive, formal definitions of an enterprise is the definition used by the U.S. Department of Commerce which is as follows:

“Enterprise is any organization, in the public or private sector of the economy, which is formed for the purpose of conducting any organized business activity which produces any products or provides any services, either for profit or not for profit.”

Note: This definition may vary over time, but seems to be fairly permanent and consistent.

Here is the definition that I have evolved over 30 years or so as I grappled with the concept of “enterprise”.  I have found this definition to be very useful:

“An enterprise is a collection of resources assembled for the purpose of providing its product and/or service to its market, in and of itself.”

Even this definition needs some explanation.

First of all, to be a completely functioning enterprise, it must have authority, governance, and responsibility for the performance of all activities that are necessary to provide its products or services.  (For brevity purposes, I use the phrase “products or services” but by definition it includes all products or services in a distinctive line of products or line of services.)

The ENTARCO definition eliminates the idea that you can create an Enterprise Architecture for something referred to as a department, division, operation, or any other segmentation of a corporation or company.  Organizational subdivisions, by definition, cannot by themselves provide a product or service to its market. 

Outsourcing some activities to another enterprise does not mean that those activities are outside the authority of the outsourcing enterprise.  Outsourcing the Customer Help Desk to another enterprise does not absolve the enterprise of its responsibility for the performance of the Customer Help Desk activities.

An enterprise may exist to pursue an economic, political, social, or religious objective, or combination thereof.  For all practical purposes, every enterprise, in some manner or another, has a “super objective” to provide some “product or service” to a “market”.  The market may be well defined and mature, or in its infancy.  A market may not even formally be known as a market, but exist only as a latent need or demand.

The most essential aspect of an enterprise is the specification of the “super-objective” of the enterprise.  An example of this would be that the creators of the enterprise specify that they are going to:

  1. Engage in the design, manufacture, and distribution of a product or product line,
  2. Provide a service to persons or other enterprises,
  3. Administer a political jurisdiction, such as federal, state, province, county, parish, township, or other political jurisdiction, or
  4. Promote a political or religious philosophy or ideology to some specified group of people, or in some geographical area, or in some political jurisdiction.

These basic specifications of purpose pretty much dictate the underlying scope, content, and structure of the enterprise.  The product and/or service provided by an enterprise will define what the enterprise is and what it needs to know, how it does what it does, and pretty much where and when it does it, who does it, and why.

 By basing the context of an enterprise on the definition of its products or services, we have a very stable basis on which to define the scope of the enterprise.  The context of the enterprise will be stable because, as long as the enterprise’s purpose is to provide that product or service to its market, the architectural foundation for the enterprise will not change.

For Enterprise Architecture purposes, it is therefore very useful to set the boundaries for an enterprise around a specification of a product or service.  The result of this concept is that what is known as a corporation many in fact be many enterprises.

As an example, many people are familiar with PEP Boys auto stores.  When looking at a PEP Boys store, you are actually seeing the manifestation of two enterprises: 1) the auto parts store, and 2) the auto repair store.  PEP Boys Auto, the corporation, consists of two enterprises – one an automotive parts enterprise, the other an automotive repair enterprise.  These can easily be seen as highly related enterprises, but distinct in that one is primarily a product-based enterprise and the other is a highly services-based enterprise. 

These two enterprises are fundamentally different because their products and services are different.  However, there may be commonality that can be leveraged between the two enterprises.  Once the individual Enterprise Architectures exist, it will be fairly easy to identify and take advantage of these commonalities.  These two architectures, as a set, could be considered a “Federated Enterprise Architecture”.  In this case, it is very important to maintain the distinction between the two enterprises; this includes designing an Enterprise Data Architecture that explicitly preserves the data relative to the two, distinct enterprises.  

Sometimes a corporation, in its entirety, is the enterprise.  However, sometimes it may, in fact, be two or more enterprises.  General Electric Corporation is more than one enterprise for Enterprise Architecture purposes.  General Electric, in its entirety, likely involves more than one corporation because it may wholly-own enterprises that are also corporations. 

On the other hand, if the enterprise that designs, builds, and sells cars decides to stop doing that and decides to become a home builder, it will become a different enterprise for Enterprise Architecture purposes, because the basic product and/or service it provides is now different and the fundamental nature of the enterprise will have changed.

Corporation, company, partnership, trust, and sole proprietor are not names that specify an enterprise; they are forms of organization for an enterprise.  An enterprise may start out as a sole proprietorship, grow and become a partnership, and grow again to become a corporation.  This is a case of the same enterprise changing its form of organization over time. 

This transformation may cause some additional processes and data to be relevant to the enterprise, but the basic scope and content of the enterprise, based on its products and services, will remain very stable.  If an enterprise that designs, builds, and sells cars starts up a consumer finance enterprise, there is now a new, second enterprise, such as the case with General Motors Corporation when they started General Motors Credit Corporation.  The fact that they were both called “corporations” is not the determining factor in whether they are one or two enterprises.

It is very important to understand that an enterprise exists only in concept.  You cannot see “the enterprise”.  Yes, we know it exists in some form, but it does not exist in the sense that a car, or a house, or a building exists.  We can create an enterprise, we can define it, we can design it, and we can create physical objects that represent elements of an enterprise.  When you stand and look at the building that houses the headquarters or home office of an enterprise, you are not seeing “the enterprise”, you are seeing a building that may or may not be owed by the enterprise.  The enterprise headquarters could be moved to a different building and the enterprise would still exist as it did before, but now it occupies or resides at a different address.  The building exists physically, but the enterprise exists only as a concept.

This idea also applies to the concept of “organization”.  The organization does not exist in a physical sense.  You can draw me a picture of an organization chart that shows me the positions and their relationships, and you can show me where the manager and the staff sit; however, you cannot see a physical thing called an “organization”.  This is the nature of concepts – they exist, they can be described, but they only exist because someone conceived them, defined them, and described them in a recognizable manner.

An Architect is a person who designs buildings and, in many cases, also supervises the construction of those buildings.

In the building analogy, the Architect designs the building to an excruciating level of detail.  Then a builder (i.e. general contractor) takes the architect’s design and builds whatever the architect(s) designed.  If the thing to be built is large and/or complex, there may be several builders (i.e. subcontractors) involved in building the thing the architects designed.

Within the realm of Enterprise Architecture, the Enterprise Architect designs the enterprise and must supervise the construction of the enterprise from the architect’s designs in order to ensure the integrity of the architectural drawings and/or models.  The enterprise engineers (i.e. applications or systems analysts) design the applications using the Enterprise Architect’s drawings as their specifications.  The General Contractor (usually the IT organization) using various sub-contractors, such as programmers, database administrators, user-manual writers, and application testers actually construct, test, and implement the components of the enterprise.

Enterprise Architecture

Therefore, Enterprise Architecture is theis the art or science of designing and constructing enterprises; a formation or construction as if the result of conscious act; a unifying or coherent form of method or style of building”.

Accordingly, the word “architecture”, by definition, includes both the design of an enterprise and the construction of an enterprise.

A key clarification here, which I will address more fully later, is that architecture is about designing and building an enterprise and perhaps maintaining it throughout its life; it is not about managing the use of the enterprise once it exists.  Managing the use of the enterprise once it exists is called Management or Enterprise Management.

Enterprise Architecture produces the conceptual or metaphysical enterprise and its transformation into a functioning enterprise.  Enterprise Architecture is about:

  1.  The processes an enterprise must perform to provide its products or services to its market(s),
  2.  The data that describes the performance of the activities that are performed by the enterprise in the course of providing its products or services to its market(s),
  3.  The geographical placement (i.e. the kinds of logical business units) where the enterprise conducts its affairs,
  4.  The organization of the enterprise’s resources for use in conducting its affairs,
  5.  The tracking of the conduct of the enterprise’s affairs over time, and
  6. The basic motivations (i.e. the products and services) that cause the enterprise to exist.

 The Enterprise Data Architecture is one of the six (6) primary sub-enterprise architectures.  This architecture consists of the identification and definition of the “things” the enterprise needs to know about, and the business rules that govern the relationships amongst the “things”.

The Data Architecture includes the Database Architecture.  The Database Architecture consists of the System, Technology, and Detailed Representations of the Data Architecture.  The Database Architecture must map to the ZFEA System Data Model.

The Enterprise Process Architecture consists of the identification, decomposition (i.e. structure), and specification of the processes that the enterprise must perform in order to provide its product or service to its market.

The Process Architecture includes the Applications Architecture.  The Applications Architecture consists of the System, Technology, and Detailed Representations of the Process Architecture.  The Applications Architecture must map to the ZFEA System Process Model.  The applications are the physical manifestation of the business processes.  If an enterprise exists and is operating, all of its processes and the automated and manual applications are in place.  However, the critical issues are:

  1. How well are the business processes designed, and
  2. How well do the applications implement the business processes?

 The Enterprise Geographic Architecture consists of the identification and definition of the locations of activities and/or locations of properties controlled by the enterprise.

The Geographic Architecture is the Network Architecture, as referred to in the Zachman Framework for Enterprise Architecture.  The lower two (2) Rows in the Network column in the Zachman Framework for Enterprise Architecture are often referred to as the “Information Technology Architecture”.  People tend to think that these two (2) Rows constitute the “architecture of interest” to the enterprise and therefore erroneously refer to it as the Enterprise Architecture.

The Enterprise Organization Architecture consists of the aggregate of units of resources and their structure created for the purpose of carrying out the activities of the enterprise.

The Organization Architecture is the equivalent of the People column in the Zachman Framework for Enterprise Architecture.  People in positions of authorized Job Classifications assigned to organizational units actually perform the activities of the enterprise.

The Enterprise Temporal Architecture consists of units of time and the relationships among them.  The structure of time is used to reflect the events and their occurrences in relationship to each other and their duration.

The Temporal Architecture corresponds to the Time column in the Zachman Framework for Enterprise Architecture.

The Enterprise Motivation Architecture consists of the governance established for the enterprise.  The governance includes the declaration of the purpose of the enterprise and the policies that are enacted to provide the guidance necessary to achieve the expected behavior of the enterprise.

Each of the above sub-architectures is refined based on the representations of the five (5) perspectives as specified by the Zachman Framework for Enterprise Architecture.  The five (5) perspectives are:

  1. Scope (Model) – Planner’s perspective,
  2. Business Model – Owner’s perspective,
  3. System Model – Designer’s perspective,
  4. Technology Model – Builder’s perspective,
  5. Component Model – Subcontractor’s perspective, and
  6. Operations Model – the functioning enterprise.

 A very important aspect of these perspectives is that each perspective is not always the result of adding more detail at each level.  Level 2 has more detail than Level 1, and Level 3 has an excruciating level of detail.  Level 3, the System Model is a complete, detailed specification of the design of the enterprise. 

No enterprise design elements can be introduced in Levels 4 and 5.  Very limited content can be introduced at Levels 4 and 5 and only for very limited reasons.  Additionally, Levels 4 and 5 must conform to the design as specified in the Level 3 perspective.  If the people working at Level 4 and 5 have an issue with the content of the Level 3 perspective, they must get agreement from the Enterprise Architect to affect a change to the Level 3 content. 

Therefore, Enterprise Architecture is the discipline of designing and constructing an enterprise.  It is the collective actions to create the Data, Process, Geographic, Organizational, Temporal, and Motivation Architectures for an enterprise.  The result of a completed Enterprise Architecture is an enterprise.

The most essential sub-architectures are the Data and Process Architectures.  However, these aspects of an enterprise have been pretty much ignored in terms of management attention and effort. 

Many other disciplines necessary to ensure the effective functioning of an enterprise have experienced dramatic changes over the last 50 years or so.  Data or Information Management and Business Process Management have existed pretty much on the margins of enterprise management in most enterprises.  Enterprise management teams have too often abdicated their responsibility for these activities to the IT organization; and the IT organization has too often assumed that whatever enterprise staff told them was sufficient to deliver information systems.  

It is time for these disciplines (i.e. Data/Information Management and Business Process Management) to be recognized as critical to the future well-being of any enterprise.  If these two aspects of Enterprise Architecture are not at the forefront of an Enterprise Architecture program, it should not be called an Enterprise Architecture program.  If these two sub-architectures are not most prominent during the definition of the program, then the program is probably something much less than it needs to be and is probably one of more of the following impostors, posing as Enterprise Architecture:

  1. Applications Architecture,
  2. Database Architecture,
  3. Network Architecture, and/or
  4. Infrastructure Architecture.

The manifestation of the enterprise is the data, databases, processes, procedures and applications, organization structure, kinds of geographical locations where the enterprise resides, its history, and its motivations as represented by:

  • Row 6 in the ZFEA,
  • As they are constructed in Rows 4 and 5 by the Builders, such Systems Analysts, Programmers, and Database Designers (in the case of computerized implementation of the business processes and data) or by Procedural Analysts and Writers (in the case of manual systems), and
  • The architected models produces by Enterprise Architects in ZFEA Columns 1-6, Rows 1-3.

 All of the above constitutes the foundation of any enterprise.  Without this foundation there is no functioning enterprise.  Whether or not all of the enterprise artifacts exist explicitly or implicitly for the enterprise, all the fundamental elements of the Zachman Framework for Enterprise Architecture will be present in some form, to some extent. 

When an enterprise exists and is operating, all of the ZFEA Row 6 manifestations of the enterprise exist.  However, all of the ZFEA Rows 1-5 artifacts may not exist explicitly.  If they do not exist, then the explicit Enterprise Architecture for the enterprise does not exist.  The implied Enterprise Architecture is embedded in the ZFEA Row 6 manifestations of the enterprise.  If you want to create the “as-is” Enterprise Architecture of a given enterprise, you are going to have to reverse-engineer and reverse-architect the ZFEA Row 6 manifestations of said enterprise.

The Enterprise Architecture, that portion which addresses the architecture and design of the enterprise is represented by ZFEA Columns 1-6, Rows 1-3. 

The ZFEA Columns 1-6, Rows 4-5, address the design and construction of the enterprise architecture.

 

2.0     Distinguishing Between Enterprise Architecture and Management Techniques

 Enterprise Management is the actions taken by people with the responsibility and authority to manage (i.e. plan and control), decide, and direct the affairs of the enterprise.  Enterprise Management is primarily engaged in the operation of the enterprise and is rather casually involved in the architecting, designing, and construction of the enterprise.

One of the major areas of confusion about what Enterprise Architecture is and what management techniques are arises from being confronted with the myriad of management concepts, principles, and techniques that exist as the panacea for correcting the ills of an enterprise or for positioning the enterprise to thrive in today’s world.  The list of these is long and will get longer.  A few of these that have achieved some degree of acceptance at one time or another are:

  1. Management by Objectives (MBO),
  2. Management by Walking Around (MBWA),
  3. Zero-Based Budgeting,
  4. Quality Circles,
  5. Participative Management,
  6. Total Quality Management (TQM),
  7. Six Sigma Quality,
  8. Decentralization,
  9. Responsibility Accounting,
  10. Just-In-Time (JIT) Inventory Management,
  11. Customer Relationship Management (CRM),
  12. Balanced Scorecard (BSC), and
  13. Enterprise Resource Planning (ERP).

You will notice no mention of Enterprise Architecture in this group.  The above list, except possibly Business Process Re-engineering, are “management techniques”, they are not Enterprise Architecture techniques.  Every one of the techniques cited above could just as easily be applied to the function of Data Processing, Information Systems, or Information Technology management, as to any other functional area of the business.

3.0     Enterprise Architecture: What it is not.

 Enterprise Architecture is not a program or project that management chooses to implement.  Enterprise Architecture is fundamental and always present.  The issue is not whether to do Enterprise Architecture or not, but how to do it.  Enterprise Architecture is not a new program… it is doing what is already being done, better… much better. 

 The problem is that, in the past, what would be considered Enterprise Architecture efforts were very sporadic, fragmented, random, un-disciplined, trial-and-error activities.  From an Enterprise Architecture point of view, looking backward or looking at where we have come from, we can view these activities in the context of what we now know as Enterprise Architecture.  Over the last 40 years there have been many ideas, concepts, techniques, and initiatives that, in retrospect, can be seen as efforts that we now understand as honest attempts at Enterprise Architecture.  A very short list of these follows:

  1.  Hierarchical Input Process Output Technique,
  2. Structured Programming,
  3. Structured Design,
  4. Structured Analysis,
  5. Busienss Systems Planning
  6. Non-procedural Programming Languages,
  7. Rapid Application Development,
  8. Relational Modeling,
  9. Object-oriented Analysis and Design,
  10. Information Engineering,
  11. Enterprise Architecture
  12. Client-Server,
  13. Web Services,
  14. Middleware,
  15. Computer-aided Software Engineering,
  16. Customer Relationship Management,
  17. Business Transformation Enabling Program (BTEP)
  18. Process Engineering/Re-engineering,
  19. Agile methods,
  20. Use Cases, and
  21. Conical data modeling
  22. Government Strategic Reference Models (GSRM)
  23. Etc, etc.

 Each of the approaches cited above represents honest efforts to address recognized issues or problems, or efforts to improve the activities undertaken to provide “systems” that are a manifestation of the implementation of an enterprise. However, none of these ideas, concepts, techniques, and initiatives was ever part of an integrated, coordinated plan for engineering and manufacturing the enterprise.

Furthermore, Enterprise Architecture, as it is practiced today by most people, is still a pretty random, un-disciplined, fragmented, and trial-and-error activity. 

What is most often being portrayed as Enterprise Architecture today are most often traditional and conventional practices and evolving technology being repackaged and sold as Enterprise Architecture.

Customers have become convinced they need Enterprise Architecture, but don’t know much about it. This is yet another situation which is ripe for the selling of another “silver bullet” solution.  It is not the acquisition and implementation of new technology; it is developing the capability to define and analyze the requirements of the enterprise (i.e. architect the enterprise), design the enterprise (i.e. engineer the enterprise using the architectural drawings), and construct the enterprise (i.e. transform the architectural drawings and engineering designs into physical manifestations of the enterprise) that ultimately implement the enterprise in a much improved manner.

4.0     Choosing How to Do Enterprise Architecture

 Enterprise Architecture should be viewed as a comprehensive definition of the framework for re-inventing the function known over time as data processing, information systems, or information technology, or whatever label you choose.  It is a framework for at last having a comprehensive, integrated, rational, structured means to manage the architecting, engineering, and manufacturing of an enterprise.

 A very significant step forward would be to re-name the function commonly known as Information Technology.  The term “Information Technology” sometimes causes very negative reactions—it promotes an emphasis on the subjects “information” and “technology”.  This is a great boon to the “information technology vendors”!  However, it says nothing about a function whose primary purpose is to architect, engineer, and manufacture an enterprise.  I suggest that collectively we seriously consider a new term—something like “Enterprise Management Services”, which would lead us to think in terms of an “enterprise”, “enterprise management”,  and providing services that enhance our ability to manage and operate an enterprise.

 I believe that the current IT function is quite well positioned to assume this role.  However, the IT leadership, along with enterprise leadership, needs to recognize that their primary responsibility is to facilitate the architecting, engineering, and manufacture of the enterprise they support in a more disciplined manner in order to achieve dramatically better results than are currently being achieved.  Their role is not only to be the “technology acquirer and implementer” for the enterprise.  It should also be to lead the architecting, engineering, and manufacture of the enterprise.  (Let me strongly emphasis that this does not imply assuming the role of managing the enterprise.)   

This is a dramatic shift in the traditional role of a CIO. Adopting an approach to Enterprise Architecture based on the Zachman Framework for Enterprise Architecture is the first step in fulfilling that responsibility.

 The next step—choosing how to do Enterprise Architecture—is critical and difficult.  The principles, concepts, goals and objectives, the methods, tools and skills, and the organization of these is what will determine whether you achieve the potential that is now being recognized as the value of Enterprise Architecture.

One of the basic ways to separate the Enterprise Architecture “wheat from the chaff”, so to speak, is to closely examine the methodology espoused by the Enterprise Architecture practitioner, or Enterprise Architect, if you will.  Most so-called Enterprise Architects don’t have a formal, defined, rigorous, practiced methodology.  Also, because the concept of Enterprise Architecture is not well defined, just about anything can be presented as being “Enterprise Architecture”.  As a matter of fact, even some of the leading advocates for Enterprise Architecture declare that there are “many ways to implement Enterprise Architecture”.  There may be more than one, but I can guarantee you that there are not many!

Suffice it to say that listening to a 1, 2, or 3-hour presentation by a consulting firm, or attending a 2, 3, 4, or 5-day seminar is not adequate to understand and select a methodology for Enterprise Architecture.  Anyone undertaking an Enterprise Architecture effort should invest a good amount of time and effort in examining the methodologies that are being promoted as Enterprise Architecture methodologies.

In accordance with the Zachman Framework for Enterprise Architecture, the Row 1 – Scope Models must establish the complete context for the development of the Row 2-through-5 models of the Enterprise Architecture.

The EA practitioner must be able to demonstrate that they have a methodology for implementing the entire Zachman Framework for Enterprise Architecture.  By demonstration I mean that:

  1. The EA practitioner can provide documentation that articulates the methodology that is proposed,
  2. The methodology that is proposed has been applied successfully in at least two (2) enterprises where the Row 1 Models have been developed followed by the development of the Row 2-through-5 Models,
  3. At least three (3) large enterprise applications and enterprise databases have been implemented in production using the proposed methodology, and
  4. Experienced personnel who have led such efforts are available and will lead the effort.

The proposed methodology and its demonstrated use must result in:

  1. Alignment of implementations with the requirements from Row 3.  This means that the Row 3 models (especially the Data and Process column models), which specify the business requirements (i.e. specification of the business data and processes), must be fully incorporated in the implemented Databases (i.e. Column 1, “Data”) and applications (i.e. Column 2, “Process”).  In other words, the Databases and Applications that are implemented must incorporate/reflect the business requirements as specified in the Row 3 Models.
  2. Models which eliminate, or minimize, redundant data and processes throughout the enterprise,
  3. Applications and databases which eliminate, or minimize non-redundant data and processes,
  4. Engineering of the business processes.  This means that the methodology must incorporate methods and techniques that provide for the identification, design/specification, and documentation of “how” the business processes are to be performed.  The engineering or specification of the business processes are requirements that must be incorporated in any application acquired to support the business process(es),
  5. Integration of data and processes across the enterprise, and
  6. Enhance the ability to responsively manage and implement changes to the enterprise.

 The above criteria will help ensure that the EA practitioner has the knowledge, experience, and credentials required to function as a legitimate Enterprise Architect.

There are emerging, though sparse, metrics that demonstrate phenomenal value and benefits for a “good” Enterprise Architecture program.  What is critical to recognize is that the value and benefits are not inherently achieved just because an Enterprise Architecture program is put in place.  The extent and type of changes introduced by the entire Enterprise Architecture program are what will determine the value and benefits achieved.  Implementing an Enterprise Architecture program while not changing any of the existing or current methods of managing data and business processes will yield marginal, disappointing, and suboptimal results.

In conclusion, an Enterprise Architecture methodology must be designed to produce specified, predictable results.  It must specify not only “What to do” but most importantly, “How to do it” and “Why”.  Moreover, it must be based on a set of concepts and principles that theoretically, logically, and empirically establish a basis for producing an Enterprise Architecture that can be used to architect, engineer, manufacture, and implement an enterprise and that the underlying architecture can be demonstrated to be aligned, integrated, flexible, and responsive to change.

Douglas T. Erickson, President ENTARCO USA Inc.

Copyrighted 2011 by ENTARCO USA Inc.