BI test

未分类

• Big Volume
  The amount of data is enormous

• Big Velocity
  Data are generated very fast, often faster than the ability to process them

• Big Variety
  Multimedia data constitute an important data variety

• Big Veracity
  uncertainty

Better decisions： balance between accuracy and speed

model: A model is a simplified representation of reality.

The Benefits of Models

• Ease of manipulation
• Compression of time
• Lower cost of analysis on models
• Cost of making mistakes on experiments
• Inclusion of risk/uncertainty
• Evaluation of many alternatives
• Reinforce learning and training

Problem: Problem is the difference between what people desire (or expect) and what is actually occurring

Potential issues in data/information collection and estimation

• Lack of data
• Cost of data collection
• Inaccurate and/or imprecise data
• Data estimation is often subjective
• Data may be insecure
• Key data may be qualitative
• Data change over time (time-dependence)

Outcome of intelligence phase: A Formal Problem Statement

Descriptive models

Benefit: time saving
bounded rationality
- Simulation - most common descriptive modeling method

The Choice Phase

Includes the search, evaluation, and recommendation of an appropriate solution to the model

DSS: A DSS is typically built to support the solution of a certain problem

• DSS often has its own database and models

Components of DSS

Data Management Subsystem

• Includes the database that contains the data
• Database management system (DBMS)
• Can be connected to a data warehouse

Model Management Subsystem

• Model base management system (MBMS)

Knowledgebase Management Subsystem

• Organizational knowledge base

User Interface Subsystem

Phases of Decision-Making Process

1) Intelligence
2) Design
3) Choice
4) Implementation
5) Monitoring

Data warehouse: A physical repository where relational data are specially organized to provide enterprise-wide, cleansed data in a standardized format

A data warehouse is a subject-oriented, integrated, time-variant, and nonvolatile collection of data in support of management’s decision

-

Characteristics of DW
- Subject oriented
- Integrated
- Time-variant (time series)
- Nonvolatile
- Summarized
- Not normalized
- Metadata
- Web based, relational/multi-dimensional
- Client/server
- Real-time and/or right-time (active)

Definitions

• Operational data stores (ODS)
  A type of database often used as an interim area for a data warehouse
• Oper marts
  An operational data mart.
• Enterprise data warehouse (EDW)
  A data warehouse for the enterprise.
• Metadata
  Data about data. In a data warehouse, metadata describe the contents of a data warehouse and the manner of its acquisition and use

Two Approaches:

• Query-Driven (Lazy)

• Warehouse (Eager)

• Direct benefits

  • Allows end users to perform extensive analysis
  • Allows a consolidated view of corporate data
  • Better and more timely information
  • Enhanced system performance
  • Simplification of data access

• Indirect benefits

  • Enhance business knowledge
  • Present competitive advantage
  • Enhance customer service and satisfaction
  • Facilitate decision making
  • Help in reforming business processes

OLTP: On Line Transaction Processing
– Describes processing at operational sites
OLAP: On Line Analytical Processing
– Describes processing at warehouse

OLTP
- Mostly updates
- Many small transactions
- Mb-Tb of data
- Raw data
- Clerical users
- Up-to-date data
- Consistency, recoverability critical

OLAP
- Mostly reads
- Queries long, complex
- Gb-Tb of data
- Summarized, consolidated data
- Decision-makers, analysts as users

Types of OLAP Servers

• Relational OLAP (ROLAP)
  – Use relational or extended-relational DBMS to store and manage warehouse data and OLAP middle ware to support missing pieces
  – Include optimization of DBMS backend, implementation of aggregation navigation logic, and additional tools and services
  – greater scalability

• Multidimensional OLAP (MOLAP)
  – Array-based multidimensional storage engine (sparse matrix techniques)
  – fast indexing to pre-computed summarized data

• Hybrid OLAP (HOLAP)
  – User flexibility, e.g., low level: relational, high-level: array
• Specialized SQL servers
  – specialized support for SQL queries over star/snowflake schemas

ROLAP

• Provides a less natural view of data
• Slower computation but support larger amounts of data
• More scalable
• SQL provides a standard query language
• Management features such as concurrency tend to be more mature the MOLAP
• Better suited for large databases than MOLAP
• Sometimes an additional layer may be created to give user the a hypercube view derived from the underlying relational database

MOLAP

• Provides a more natural view of data
  
  • More intuitive and simpler to view data directly in the form of a cube rather than multiple 2D tables
• Provide faster computation but support smaller amounts of data
• Less scalable
• As the number of values increases, the number of the cube’s cells increases exponentially
  
  • Can be very large because a separate cell must be available for each combination of data

Data Integration and the Extraction, Transformation, and Load (ETL) Process

Integration :

• data access
• data federation
• change capture

• Extraction -- reading data from a database
• Transformation -- converting the extracted data
  from its previous form into the form in which it
  needs to be so that it can be placed into a data
  warehouse or simply another database
• Load -- putting the data into the data warehouse