SKI E&S Data Intelligence Platform implementation

Background

AI and digital transformation have become essential in today's business environment. Beyond simply accumulating large datasets, the key challenge is how to explore and analyze them effectively and turn them into business value.

SKI E&S pursued an enterprise integrated data platform to establish a foundation for AI/DT and expand practical data use. It aimed to improve data-driven operations by collecting internal structured, unstructured and real-time data together with external sources, creating a shared cloud repository and environment for analysis, discovery and use.

Scope

• Data ingestion: internal and external structured, unstructured and real-time sources
• Processing and storage: enterprise data lakehouse, data warehouse and data marts
• Analytics and visualization: personal analytics workspaces and BI environments
• Data catalog: business data catalog and discovery framework
• Governance: principles, policies and processes
• Infrastructure: cloud implementation and operation

Implementation

1. Data ingestion:

We consolidated scattered internal and external structured, unstructured and real-time data into a stable Raw Zone, the foundation of enterprise data assets. We also automated large periodic batches and real-time ingestion processes.

2. Data processing and storage:

– Data lake and marts: We processed and cleansed source data into lakes and marts for business analysis and decision-making, designing storage structures and retention policies around the data lifecycle.

– Airflow development: We implemented batch and periodic ingestion, loading and processing workflows, with pipeline scheduling, reprocessing and monitoring.

– Platform migration and integration: We moved existing renewable energy services to the new platform, verified migrated data consistency and stabilized services through parallel operation.

3. Analytics and visualization:

Dashboards and reports built around key metrics made business insights visible at a glance. Personal analysis and BI visualization environments improved data access and usability.

4. Data catalog:

We built technical metadata and a business data catalog, with automatic generation, Korean-language descriptions and API integration to make data easy to discover and understand.

5. Data governance:

We assessed current management practices and defined governance principles, policies and core processes to systematize enterprise data management.

Results and expected benefits

SKI E&S successfully established an enterprise Data Intelligence Platform spanning collection, processing, cleansing, analysis, use and governance.

Scattered data became integrated assets. Lifecycle-aware storage and automated pipelines improved efficiency and scalability, while the stable migration of the renewable energy platform enabled unified enterprise data management and use.

An intuitive catalog and BI environment allowed users to find and immediately use data, supporting faster, more accurate decisions.

Clear governance principles and processes established a lasting basis for reliability and quality, accelerating enterprise AI/DT and ongoing data-driven business innovation.