Data Engineering in 5 Years Of Working In Data Platform

• 2 min read

After 5 year working data and 7 years working in data industry, 10+ out sourcing project in wide domain such as ingestion, migration, assessment, poc, poc,pov, large enterprise, vendor cooperation,etc. To communicate with all type of project and people are working in different domain, the concept and logical tech-term are being used across which is not important who they are and where they are working on.

The list in following headers are short version where you can use for communication. On other hand, use it for learning and data system check list. I mentioned some in https://de-book.longdatadevlog.com, check it out.

Data Architecture

  • Data Architecture
  • Data Lake
  • Data Mart
  • Lakehouse
  • Data Mesh
  • Data Product
  • Data Warehouse
  • Kappa Architecture
  • Lambda Architecture
  • Medallion Architecture

Data Ingestion

  • Change Data Capture
  • Delta Load
  • Full Load

Data Management

  • Data Catalog
  • Data Governance
  • Database Management System
  • Metrics Layer
  • Relational Database Management System
  • Semantic Layer

Data Modeling

  • Activity Schema
  • Cardinality
  • Data Modeling
  • Data Vault Modeling
  • Denormalization
  • Dimensional Modeling
  • Normalization
  • One Big Table
  • Relational Modeling

Data Processing

  • Batch Data Processing
  • Online Analytical Processing
  • Online Transaction Processing
  • Stream Data Processing
  • Workflow Orchestration

Data Governance, Security and Compliance

  • Data Security
  • Data Masking
  • Data Privacy
  • Data Lineage
  • Data Quality

Data Storage

  • Column-oriented Database
  • Database
  • Document Database
  • Graph Database
  • In-Memory Database
  • Key-Value Database
  • Non-relational Database
  • Relational Database
  • Timeseries Database
  • Wide-column Database
  • Metadata Management

Data Engineering Prep

Check this document out prep

Guides

Check blogs out here