Data Integration Best Practices In Azure Synapse Analytics
Trainer(s): Abhishek Narain
Provider: DPS 2022 (Data Platform Summit)
Duration: 8 Hours
Subtitles: Yes
Regular Price: USD 299
Offer Price: USD 74.75 (75% OFF)
Discount Code: HAPPY75
(GST + Internet Transaction Fee additional)
Subscription Period: Lifetime Access
Abstract:
Azure Synapse contains the same Data Integration engine and experiences as Azure Data Factory, allowing you to create rich at-scale ETL pipelines without leaving Azure Synapse Analytics.
- Ingest data from 100+ data sources
- Code-Free ETL with Data flow activities
- Orchestrate notebooks, Spark jobs, stored procedures, SQL scripts, and more (ELT)
This training will cover the best practices when using Synapse pipelines and is targeted at a data engineer who is new to Azure/ Azure Synapse Analytics.
Modules
- ADF/ Synapse Pipelines Overview
- Best practices
- Metadata encryption (Microsoft, Customer-Managed Keys)
- Source Control
- Secure Authentication – Managed identity (MSI) and AKV integration
- Access control/ RBAC in Synapse
- Managed Virtual Network-enabled workspace
- Monitoring and alerting (Observability)
- Copy: High-perf data integration (Extract-load)
- Data flows: Code-free transformation (Extract-transform-load)
- Data quality, data masking, SCD type2
- Data mapper and lake databases
- Change Data Capture-based incremental extractions from various sources
- Scripts, Spark notebook, and Stored Procedure: Code-based transformation (Transform)
- Continuous Integration and delivery
- Real-world case studies
- Challenge, Q&A