We are partnering with a dynamic and innovative company that is focused on harnessing the power of data to drive business insights and optimize decision-making. They are looking for an experienced Data Engineer to join their team and take a pivotal role in developing and managing data systems, ensuring data integrity, and driving data governance.
Job Summary:
The Data Engineer will be responsible for designing, implementing, and maintaining data management solutions in collaboration with cross-functional teams. This role requires deep expertise in Collibra, C#, SQL, REST APIs, and Azure services to integrate systems, manage large datasets, and ensure compliance with data governance standards.
Job Summary:
As a Data Engineer, you will work with cross-functional teams to design, implement, and maintain data management solutions. You will leverage your expertise in Collibra, C#, SQL, REST APIs, and Azure services to integrate systems, manage large datasets, and ensure compliance with data governance standards. This role requires a solid understanding of both data engineering and software development practices to support data management and enable organizational data initiatives.
Key Responsibilities:
- Data Integrity & Management: Collect, validate, and manage large datasets to ensure accuracy, integrity, and alignment with business needs.
- Collibra Integration: Build, configure, consume, and maintain bi-directional API connectivity between Collibra and other systems, leveraging Collibra workflows, Edge, connectors, and APIs.
- Data Profiling & Lineage: Implement data profiling and lineage analysis to visualize relationships, dependencies, and the flow of data across systems.
- Custom Application Development: Collaborate with teams to build custom data management applications using C#, SQL, and REST APIs, ensuring alignment with organizational data strategies.
- Collibra Workflows & Governance: Develop and maintain data management processes using Collibra Workflows, ensuring compliance with data governance standards and industry best practices.
- Metadata Management: Develop and maintain comprehensive metadata repositories, including data dictionaries, models, and workflows, to document and track data assets and their usage.
- Data Marketplace Management: Oversee and manage the "Data Marketplace" to facilitate easy access to and utilization of data assets by authorized users.
- Collaboration & Support: Work closely with business and technical teams to design data management solutions and provide support and training to end-users on data tools and best practices.
Required Qualifications:
- 5+ years of experience in data engineering, with a strong focus on Collibra development and integrations.
- Expertise in Collibra workflows, APIs, and setting up data marketplace features.
- Strong software engineering skills, particularly with C# (freeform writing and reading).
- Proficiency in SQL Server and a strong understanding of database management.
- Experience with REST APIs, particularly on Azure.
- Familiarity with Azure Web Services, SQL Managed Instances, and Storage Accounts.
- Strong analytical skills and the ability to manage and work with large, complex datasets.
- Experience in developing and maintaining metadata repositories, data dictionaries, and data models.
- Understanding of data governance and compliance standards, particularly within the context of data management tools like Collibra.
Preferred Qualifications:
- Experience with Azure Data Factory, Azure Synapse Analytics, or other Azure data tools.
- Familiarity with Data Governance best practices and frameworks.
- Strong communication skills with the ability to collaborate effectively across teams.
