Empresa: CI&T
At CI&T, we help large enterprises transform the potential of AI into real business impact with AI Deployment, AI-native execution, and tech-integrated business solutions. With 30 years of experience in technological transformation, we accelerate innovation with expertise in Agentic SDLC, Application modernization, Data & AI, Martech and Business strategy. We are 8,000 CI&Ters across more than 25 countries, collaborating to build solutions with real impact. AI is already part of how we work, evolve, and innovate every day. Responsibilities: - Build ingestion pipelines for Databricks using a medallion architecture (bronze, silver, and gold layers), leveraging Auto Loader and Delta Lake. - Implement "before-and-after" reconciliation pipelines, including record counts, field-by-field comparisons, key/table mapping between versions, and large-scale validation rule application. - Model the gold layer for reporting, covering approval/rejection rates, reasons for outcomes, and views segmented by cluster and object. - Expose results via an API for a web application (using Databricks SQL Warehouse with the Statement Execution API, or a custom-built service) and document the API contract. - Configure governance within Unity Catalog—including permissions, data lineage, and auditing—which are critical for client production data. - Orchestrate processes using Workflows/Jobs, incorporate data quality checks (such as DLT expectations or similar), and manage version control via Databricks Asset Bundles. Requirements: - Experience in data engineering, with at least 2 years specifically using Databricks (PySpark, Delta Lake, Unity Catalog, Workflows). - Advanced proficiency in SQL and Python. - Experience with data reconciliation or validation within migration projects. - Knowledge of SAP data structures. - Experience in developing reports based on system data. - Advanced English for conversation. #LI-TO1 Our benefits: -Health and dental insurance -Meal and food allowance -Childcare assistance -Extended paternity leave -Partnership with gyms and health and wellness professionals via Wellhub (Gympass) TotalPass; - Profit Sharing and Results Participation (PLR); -Life insurance - Continuous learning platform (CI&T University); -Discount club -Free online platform dedicated to physical, mental, and overall well-being -Pregnancy and responsible parenting course -Partnerships with online learning platforms -Language learning platform And many more! More details about our benefits here: https://ciandt.com/br/pt-br/carreiras At CI&T, inclusion starts at the first contact. If you are a person with a disability, it is important to present your assessment during the selection process. See which data needs to be included in the report by clicking here . This way, we can ensure the support and accommodations that you deserve. If you do not yet have the assessment, don't worry: we can support you in obtaining it. We have a dedicated Health and Well-being team, inclusion specialists, and affinity groups who will be with you at every stage. Count on us to make this journey side by side.