Databricks Consultant

Senior Databricks Consultant (Data Engineering / ML) | Data & AI Consulting | Sydney (Hybrid)
 
Lumiq Talent is partnering with two respected Data & AI consultancies in Sydney, both delivering modern Databricks platforms for some of Australia's largest enterprise brands. Each is growing its Databricks practice and looking for experienced engineers who enjoy client-facing work and want real ownership of what they build.
One conversation with us gives you access to both opportunities. One role leans towards platform and data engineering, the other bridges data engineering and machine learning, and we'll help you work out which fits you best.
 
The role
 
You'll design, build and optimise enterprise Lakehouse platforms on Databricks, turning large, complex datasets into reliable foundations for analytics and AI. This is a consultative role, so you'll be working directly with clients: running workshops, shaping data strategy and acting as a trusted technical advisor rather than sitting in a back office.
 
What you'll be doing

• Architecting and building scalable, secure Lakehouse platforms using Delta Lake and Unity Catalog
• Developing batch and streaming pipelines with PySpark, Structured Streaming, Auto Loader and Spark Declarative Pipelines
• Migrating legacy data warehouses and Hadoop estates onto Databricks
• Working with data scientists and ML engineers to operationalise models using MLflow and Databricks Model Serving
• Tuning performance and cost across large-scale client environments
• Setting up governance and security in Unity Catalog, including access controls, lineage and row/column-level security
• Championing CI/CD, Infrastructure as Code and automated testing across data and ML workloads

What you'll bring

• Strong, hands-on commercial experience building on Databricks
• Advanced Spark and SQL, with strong Python (PySpark); Scala is a bonus
• Experience with Delta Lake, Unity Catalog, Databricks Workflows and tools such as Airflow, dbt, Kafka or Fivetran
• A solid understanding of Medallion Architecture and when it adds value
• Working knowledge of at least one major cloud (Azure, AWS or GCP)
• A track record of shipping production pipelines, with good instincts for cost and performance
• Clear communication skills and the confidence to explain technical trade-offs to non-technical stakeholders

Nice to have

• Experience operationalising ML models and working across the ML lifecycle
• Databricks certifications such as Data Engineer Professional or Machine Learning Associate/Professional (if you're not certified yet, both firms will support you to get there)
• Previous experience in a consultancy or client-facing environment

Why these teams
 
Both are boutique, collaborative consultancies with little hierarchy and a genuine focus on their people. You'll have the autonomy to solve complex problems, strong partner ecosystems across Databricks and the wider modern data stack, funded certifications, and competitive packages with benefits that support life outside work.
 
Interested?
Apply through this advert for a confidential conversation. Even if you're only casually exploring, we're happy to share more about both teams.